Scan troubleshooting checklist
Use this checklist and ensure that you have tried these solutions before you contact support.
Allow-list the crawler throughout your hosting setup
Remember to include your CMS, WAF, security tools, CDNs, and any others that you use.
For information about the IP addresses including those for Heartbeat, the Ports used, and User agents, visit Data hosting and security.
Ensure that your hosting infrastructure is able to process the amount of requests generated by the crawler
It is possible to set a slower scan speed from within UI:
- Slow: 15 connections per minute.
- Normal (recommended): 60 connections per minute.
- Faster: 90 connections per minute.
- Very fast (be cautious): 120 connections per minute.
- Superfast (advanced users only) 180 connections per minute.
Notes
- Some CMS providers have implemented scan rate limiting on their users. Visit this article for more information on why you should consider reducing the connection speed: CMS rate limiting on bots and crawlers.
- The server answer time may be adversely affected if the connections per minute setting is Faster or higher. At these settings, the speed of the scan may not change if the server is slow to respond.
- The crawler can do a maximum of five scans per IP address simultaneously. When several websites use the same IP, the scan uses the slowest connections per minute setting from the IP address group.
For full instructions for admin users on how to set up the scan and adjust the speed, visit Domain scan setup.
Check if you have any password-protected or internal-only pages.
The scan cannot proceed if you have not granted access to pages that require a password. For instructions on how to set this up, visit:
Ensure that your CMS does not prevent access to crawlers or bots
In some CMS applications, it is possible to prevent one or more pages from being indexed by search engines. Such settings may be called “Hide from external search engines”, “Don’t index this page”, or similar. Be aware that if such settings are turned on for a page, the CMS may prevent access to the Web Governance crawler and the page will not be scanned.
Additional resources
For more information, visit:
If you have tried all of the above and scans still do not run as expected, contact the support team by email or use the Live Support Chat feature inside the application on the main menu toolbar.
Scan troubleshooting checklist
Scan troubleshooting checklist
Use this checklist and ensure that you have tried these solutions before you contact support.
Allow-list the crawler throughout your hosting setup
Remember to include your CMS, WAF, security tools, CDNs, and any others that you use.
For information about the IP addresses including those for Heartbeat, the Ports used, and User agents, visit Data hosting and security.
Ensure that your hosting infrastructure is able to process the amount of requests generated by the crawler
It is possible to set a slower scan speed from within UI:
- Slow: 15 connections per minute.
- Normal (recommended): 60 connections per minute.
- Faster: 90 connections per minute.
- Very fast (be cautious): 120 connections per minute.
- Superfast (advanced users only) 180 connections per minute.
Notes
- Some CMS providers have implemented scan rate limiting on their users. Visit this article for more information on why you should consider reducing the connection speed: CMS rate limiting on bots and crawlers.
- The server answer time may be adversely affected if the connections per minute setting is Faster or higher. At these settings, the speed of the scan may not change if the server is slow to respond.
- The crawler can do a maximum of five scans per IP address simultaneously. When several websites use the same IP, the scan uses the slowest connections per minute setting from the IP address group.
For full instructions for admin users on how to set up the scan and adjust the speed, visit Domain scan setup.
Check if you have any password-protected or internal-only pages.
The scan cannot proceed if you have not granted access to pages that require a password. For instructions on how to set this up, visit:
Ensure that your CMS does not prevent access to crawlers or bots
In some CMS applications, it is possible to prevent one or more pages from being indexed by search engines. Such settings may be called “Hide from external search engines”, “Don’t index this page”, or similar. Be aware that if such settings are turned on for a page, the CMS may prevent access to the Web Governance crawler and the page will not be scanned.
Additional resources
For more information, visit:
If you have tried all of the above and scans still do not run as expected, contact the support team by email or use the Live Support Chat feature inside the application on the main menu toolbar.
Web Governance