Scan troubleshooting checklist¶
Use this checklist and ensure that you have tried these solutions before you contact support.
Allow-list the Optimize crawler throughout your hosting setup¶
Remember to include your CMS, WAF, security tools, CDNs, and any others that you use.
IP addresses:¶
Whitelist the IP address for your region:
- USA and Canada
- UK and Europe
- Australia, New Zealand, and Japan
For more information, see the user guide articles:
Make sure the Optimize crawler user agents are allow-listed¶
Primary user agent: Mozilla/5.0 (compatible; Monsidobot/2.2; +http://http://monsido.com/bot.html ; [email protected]).
Alternative user agent (we only use this if the first one fails): Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36
Ensure that your hosting infrastructure is able to process the amount of requests generated by the Optimize Crawler¶
It is possible to set a slower scan speed in the Optimize UI:
- Slow: 15 connections per minute.
- Normal (recommended): 60 connections per minute.
- Faster: 90 connections per minute.
- Very fast (be cautious): 120 connections per minute.
- Superfast (advanced users only) 180 connections per minute.
For full instructions on how to set up the scan and adjust the speed, see the user guide article:
Domain scan setup.
Check if you have any password-protected or internal-only pages.¶
The scan cannot proceed if you have not granted access to pages that require a password. For instructions on how to set this up, see the user guide article:
Ensure that your CMS does not prevent crawlers/bots from accessing a page
In some CMS applications, it is possible to prevent one or more pages from being indexed by search engines. Such settings may be called “Hide from external search engines”, “Don’t index this page”, or similar. Be aware, that if such settings are turned on for a page, the Optimize crawler may not be allowed by the CMS to access and scan that particular page.
Additional resources¶
For more information, see the following articles:
Some reasons why the scan might stop running¶
See the following articles for answers to some more frequently-asked questions about scan progress:
If you have tried all of the above and scans are still not running as expected, contact the support team by email or use the Live Support Chat feature inside the application on the main menu toolbar.