Introduction¶

This article gives instructions on how to set up and configure a domain scan.

IP Addresses that we use¶

• USA and Canada 35.226.117.128
• UK and Europe 130.211.65.55
• Australia and New Zealand 35.189.0.46

Ports that we use¶

• 443 for HTTPS
• 80 for HTTP.

The User Agent¶

The user agent details are:

• Mozilla/5.0 (compatible; Monsidobot/2.2; +https://monsido.com/bot.html; [email protected]).

Instructions¶

This section gives instructions on how to configure a recurring domain scan.

1. Click Admin Settings (gear icon) on the toolbar at the top of the Domain Overview page.

   

   The Domain Settings page opens.

2. Click Action on the same row as a domain.

   

   The Action menu expands.

   
3. Select Edit Domain.
4. Fill out the requested information in the following sections.

For more information, see the user guide article:

Add, edit, or remove a domain

Domain details¶

This section gives instructions on how to fill out the Domain Details section.

• Title: Enter a title for the domain.

• URL: Enter the URL for the domain.

  Note

  The scan also includes all pages that are linked to the URL.

• CMS: Click the drop-down arrow and select the CMS.
• Note

  Some CMS providers have implemented scan rate limiting on their users.

  For more information, see:

  CMS rate limiting on bots and crawlers (FAQ)

  If the CMS is not on the list, contact the support team to see if it is possible for our team to add it.

CMS URL: Click in the text field and type in the base URL for the CMS page. This field is required.

Language: Click the drop-down arrow and select the language for the domain scan. Select the primary language for the domain.

For advanced help files, see:

CMS Integration 

Configure scans¶

This section gives instructions on how to fill out the Scan section.

Crawl Automatically: Toggle the switch to ON to set up a schedule for recurring scans. Toggle the switch to OFF to scan manually when needed.

• Scan Frequency: Click the drop-down arrow and select the scan frequency.
  • Choose:
    • Weekly
    • Every 2 weeks
    • Every 4 weeks
    • Every 8 weeks
    • Every 12 weeks.
  • Note

     This is an add-on feature. If this option is not available and you wish to change your scan frequency, contact your sales representative to upgrade the account. The default scan frequency is weekly.

• Day: click the drop-down arrow and select the day of the week for the recurring scan.
• Time: Click in the number field and enter the time of day (12-hour clock) and then choose AM for 00:00-11:59 or PM for 12:00 to 23:59.

Connections per minute: Click the drop-down arrow and select Connections per minute. The choices are:

• Slow: 15 connections per minute.
• Normal (recommended): 60 connections per minute.
• Faster: 90 connections per minute.
• Very fast (be cautious): 120 connections per minute.
• Superfast (advanced users only) 180 connections per minute.

Max¶

Max scanned pages: Enter the maximum number of pages to do a scan on, as specified on your Acquia contract.

Crawl (Scan) Options¶

This section gives instructions on how to fill out the Crawl (scan) Options section.

Toggle the switches ON to activate the different scan options.

• Scan subdomains: Toggle the switch ON to include subdomains in the scan.

• Spelling ignore capitalized words: Toggle the switch ON to ignore words that begin with a capital letter.

  Note

  Words that the user includes in the Optimize dictionary are case-sensitive. To instruct the scan to ignore words that are written in all caps, and bypass the need to, for example, add the word to the dictionary, toggle this switch to ON.

• Case sensitive URLs: Toggle the switch ON to include case-sensitive URLs in the scan.

  Note

  This is set to the optimal position by default.

  Example OFF: http://monsido.com/Foo/Bar is seen as the same link as: 
  http://monsido.com/foo/bar.

  Example ON: The same links above will register as two different links.

  Note

  In the OFF position, the scan sees all URLs as lowercase - no matter how they are written on the target website. Turn OFF only in rare cases where the target website does not recognize caps and reads all links in lowercase, which results in duplicates in the flagged scan issues. This is an advanced configuration that should normally be left to the default setting.

  A better solution is to disable the case-sensitive feature and use only regex.

• Render pages and execute JS while crawling: Toggle the switch ON to render pages and execute JavaScript during the scan.

  When this option is turned ON, the website renders pages and executes JavaScript during the scan. This allows the scan to take into account any changes caused by Javascript during a website scan.

  Note

  If the Accessibility feature is enabled, changes caused by JavaScript are automatically accounted for by default.

• Mark 403 as broken link: Toggle the switch ON to mark links on pages that return error 403 (forbidden errors) as a broken link.

• Ignore canonical URLs: Toggle the switch ON to instruct the scan to ignore canonical URLs.

  More info: A canonical URL is the URL of the page that Google determines is most representative, taken from a set of duplicate pages on your website.

• Use language attribute: Turn the switch OFF or ON. When it is ON, the scan does not try to auto-detect the page language. This means that the spell check is only done for the language specified in the lang attribute.

  The Use Lang Attribute feature allows the scanner to use the correct spell check for HTML pages that have multiple languages.

  Note

  The primary language of the HTML page must be correctly defined as a lang attribute on the <html> tag. Additionally, each subsection of the page that is in any other language must be wrapped in a tag with a proper lang attribute value.
  See below for an example:

  • copy

      <html lang=en">
        <head>
          <title>Hello World in other languages</title>
        </head>
        <body>
          <h1>Hello World in other languages</h1>
          <p>
            Spanish: <span lang="es">Hola Mundo</span><br>
            Danish: <span lang="da">Hej verden</span><br>
            French: <span lang="fr">Bonjour le monde</span>
          </p>
        </body>
      </html>

Remember: When enabled, the scan does not auto-detect the page language. For example, spell checks are always for the language specified in the lang attribute.

Statistics feature options¶

Some websites - including CP-built sites - have URLs that use a mix of lowercase and uppercase letters, for example https://www.example.com/TestPage

While the crawler saves the URL as all lowercase in our database, our Statistics module provider Matomo does not follow that practice and saves it as it is written by the website. This results in a scenario where the scan cannot map page statistics for the page https://www.example.com/TestPage to the page in our database because it was saved as https://www.example.com/testpage.

This option forces Matomo to put all URLs in lowercase so that they match the entries in our database.

• Convert all URLs to lower case: Toggle the switch ON to convert all URL addresses to lowercase.

Advanced domain options¶

This section gives instructions on how to fill out the Advanced Domain Options section.
For more information, see the user guide article:

Path Constraints and Link Exclusions

• Path Constraints:¶

  • Search: Enter a search parameter for matching strings within the Constraint Patterns list.
  • Constraint pattern: Enter a constraint pattern.

  • + Add: Click + to add a new Constraint pattern. An empty row is added to the list.

    Note

    The window only shows the first five items. With more than five list items, a paginate function begins to sort consecutive list items.

  • Delete: Click the trashcan icon to delete an item from the list.

• Link Excludes¶

  • Search: Enter a search parameter for matching strings within the Link excludes list.
  • Exclude pattern: Enter a pattern to exclude from the scan.

• + Add: Click + to add a new Exclude pattern. An empty row appears in the list.
• Delete: Click the trashcan icon to delete an item from the list.
• Internal URLs:
  • Operator: Click the drop-down arrow to select Contains, Starts with, or Regex.
  • Url: Type a URL in the field.
• Delete: Click the trashcan icon to delete the row.
• + Add: Click to add a new Input Selector. An empty row appears in the list.

For more information, see the user guide articles:

• Path constraints and link exclusions
• CMS rate limiting on bots and crawlers (FAQ)

Login Type¶

This section gives instructions on how to fill out the Login Type section.

Click the drop-down arrow and select a login type. The choices are:

• None: The site does not require user logins.
• Form: The field expands. The expanded input fields specify the CSS selectors that identify the various form fields and buttons. The following input fields open:
  • Input selector: (Required field) Enter the CSS ID or class selector to search for.
  • Default Value: Click to hide or show the input text. Click in the dialog box to set a default value for the selector.
  • Delete: Click the trashcan icon to delete the row.
  • + Add: Click to add a new Form. An empty row appears in the list.
  • Submit selector: (Required field) Click in the dialog box and type in a string (without spaces) to identify the submit button for the form.
  • Verify URL: (Required field) Click in the dialog box and type in a URL address that the user will redirect to if the form is valid.
• Office 365 (BETA): Beta - still under development. This field allows information for Office 365 login.
  • Email: Click in the dialog box and type in a valid Office 365 email address.
  • Password: Click in the dialog box and type in the Office 35 password for the email account.
  • Remember me: Turn the switch ON to remember user credentials.
• Basic Auth:
  • Username: Click in the input field and type in a username field identifier.
  • Password: Click in the input field and type in a password field identifier.
  • Hosts: Click in the input field and type in the URL to the host website.
• Custom Multistep: This option covers login flows that require special integration. Contact the support team for assistance.
  • Verify URL: Click in the dialog box and type in a URL address for the user to redirect to if the login is valid.
  • Username: Click in the input field and type in a username field identifier.
  • Password: Click in the input field and type in a password field identifier.
  • JS: This is a field to enter JavaScript strings for the scan to identify.
  • Delete: Click the trash can icon to delete the row.
  • + Add: Click to add a new Custom Multistep. An empty row appears in the list.

Features¶

This section gives instructions on how to fill out the Features section.

• Accessibility: Click the drop-down arrow and select the accessibility level that is desired or required for the website.

  Important

  When you change the accessibility level, the existing Accessibility Score scan information is erased and the new standard starts over from the beginning with the next scan. All of the history is lost. It is advised to activate a new level only when you are ready to make the updates needed, or if it becomes a legal requirement for your location/organization.
  If you change the scan to Exclude checks that require manual review, you might not see other potential accessibility violations on your website.

  Note

  Level AA of WCAG 2.1 is the legal requirement to be considered accessible. WCAG 2.2 is newly released and will soon be the standard, taking over from WCAG 2.1.

• Source code excludes: Click the drop-down arrow and select domains to exclude from the scan.

  For more information, see the user guide articles:

  • Configure source code exclusions on a scan
  • Configure source code exclusions with CSS selectors
• PDF accessibility: Click the drop-down arrow and select the level of accessibility for the PDF pages included in the scan. The choices are WCAG 2.0 and WCAG 2.1.
• Readability: Click the drop-down arrow and select a readability standard to determine the level of difficulty of the website text, to ensure that the correct audience is targeted. The choices are Flesch Kincaid and LIX.
   
• Note

  • The two types of readability tests that we use are Flesch Kincaid for English and LIX for foreign languages.

    • The Flesch–Kincaid readability test for English text measures word length, number of syllables per word, and sentence length. The formula for the Flesch readability score (FRES) test is as follows:

      206.835 - 1.015 (total words/total sentences) - 84.6 (total syllables/total words)

    • The LIX method is used for foreign languages and the readability formula is as follows:
      ​LIX = A/B + (C x 100)/A, where
      ​A = Number of words
      ​B = Number of periods (defined by period, colon, or capital first letter)
      ​C = Number of long words (More than 6 letters)

      For more information, see the user guide article:

      Quality Assurance: Readability

• Minimum words: Enter the minimum number of words that a page should contain for the readability score to begin calculating. Pages that have fewer words than indicated are not included in the readability score.

• Statistics: Toggle ON to turn on this feature and begin to track website visitor statistics.

  For more information, see the user guide article:

  Statistics.

• PageCorrect: Toggle ON to turn on this add-on. Click the link to navigate to the PageCorrect Settings page.

  PageCorrect allows users to identify and correct website errors such as quality assurance errors and accessibility issues.

  For more information, see the user guide article:

  PageCorrect

• Data Privacy: Toggle the switch OFF or ON. Click the link to navigate to the Domain Data Privacy Settings page.

  The Data Privacy module identifies potential data privacy issues. It can serve as a form of documentation that allows website owners to show their efforts to be compliant with local regulations.

  For more information, see the user guide article:

  Data privacy

• Scan documents: Toggle the switch ON to include documents in the scan.

  The scan is a powerful tool that does a scan on all pages of a website. With Scan documents selected, we also do a scan of documents on the website. The document scan identifies all of the same types of issues as a domain scan.

• PageAssist: Toggle the switch ON to activate. Click the link to navigate to the PageAssist Settings page.

  PageAssist allows website owners to make custom visual adjustments to suit their unique requirements.

  For more information, see the user guide article:

  PageAssist

• Consent Manager: Toggle the switch ON to activate.

  The Consent Manager module includes everything that you need to classify and set up user cookie acceptance and website cookie banners. This module also includes automatic cookie scan detection and cookie maintenance.

  For more information, see the user guide article:

  Consent Manager

Domain Groups¶

Set up groups to assign specific users to a subset of pages on one or more domains that the scan has been done on. Created groups appear under the domain name.

For more information, see the user guide articles:

• Create and configure groups

Users¶

The Users section shows a list of users with access to the Acquia Optimize interface. This section gives instructions on how to add permissions to users in the Users section.

The list is presented in table format.

The table column headers are:

• User: The Username for the user.
• Visible: Tick the box to make the control panel visible to the user.
• Send report: Tick the box to send reports to the user.

• Admin: Tick the box to give the user admin permissions. Admins can add, edit, and delete.

  For more information, see the user guide article:

  Create and configure users

Click Save. The settings pane closes and the Domains page is visible.

It is now possible to begin an on-demand scan to see the configuration results. Click Action on the same row as a domain and select Start on-demand scan.

Start an on-demand scan¶

1. Click Admin Settings (gear icon), on the main menu bar.

   

   The Domain Settings page opens.

   

2. On the same row as a domain, click Action.

   

   The Action menu expands.

   
3. Select Start on-demand scan.

Additional resources¶

For more information, see the articles:

• CMS Integration
• What is Monsidobot?
• How does the scan work on my site?