This article provides instructions and examples for how to set up path constraints and link exclusions for the scan.
Use path constraints to process all pages that begin with the parameter you set. If you plan to do the scan on a path-based domain, you must set up path constraints because the crawler processes everything it can find on the domain.
Use link exclusions to exclude a word or a Regular Expression (regex). This instructs the crawler to completely ignore a link on the pages. Pages that match the pattern are not tested. Exclude parameterized or duplicate URLs to help prevent inflated page counts and ensure that the scan results accurately reflect the unique content on your site.
Path constraints
This section provides instructions and examples on how to set up path constraints.
Notes
Path constraints are regexes.
A regex is an API that defines string patterns. Use a regex to search, manipulate, and edit a string in Java. Email validation and passwords are two examples of strings where a regex can define the constraints.
Regexes in source code exclusions are not 100% compatible with the
Policies
module. The languages are different (Java and Ruby).
Instructions
Log in to Web Governance. The system displays the Domain Overview.
Click Action on the same row as the domain that you want to work with and select Edit Domain.
Scroll to the Advanced Domain Options section.
Complete the requested information in the Path Constraints section:
Constraint pattern: Enter a constraint pattern. See the Examples section below for an example of a constraint pattern.
To add more, click +Add.
Notes
The window only shows the first five items. With more than five list items, a paginate function sorts the list items. Use the Search feature to filter for specific constraints.
Some CMS selections have pre-filled parameters, as per the majority of users with those CMS operators. Click Default to remove them if needed. The Default button only appears for certain operators.
Examples
Use Path Constraints to instruct the scan to only process parts of a domain. The scan processes URLs that match the pattern, and ignores URLs that do not match the pattern.
A Path Constraint can consist of a word or a regex. In most cases, users set up Path Constraints to restrict the scanner to only recognize parts of a site with a specific pattern.
For example: /en/
This constraint instructs the scan to handle any URL that does not begin with /en (such as, http://foo(dot)com/fr/bar) as an external link. For example, the crawler tests the link to verify that it is operational, but does not follow any links on http://foo(dot)com/fr/bar.
Important
Make sure that the URL for the domain is set to a page that matches the constraint. If this is not done, only one page will be included in the scan, since the crawler cannot proceed to any other page than the page it starts on.
For example, with a constraint of: ^/en/booking, the crawler cannot start on http://foo(dot)com. The crawler will request http://foo(dot)com, receive the page, and find that no links match http://foo(dot)com/en/booking. Because of this, only the first page is scanned.
Notes
Regexes in path constraints are not 100% compatible with the Policies module. The languages are different (Java and Ruby).
This section provides instructions and examples on how to set up the link exclusions. Link exclusions are regexes.
Notes
A regex is an API to define String patterns. Use a regex to search, manipulate, and edit a string in Java. Email validation and passwords are two examples of strings where Regex can define the constraints.
Regexes in link exclusions are not 100% compatible with the Policies module. The languages are different (Java and Ruby).
Choose to exclude a word or a regex. Use a Link Exclusion to instruct the crawler to completely ignore the link. Pages that match the pattern are not included in the scan. Exclude parameterized or duplicate URLs to help prevent inflated page counts and ensure that the scan results accurately reflect the unique content on your site.
Use link exclusions to:
Filter out print pages with a pattern such as print=true. This instructs the scan to ignore (and not test) any URL with the pattern, for example http:/ /foo.com/bar?print=true.
Filter out redirected login pages with a pattern such as http://foo(dot)com/bar/login.aspx?return_url=zyx.</i. This instructs the scan to ignore all URLs with the pattern, for example: http://foo(dot)com/bar/login.aspx?return_url=zyx.
Exclude parameterized URLs to avoid inflated page counts. For example, add a pattern like: posts_per_page= to instruct the scan to exclude URLs such as http://foo.com/bar?posts_per_page=10.>
Tip! If "Scan subdomains" is turned on for the domain, use the § sign in front of the exclude pattern to match URLs that use the full string instead of the relative one. For example, to exclude the scan for the "blog" subdomain, enter as the pattern: §http://blog.foo.bar
Examples
This section provides a few examples on how you can use a link exclusion to instruct the scan to ignore parts of the site with a particular pattern.
Use link exclusions to:
Filter out print pages with a pattern such as print=true. This instructs the scan to ignore (and not test) any URL with the pattern, for example http:/ /foo.com/bar?print=true.
Filter out redirected login pages with a pattern such as http://foo(dot)com/bar/login.aspx?return_url=zyx.</i. This instructs the scan to ignore all URLs with the pattern, for example: http://foo(dot)com/bar/login.aspx?return_url=zyx.
Exclude parameterized URLs to avoid inflated page counts. For example, add a pattern like: posts_per_page= to instruct the scan to exclude URLs such as http://foo.com/bar?posts_per_page=10.>
Tip! If "Scan subdomains" is turned on for the domain, use the § sign in front of the exclude pattern to match URLs that use the full string instead of the relative one. For example, to exclude the scan for the "blog" subdomain, enter as the pattern: §http://blog.foo.bar
For example: /fr/
This instructs the scan to process any URLs that begin with /fr (for example, http://foo(dot)com/fr/bar) as an external link.
This means that the scan tests the link but does not follow any links on http://foo(dot)com/fr/bar.
The scan processes all pages EXCEPT for those that contain /fr.
Instructions
There are two methods to use for setting up link exclusions in Web Governance.
Method 1: Through the domain Action button.
Method 2: Through Admin settings.
Method 1
Log in to Web Governance and locate your domain on the Domain Overview page.
On the same row as the domain that you want to work with, click Action and select Edit Domainfrom the drop-down list.
Scroll to the Advanced Domain Options section.
Complete the requested information in the Link excludes section:
Exclude pattern: Enter a pattern to exclude from the scan.
Notes
The window only shows the first five items. With more than five list items, a paginate function begins to sort consecutive list items.
Search: Enter a search parameter for matching strings within the list.
This article provides instructions and examples for how to set up path constraints and link exclusions for the scan.
Use path constraints to process all pages that begin with the parameter you set. If you plan to do the scan on a path-based domain, you must set up path constraints because the crawler processes everything it can find on the domain.
Use link exclusions to exclude a word or a Regular Expression (regex). This instructs the crawler to completely ignore a link on the pages. Pages that match the pattern are not tested. Exclude parameterized or duplicate URLs to help prevent inflated page counts and ensure that the scan results accurately reflect the unique content on your site.
Path constraints
This section provides instructions and examples on how to set up path constraints.
Notes
Path constraints are regexes.
A regex is an API that defines string patterns. Use a regex to search, manipulate, and edit a string in Java. Email validation and passwords are two examples of strings where a regex can define the constraints.
Regexes in source code exclusions are not 100% compatible with the
Policies
module. The languages are different (Java and Ruby).
Instructions
Log in to Web Governance. The system displays the Domain Overview.
Click Action on the same row as the domain that you want to work with and select Edit Domain.
Scroll to the Advanced Domain Options section.
Complete the requested information in the Path Constraints section:
Constraint pattern: Enter a constraint pattern. See the Examples section below for an example of a constraint pattern.
To add more, click +Add.
Notes
The window only shows the first five items. With more than five list items, a paginate function sorts the list items. Use the Search feature to filter for specific constraints.
Some CMS selections have pre-filled parameters, as per the majority of users with those CMS operators. Click Default to remove them if needed. The Default button only appears for certain operators.
Examples
Use Path Constraints to instruct the scan to only process parts of a domain. The scan processes URLs that match the pattern, and ignores URLs that do not match the pattern.
A Path Constraint can consist of a word or a regex. In most cases, users set up Path Constraints to restrict the scanner to only recognize parts of a site with a specific pattern.
For example: /en/
This constraint instructs the scan to handle any URL that does not begin with /en (such as, http://foo(dot)com/fr/bar) as an external link. For example, the crawler tests the link to verify that it is operational, but does not follow any links on http://foo(dot)com/fr/bar.
Important
Make sure that the URL for the domain is set to a page that matches the constraint. If this is not done, only one page will be included in the scan, since the crawler cannot proceed to any other page than the page it starts on.
For example, with a constraint of: ^/en/booking, the crawler cannot start on http://foo(dot)com. The crawler will request http://foo(dot)com, receive the page, and find that no links match http://foo(dot)com/en/booking. Because of this, only the first page is scanned.
Notes
Regexes in path constraints are not 100% compatible with the Policies module. The languages are different (Java and Ruby).
This section provides instructions and examples on how to set up the link exclusions. Link exclusions are regexes.
Notes
A regex is an API to define String patterns. Use a regex to search, manipulate, and edit a string in Java. Email validation and passwords are two examples of strings where Regex can define the constraints.
Regexes in link exclusions are not 100% compatible with the Policies module. The languages are different (Java and Ruby).
Choose to exclude a word or a regex. Use a Link Exclusion to instruct the crawler to completely ignore the link. Pages that match the pattern are not included in the scan. Exclude parameterized or duplicate URLs to help prevent inflated page counts and ensure that the scan results accurately reflect the unique content on your site.
Use link exclusions to:
Filter out print pages with a pattern such as print=true. This instructs the scan to ignore (and not test) any URL with the pattern, for example http:/ /foo.com/bar?print=true.
Filter out redirected login pages with a pattern such as http://foo(dot)com/bar/login.aspx?return_url=zyx.</i. This instructs the scan to ignore all URLs with the pattern, for example: http://foo(dot)com/bar/login.aspx?return_url=zyx.
Exclude parameterized URLs to avoid inflated page counts. For example, add a pattern like: posts_per_page= to instruct the scan to exclude URLs such as http://foo.com/bar?posts_per_page=10.>
Tip! If "Scan subdomains" is turned on for the domain, use the § sign in front of the exclude pattern to match URLs that use the full string instead of the relative one. For example, to exclude the scan for the "blog" subdomain, enter as the pattern: §http://blog.foo.bar
Examples
This section provides a few examples on how you can use a link exclusion to instruct the scan to ignore parts of the site with a particular pattern.
Use link exclusions to:
Filter out print pages with a pattern such as print=true. This instructs the scan to ignore (and not test) any URL with the pattern, for example http:/ /foo.com/bar?print=true.
Filter out redirected login pages with a pattern such as http://foo(dot)com/bar/login.aspx?return_url=zyx.</i. This instructs the scan to ignore all URLs with the pattern, for example: http://foo(dot)com/bar/login.aspx?return_url=zyx.
Exclude parameterized URLs to avoid inflated page counts. For example, add a pattern like: posts_per_page= to instruct the scan to exclude URLs such as http://foo.com/bar?posts_per_page=10.>
Tip! If "Scan subdomains" is turned on for the domain, use the § sign in front of the exclude pattern to match URLs that use the full string instead of the relative one. For example, to exclude the scan for the "blog" subdomain, enter as the pattern: §http://blog.foo.bar
For example: /fr/
This instructs the scan to process any URLs that begin with /fr (for example, http://foo(dot)com/fr/bar) as an external link.
This means that the scan tests the link but does not follow any links on http://foo(dot)com/fr/bar.
The scan processes all pages EXCEPT for those that contain /fr.
Instructions
There are two methods to use for setting up link exclusions in Web Governance.
Method 1: Through the domain Action button.
Method 2: Through Admin settings.
Method 1
Log in to Web Governance and locate your domain on the Domain Overview page.
On the same row as the domain that you want to work with, click Action and select Edit Domainfrom the drop-down list.
Scroll to the Advanced Domain Options section.
Complete the requested information in the Link excludes section:
Exclude pattern: Enter a pattern to exclude from the scan.
Notes
The window only shows the first five items. With more than five list items, a paginate function begins to sort consecutive list items.
Search: Enter a search parameter for matching strings within the list.
Operator: Click the drop-down arrow to select Contains, Starts with, or Regex.
URL: Enter a URL in the field.
+ Add: Click to add a new Input Selector row if needed.
Important
It is possible to do a link exclusion for a link that is attached to an image. The link is then excluded from the scan. However, the image itself could still appear on the SEO and QA pages as an issue to be fixed if it does not meet other requirements (for example, missing ALT text).
Misspellings
Language validation
Readability
Did not find what you were looking for?
If this content did not answer your questions, try searching or contacting our support team for further assistance.
Link excludes
Internal URLs:
Operator: Click the drop-down arrow to select Contains, Starts with, or Regex.
URL: Enter a URL in the field.
+ Add: Click to add a new Input Selector row if needed.
Important
It is possible to do a link exclusion for a link that is attached to an image. The link is then excluded from the scan. However, the image itself could still appear on the SEO and QA pages as an issue to be fixed if it does not meet other requirements (for example, missing ALT text).
Misspellings
Language validation
Readability
Did not find what you were looking for?
If this content did not answer your questions, try searching or contacting our support team for further assistance.
Web Governance