--- title: "How to create path constraints and link exclusions" date: "2021-12-16T11:58:59+00:00" summary: "Learn how to set up path constraints and link exclusions for efficient website scanning and crawling." image: type: "page" url: "/web-governance/how-create-path-constraints-and-link-exclusions" id: "d337ce00-1524-4fb0-b5cb-d891a3c53ff7" --- Table of contents will be added Introduction ------------ Use path constraints to instruct the scan to only process parts of a domain. The scan compares URLs in the site with the pattern given. URLs that match the pattern are scanned and URLs that do not match the pattern are considered external links. Use link exclusions to instruct the scan to ignore parts of the site with a pattern. For example, `/fr/`. The difference between the two is that in path constraints ALL pages under `/en` are scanned, and nothing else. In the link exclusions, all pages EXCEPT `/fr` are scanned. A Regular Expression (regex) is an API that defines string patterns. Use a regex to search, manipulate, and edit a string in Java. Email validation and passwords are two examples of strings where Regex can define the constraints.  Note Regular Expressions in source code exclusions are not 100% compatible with the Policies module. The languages are different (Java and Ruby). This document provides instructions on how to set up path constraints and link exclusions. Path constraints ---------------- A path constraint can consist of a word or a regular expression. In most cases, users set up path constraints to only crawl pages that contain the specfied parameter, and excludee any URL that does not contain it. Note Path constraints are required on path-based domains because without them, the crawler will crawl everything it can find on the domain. ### Create a path constraint 1. Log in to your Web Governance account. 2. Locate the domain that you want to work with. 3. Click **Action**. 4. Select **Edit Domain.** 5. Scroll to the **Advanced Domain Options** section. 6. In the Path Constraints section: * Search: Enter a search parameter for matching strings within the Constraint Patterns list. * Constraint pattern: Enter a constraint pattern. * \+ Add: Click + to add a new Constraint pattern. An empty row is added to the list. Note Some CMS selections have pre-filled parameters, as per the majority of users with those CMS operators. Click **Default** to remove them if needed. The Default button only appears for certain operators. For more information and examples, read [Class Patterns by Oracle](https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html) Important Make sure that the URL in the domain settings matches the path constraint. If this is not done, the crawl may go into retry, since the crawler starts on a page that conflicts with the constraint. For example, with a path constraint such as `/en/booking`, the crawler cannot begin on `http://foo.com`. The crawler will request `http://foo.com`, receive the page, and find that no links match `http://foo.com/en/booking`, and go into retry. ### Delete a path constraint 1. Log in to your Web Governance account. 2. Locate the domain that you want to work with. 3. Click **Action**. 4. Select **Edit Domain** 5. Scroll to the **Advanced Domain Options** section. 6. Locate the entry. 7. Click **Delete** (a trashcan icon) to delete it from the list. Link exclusions --------------- Use link exclusions to instruct the crawler to completely ignore certain pages or entire sections of the website. Pages that match the pattern are not included in the scan. Exclude parameterized or duplicate URLs to help prevent inflated page counts and ensure that the scan results accurately reflect the unique content on your site. Tip! If "Scan subdomains" is turned on for the domain, use the § sign in front of the exclude pattern to match URLs that use the full string instead of the relative one. For example, to exclude the scan for the "blog" subdomain, enter the pattern: `§http://blog.foo.bar`. ### Create a link exclusion 1. Log in to your Web Governance account. 2. Locate the domain that you want to work with. 3. Click **Action**. 4. Select **Edit Domain.** 5. Scroll to the **Advanced Domain Options** section. 6. In the _Link Excludes_ section: * Search: Enter a search parameter for matching strings within the Link excludes list. * Exclude pattern: Enter a pattern to exclude from the scan. * \+ Add: Click **+** to add a new Exclude pattern. An empty row appears in the list. * Delete: Click the trashcan icon to delete an item from the list. * Internal URLs: * Operator: Click the drop-down arrow to select Contains, Starts with, or Regex. * URL: Type a URL in the field. * Delete: Click the trashcan icon to delete the row. * \+ Add: Click to add a new Input Selector.  Important It is possible to do a link exclusion for a link that is attached to an image. The link is then excluded from the scan. However, the image itself could still appear on the SEO and QA pages as an issue to be fixed if it does not meet other requirements (for example, missing ALT text). For more information and examples, visit the external article [Class Patterns by Oracle](https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html). ### Delete a link exclusion 1. Log in to your Web Governance account. 2. Locate the domain that you want to work with. 3. Click **Action**. 4. Select **Edit Domain** 5. Scroll to the **Advanced Domain Options** section. 6. Locate the entry. 7. Click **Delete** (a trashcan icon) to delete it from the list. ### Examples * Filter out and exclude language-specific paths print pages with a pattern such as `/fr/`.  This instructs the scan to ignore (and not scan) any URL with the pattern, for example `http://foo.com/fr/`. * Filter out redirected login pages with a pattern such as `http://foo.com/bar/login`. This instructs the scan to ignore all URLs with the pattern, for example: `http://foo.com/bar/login`. Relies on regex. * Exclude parameterized URLs to avoid inflated page counts. For example, add a pattern like: posts\_per\_page= to instruct the scan to exclude URLs such as `http://foo.com/bar?posts_per_page=10`. Relies on regex. Additional resources -------------------- For more information, read [Configure Domain Scans](https://docs.acquia.com/acquia-optimize/admin/configure-scans/domain-scans). For advanced instructions on this topic, read [CMS Integration](https://docs.acquia.com/acquia-optimize/developer/cms-integration).