How to prevent crawling of non-public ACSF domains
How to prevent crawling of non-public ACSF domains | Site Factory | Acquia Product Documentation
How to prevent crawling of non-public ACSF domains
Date Published: March 11, 2022
Until a platform-wide solution is in place, the best practices approach is to add the following code to your codebase:
In the `.htaccess` file in the docroot, add
# Ensure non-live sites prevent robots from crawling. This is similar to what
# Acquia Cloud does for the *.acquia-sites.com domains.
RewriteCond %{HTTP_HOST} \.acsitefactory\.com$ [NC]
RewriteCond %{REQUEST_URI} /robots.txt [NC]
RewriteRule ^ no_robots.txt [L]
And add a file called `no_robots.txt` to the docroot with the following contents:
#
# no_robots.txt
#
# This file prevents indexing of domains in the acsitefactory.com DNS zone
#
# See also: https://docs.acquia.com/cloud-platform/manage/domains/#crawling-is-blocked-for-acquia-sites-com-domains
#
User-agent: *
Disallow: /
Site Factory