About Cloud Platform logging

About Cloud Platform logging

Frequently asked questions

The following logs are available through various methods, such as downloading or log forwarding:

The following logs available through various methods, such as streaming, download, or log forwarding:

Downloading active log files

To download active log files:

1. Sign in to the Cloud Platform user interface as a user with the download logs permission for the subscription and environment you want.
2. Select the application and environment for which you want to download a log.

3. Select Logs > Download.

   

4. Click the Download link for the log that you want to download.

   • If the log file is available for immediate download, the system begins downloading the file.
   • If the log file must be created on-demand, the system displays a dialog box with the status of the request:
   

5. After the log file is ready for download, click Download to retrieve it.

   The system downloads the log file in the following format:

   logfilename-timestamp.tar.gz

Downloading historical logs

You can access or download historical logs by using the following:

You can access or download historical logs by using the following: Acquia CLI, Cloud Platform user interface or Cloud API. You can use SSH, rsync, and SCP to download custom log files written in a persistent directory such as /shared/logs. These logs are not automatically rotated. Therefore, you must periodically prune this directory to ensure that it does not consume bulk of the storage allocated to your subscription. Downloading logs might take up to 30 seconds because of the remote log retention service, which ensures that logs persist when environments auto-scale. 

You can access or download historical logs by using the following: SSH, rsync, SCP, Cloud Platform user interface, or other tools. Log files on Cloud Classic environments don’t persist after an infrastructure is relaunched or resized.

The Logs page allows you to select a specific date and time range when attempting to download a log file. Logs are limited to 24 hour increments. 

Historical logs are stored in a location on the infrastructure that’s optimized for fast read/write activity. While this works for simultaneously updating several log files, the directory won’t persist after infrastructure is relaunched or resized. Log files do persist after infrastructure is rebooted. A relaunch can happen at any time. For example, in the event of infrastructure failure.

Location of log files

A limited subset of logs generated by custom activities, such as cron jobs leveraging Acquia’s cron-wrapper.sh script, can be found in the /shared/logs directory. This directory is persistent and maps to your file system. Also, it can be used for all custom log files. All the logs listed in the Available log files section are available for download through the Cloud Platform user interface.

A server maintains its own log files. For each server, the log file is located at /var/log/sites/[site].[env]/logs/[servername]/[logname].log. You can find your website name and server names on the Infrastructure page. For example, the Apache error log for the Dev environment of a website named myexample on a server named srv-25 is:

/var/log/sites/myexample.dev/logs/srv-25/error.log

Apart from Cloud Hooks logs and Scheduled Cron Jobs logs, all the logs listed in the Available log files section are available for download through the Cloud Platform user interface.

Archived logs

The infrastructure creates a new log file every day and compresses, archives, and saves old logs by date in the YYYYMMDD format:

• access.log-20140527.gz
• error.log-20140527.gz
• drupal-requests.log-20140527.gz
• drupal-watchdog-20140527.gz
• php-errors.log-20140527.gz

The system archives log files by file name, and won’t archive files in the logs directory if they don’t have the default names intact like those listed above.

Accessing logs using SSH

All the logs listed in the Available log files section cannot be accessed through SSH, scp and rsync. However, if you write your custom or cron logs in one of the following directories, you can access such logs through SSH, scp and rsync.

• /shared/logs
• $HOME
• any persistent directory

You can access log files on an infrastructure using SSH. For example, to view the Apache access log on the specified infrastructure:

less /var/log/sites/[site].[env]/logs/[SERVER_NAME]/access.log

Downloading logs

You can use the following methods to download logs:

Using Acquia CLI

To download the logs for any of your Cloud environment using Acquia CLI:

1. Check the list of log files available for your environment:

   copy

   acli api:environments:log-list AH_SITE_GROUP.AH_SITE_ENVIRONMENT

   Using jq:

   copy

   acli api:environments:log-list AH_SITE_GROUP.AH_SITE_ENVIRONMENT | jq '.[] | .name'

2. Initiate the process of generating a log file of the required type, for example, apache-access :

   copy

   acli api:environments:log-create AH_SITE_GROUP.AH_SITE_ENVIRONMENT apache-access --from="2024-01-20T00:00:00+00:00" --to="2024-01-20T23:59:59+00:00"

   The Log snapshots must cover no more than 24 hours and can only be retrieved up to 30 days in the past. The Log snapshots is based on the time zone that you have set on the Cloud UI. You can check that by going to Account > profile.

3. To confirm if the Task listed for log file creation has completed successfully, run the following command:

   copy

   acli api:applications:notification-list AH_SITE_GROUP

   If the status of the log creation task is shown is completed, that means the log file is generated, and now you can download it.

   Using jq:

   copy

   acli api:applications:notification-list AH_SITE_GROUP | jq '.[] | select (.label | contains ("Log file created"))'

4. After waiting for couple of minutes or once you have checked in Cloud Platform user interface that the task is completed, download the generated file:

   copy

   acli api:environments:log-download AH_SITE_GROUP.AH_SITE_ENVIRONMENT | jq apache-access

   The output of above command will give a "url", you need to click on the URL and open it in browser to start the download of log file.

   Using jq:

   copy

   curl $(acli api:environments:log-download AH_SITE_GROUP.AH_SITE_ENVIRONMENT apache-access | jq -r '.url') -o apache-log.gz
   

Additional (inaccessible) logs

Acquia uses additional logs from the logs mentioned earlier on this page that are not available to subscribers. These logs are inaccessible for several reasons, including the potential for security issues related to shared resources.

• Binary logs: Binary log (binlog) files are used to replicate data from the master database to a replica database. All statements that change data, including create, update, and delete statements, add lines to the MySQL binlogs. Queries that only read from the database, without changing the contents, don’t add lines to the binlogs. Staging environments, despite not having redundant databases, also use binlogs to keep feature and performance parity between non-production and production environments. For more information, see Binlogs.

• Email logs: Email logs are often a shared infrastructure resource, may have sensitive information, and aren’t available to subscribers.

Related topics

• Using logs
• Searching the error logs to troubleshoot problems

The following logs are available through various methods, such as downloading or log forwarding:

The following logs available through various methods, such as streaming, download, or log forwarding:

Downloading active log files

To download active log files:

1. Sign in to the Cloud Platform user interface as a user with the download logs permission for the subscription and environment you want.
2. Select the application and environment for which you want to download a log.

3. Select Logs > Download.

   

4. Click the Download link for the log that you want to download.

   • If the log file is available for immediate download, the system begins downloading the file.
   • If the log file must be created on-demand, the system displays a dialog box with the status of the request:
   

5. After the log file is ready for download, click Download to retrieve it.

   The system downloads the log file in the following format:

   logfilename-timestamp.tar.gz

Downloading historical logs

You can access or download historical logs by using the following:

You can access or download historical logs by using the following: Acquia CLI, Cloud Platform user interface or Cloud API. You can use SSH, rsync, and SCP to download custom log files written in a persistent directory such as /shared/logs. These logs are not automatically rotated. Therefore, you must periodically prune this directory to ensure that it does not consume bulk of the storage allocated to your subscription. Downloading logs might take up to 30 seconds because of the remote log retention service, which ensures that logs persist when environments auto-scale. 

You can access or download historical logs by using the following: SSH, rsync, SCP, Cloud Platform user interface, or other tools. Log files on Cloud Classic environments don’t persist after an infrastructure is relaunched or resized.

The Logs page allows you to select a specific date and time range when attempting to download a log file. Logs are limited to 24 hour increments. 

Historical logs are stored in a location on the infrastructure that’s optimized for fast read/write activity. While this works for simultaneously updating several log files, the directory won’t persist after infrastructure is relaunched or resized. Log files do persist after infrastructure is rebooted. A relaunch can happen at any time. For example, in the event of infrastructure failure.

Location of log files

A limited subset of logs generated by custom activities, such as cron jobs leveraging Acquia’s cron-wrapper.sh script, can be found in the /shared/logs directory. This directory is persistent and maps to your file system. Also, it can be used for all custom log files. All the logs listed in the Available log files section are available for download through the Cloud Platform user interface.

A server maintains its own log files. For each server, the log file is located at /var/log/sites/[site].[env]/logs/[servername]/[logname].log. You can find your website name and server names on the Infrastructure page. For example, the Apache error log for the Dev environment of a website named myexample on a server named srv-25 is:

/var/log/sites/myexample.dev/logs/srv-25/error.log

Apart from Cloud Hooks logs and Scheduled Cron Jobs logs, all the logs listed in the Available log files section are available for download through the Cloud Platform user interface.

Archived logs

The infrastructure creates a new log file every day and compresses, archives, and saves old logs by date in the YYYYMMDD format:

• access.log-20140527.gz
• error.log-20140527.gz
• drupal-requests.log-20140527.gz
• drupal-watchdog-20140527.gz
• php-errors.log-20140527.gz

The system archives log files by file name, and won’t archive files in the logs directory if they don’t have the default names intact like those listed above.

Accessing logs using SSH

All the logs listed in the Available log files section cannot be accessed through SSH, scp and rsync. However, if you write your custom or cron logs in one of the following directories, you can access such logs through SSH, scp and rsync.

• /shared/logs
• $HOME
• any persistent directory

You can access log files on an infrastructure using SSH. For example, to view the Apache access log on the specified infrastructure:

less /var/log/sites/[site].[env]/logs/[SERVER_NAME]/access.log

Downloading logs

You can use the following methods to download logs:

Using Acquia CLI

To download the logs for any of your Cloud environment using Acquia CLI:

1. Check the list of log files available for your environment:

   copy

   acli api:environments:log-list AH_SITE_GROUP.AH_SITE_ENVIRONMENT

   Using jq:

   copy

   acli api:environments:log-list AH_SITE_GROUP.AH_SITE_ENVIRONMENT | jq '.[] | .name'

2. Initiate the process of generating a log file of the required type, for example, apache-access :

   copy

   acli api:environments:log-create AH_SITE_GROUP.AH_SITE_ENVIRONMENT apache-access --from="2024-01-20T00:00:00+00:00" --to="2024-01-20T23:59:59+00:00"

   The Log snapshots must cover no more than 24 hours and can only be retrieved up to 30 days in the past. The Log snapshots is based on the time zone that you have set on the Cloud UI. You can check that by going to Account > profile.

3. To confirm if the Task listed for log file creation has completed successfully, run the following command:

   copy

   acli api:applications:notification-list AH_SITE_GROUP

   If the status of the log creation task is shown is completed, that means the log file is generated, and now you can download it.

   Using jq:

   copy

   acli api:applications:notification-list AH_SITE_GROUP | jq '.[] | select (.label | contains ("Log file created"))'

4. After waiting for couple of minutes or once you have checked in Cloud Platform user interface that the task is completed, download the generated file:

   copy

   acli api:environments:log-download AH_SITE_GROUP.AH_SITE_ENVIRONMENT | jq apache-access

   The output of above command will give a "url", you need to click on the URL and open it in browser to start the download of log file.

   Using jq:

   copy

   curl $(acli api:environments:log-download AH_SITE_GROUP.AH_SITE_ENVIRONMENT apache-access | jq -r '.url') -o apache-log.gz
   

Additional (inaccessible) logs

Acquia uses additional logs from the logs mentioned earlier on this page that are not available to subscribers. These logs are inaccessible for several reasons, including the potential for security issues related to shared resources.

• Binary logs: Binary log (binlog) files are used to replicate data from the master database to a replica database. All statements that change data, including create, update, and delete statements, add lines to the MySQL binlogs. Queries that only read from the database, without changing the contents, don’t add lines to the binlogs. Staging environments, despite not having redundant databases, also use binlogs to keep feature and performance parity between non-production and production environments. For more information, see Binlogs.

• Email logs: Email logs are often a shared infrastructure resource, may have sensitive information, and aren’t available to subscribers.

Related topics

• Using logs
• Searching the error logs to troubleshoot problems

Frequently asked questions