How to investigate Memory Consumption problems

Issue

There are a number of reasons why your Drupal site's pages can exceed the PHP memory limit. Insufficient memory allocation within the PHP settings, too many additional modules, too much content being rendered at once, and large-data tasks are all causes for out-of-memory problems. These often manifest as PHP Fatal Error: Out of memory  or  Fatal Error: Allowed memory size of [number] bytes exhausted errors on logs.

Resolution

Here are some techniques to help you figure out what is using significant amounts of memory:

Logging and memory profiler

Memory Profiler module is a lightweight module that can be used on a production site. However, we recommend testing memory consumption on staging or development servers.

1. Increase your PHP Memory limit incrementally until the errors subside.
2. Install and enable syslog (for the drupal-watchdog.log) and the Memory profiler module. Disable the dblog module.
3. Repeat the steps that led to the initial high memory usage or Out Of Memory error.
4. Note: To get a better idea of the maximum memory use on these section, you should profile the memory usage of these pages after clearing Drupal caches.
5. Disable the Memory profiler module when testing is completed.

Each page will then log an entry to the drupal-watchdog.log, with a tag of memory profiler and the amount of memory that page used. You can easily search and analyze this. You can then either create conditional memory increases for those paths as described in Conditionally increasing memory limits, or determine if an overall site memory boost is required.

This is also a small enough footprint to use safely in production, but if you can replicate the issue on a development or staging environment, that is usually better.

If you download your Drupal Watchdog logs and have Linux-like command line tools such as grep available, you can use this query to help parse the memory profiler output for today's log:

# Run this from the shell in your /var/log/sites/[site.env]/logs/[servername] folder.
#
# This gets the latest 50 (see $num) memory-exhaustion errors from php-errors.log, 
# and then shows the corresponding lines found in $logfile (e.g. access.log, drupal-watchdog.log, etc)
# NOTE: Large values in $num can place load on your server while this runs--you should download files if you plan to do any heavy analysis.
#
num=50; logfile=access.log; grepstring=`grep "Allowed memory size" php-errors.log |tail -$num |grep -o 'request_id="[^"]*"' |awk -F'"' 'NR>1 { printf "|"; } { printf $2 }'`; egrep "$grepstring" $logfile

XHProf

XHProf can trace script execution times and profile memory usage. You will need to run XHProf locally to use it, however, as it's not offered as part of Acquia Cloud.

You will need to ensure that the XHPROF_FLAGS_MEMORY flag is enabled. More information on both enabling xhprof and the xhprof contstants is on PHP.net. For specific information on enabling and using XHProf on your Drupal site, see Using XHProf to evaluate code performance.

New Relic

New Relic is able to be used on Acquia Cloud. You can use it for determining where bottlenecks may be occurring for code, network or databases.

XDebug

XDebug, which is available on Acquia Cloud IDE, can help you step through code and inspect exactly what code is executing, the data it is generating, etc. See https://docs.acquia.com/ide/tooling/#xdebug  for more.

More information

For information on increasing memory limits, see the article on Conditionally increasing memory limits.