Scoping the impact of image processing

Handling large files

Images are an important component of many, if not most, websites. To ensure the proper display, images often require some level of processing by whatever content management system (CMS) is in use, including Drupal.

Processing can span from a single, basic image resize to fit a standard, to multiple versions of an image at various sizes and resolutions. Image processing can be complex, and can put a major hit on website resources — especially memory. Underestimating the memory usage of these processes can have severe impacts on the responsiveness of your website. Other types of files can also have an impact, but they are generally not as severe as image processing.

General recommendations

There are several ways to remediate potential issues due to large images, including the following:

Use the right image package

Consider how your website is going to deal with images. There are two major packages used for image handling with Drupal.

  • ImageMagick - ImageMagick doesn't use Apache's processes, and is installed on all Acquia servers. Acquia recommends ImageMagick over GD library.
  • GD library - GD library uses PHP memory, which adds to overall PHP needs. It has a larger array of tools for processing images, but can lead to a need for larger servers because it uses more server resources for a longer period.

Use a content delivery network (CDN)

When dealing with an image- or file-heavy website, especially one that has most of its traffic come from anonymous users, Acquia strongly recommends using a content delivery network (CDN) such as Akamai, Acquia Cloud Edge, or Fastly. A CDN can serve images at a significantly higher speed, and easily store multiple versions of an image, with minimal impact to the originating website after the various image versions are generated.

Plan your uploads

Acquia also suggests a thorough consideration of how image and file uploads will be managed. If you are regularly uploading large numbers of images, the processing capabilities of your server may not be able to keep up with traffic demands.

Download with SFTP, scp, or rsync

Acquia does not support file downloads of greater than one gigabyte. To download large files, use SFTP, scp, or rsync to access your Acquia environments.

Offload the derivative generation

To offload the generation of image derivatives, use one of the following methods:

  • Separate your production domain from your content creation domain through the use of an edit domain. You can use various methods of file synchronization, depending on the needs of your website, which includes available bandwidth.
  • Use a secondary server to store and build images. With this method, all of your website paths need to be set accordingly so that any image requests are redirected to the correct server.
  • Use a service (such as Cloudinary) that is designed for large-scale image handling.

Memory usage

Resizing images into the various types that Drupal uses can consume a large amount of memory, even if you're just using Drupal's pre-set image styles (thumbnail, medium, and large). An image that is 2000 pixels by 2000 pixels is nearly 3 megapixels, a size easily within the capabilities of most smartphones.

The following formulas calculate the memory required to open an 8-bit RGB image using the GD and ImageMagick libraries:

Based on these formulas, the memory footprint required by PHP to open one 2000 x 2000 pixel image using GD is 11.4 MB. Every time an imagecache preset is triggered, it will consume 11.4 MB of memory just loading the image. If there is a smaller image preset, that will consume additional memory. This is on top of any other images being processed at the time, as well as the standard memory usage of the website. An image dumped right from a phone can easily consume 22+ MB of memory per image.

Drupal modules

The following Drupal modules can help you with image processing. They may be useful to your website, but they should be tested thoroughly.

 

Module Description Drupal 6 Drupal 7
Media Provides an extensible framework for managing files and multimedia assets, regardless of whether they are hosted on your own website or a third-party website   Yes
Imageinfo Cache Generates image styles both after an image is uploaded and during an entity save Yes Yes
Image Style Pregenerate Generates all the images for a particular image style in bulk   Yes (In dev)
Rules Image Styles Pre-generates styles using rules   Yes (In beta)

Troubleshooting

If you suspect large images may be causing a problem, you can look to see how many there are and where — this example will list the top 10 files, by size, in the current directory. If you have not set up your website to vary the storage directory, you can run this from the /docroot/sites/all/files directory:

ls -alhS | grep -ve ^total | head

As an example, this request might return a response like this:

-rw-rw-r-- 1 mysite_d7 www-data 24M 2013-01-22 05:34 image1.jpg -rw-rw-r-- 1 mysite_d7 www-data 24M 2013-01-22 05:34 wedgeofdestiny.jpg -rw-rw-r-- 1 mysite_d7 www-data 18M 2013-01-02 23:30 image2.png-rw-rw-r-- 1 mysite_d7 www-data 18M 2012-11-09 16:17 image3.png-rw-rw-r-- 1 mysite_d7 www-data 18M 2012-11-11 22:11 testimage_0.jpg -rw-rw-r-- 1 mysite_d7 www-data 18M 2012-11-11 22:17 testimage_1.jpg -rw-rw-r-- 1 mysite_d7 www-data 18M 2012-11-11 22:00 testimage3.jpg -rw-rw-r-- 1 mysite_d7 www-data 18M 2013-01-02 23:30 sadsplinters.png -rw-rw-r-- 1 mysite_d7 www-data 17M 2013-01-28 05:26 bucketlist2.png-rw-rw-r-- 1 mysite_d7 www-data 16M 2013-04-01 13:37 yahboosnubs.png

After you've located a file that is big, and everything on the proceeding list is, you can use the identify command (part of ImageMagick) to learn specific data about it:

identify wedgeofdestiny.jpg

Results:

wedgeofdestiny.jpg JPEG 4860x5184 4860x5184+0+0 DirectClass 8-bit 25.02mb 1.410u 0:02

From this, you can determine that the image is 25 megapixels, and requires 72 MB of memory just to open the one file using GD, and 120 MB if using ImageMagick.

Add new comment

Plain text

  • No HTML tags allowed.
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.

Contact supportStill need assistance? Contact Acquia Support