Using Acquia Purge & cache tags invalidation for current Drupal version

Introduction

Tag-based cache invalidation is a performance enhancement available to sites running the current Drupal version on the Acquia Cloud platform. Cache invalidation by tags allows sites to virtually cache never-changing content forever. This results in a drastic workload shift from web servers to load balancers.

All you need are the Purge and Acquia Purge suite of modules installed on your Acquia Cloud website.

Why cache tags invalidation?

The current Drupal version features cache tags invalidation as a major performance enhancement for every type of site, but especially those with highly dynamic content. It is designed to allow very selective invalidation of cached content, allowing sites to refresh only a subset of elements on the page. This, in turn, reduces backend load and decreases page load times.

How is tag-based cache invalidation supported on Acquia Cloud?

The current Drupal version tags all content pieces it renders with a descriptive tag, for instance "node:123" for a content item. When Drupal renders that node onto a page or as part of a page (for instance as a teaser or inside a view), it collects all these tags together and allows the Purge and Acquia Purge modules to aggregate all the tags in a HTTP response header. Put differently, every page Drupal now sends to Varnish will contain a technical overview of what is shown on the page to the user. This includes not just content nodes, but everything else that is visible, from blocks to the theme that is in use.

Your Acquia Cloud load balancers are powered by Varnish, which is now configured to store these special headers and to remove them when the page is delivered to your browser or CDN. However, this is just half of the machinery at play here. When you or a colleague logs in to Drupal and edits node 123, Drupal will gather the tags that now changed and will immediately remove all the URLs from its own internal page cache. The Purge module also receives these tags from Drupal and will now queue them to have them removed from your Varnish-powered load balancers as soon as possible. The Acquia Purge module provides the plugin to the Purge module, which will actually send the right commands to Varnish to make sure all URLs are also removed from your load balancers.

Because tags cover everything rendered by Drupal, you can now put the TTL time of your site to extremely high values (e.g. a year). This instructs Varnish and CDNs to cache all pages virtually indefinitely and Purge will now proactively remove items on entity save. The major breakthrough is that web servers are freed up for only backend and authenticated traffic and push as much work as possible into Varnish. This massively improves your baseline performance, and increases your site's resilience.

How do I install and configure Purge and Acquia Purge?

1) Add Purge and Acquia Purge modules to your codebase. You should always use the latest (non-dev) available release. Enable the following modules:

purge
purge_ui
purge_drush
purge_queuer_coretags
purge_processor_cron
purge_processor_lateruntime
acquia_purge

2) We recommend that you increase your TTL/max-age to 1 month at a minimum. Navigate to /admin/config/development/performance and select 1 month in the Page cache maximum age dropdown, in the Caching section.

Alternatively, you can use Drush:

$ drush @site.env config-set -y system.performance cache.page.max_age 2764800

3) Navigate to /admin/config/development/performance/purge and click on the Add purger button in the Cache invalidation section (sometimes it might require that you rebuild caches for the Add purger button to show up). A pop-up will ask which purger you wish to add. Acquia Cloud will be selected by default. Simply hit Add. Once the purger has successfully been added, a green tick appears in the TAG column.

Alternatively, you can use Drush to add the purger:

$ drush p-purger-add --if-not-exists acquia_purge
The purger has been created!

4) To confirm your setup is complete, visit /admin/config/development/performance/purge, or run :

drush purge-diagnostics

Example output

$ drush @site.env p-diagnostics
 ------------------------------ ---------------------------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------- ----------
  Title                          Recommendation                                                                                                                                       Value                                                    Severity
 ------------------------------ ---------------------------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------- ----------
  Acquia Cloud                                                                                                                                                                        app.env (8.x-1.4)                                 OK
  Acquia Purge Recommendations                                                                                                                                                        Nothing to recommend!                                    OK
  Capacity                       Your system can invalidate 100 items when you're processing through webserver-initiated requests. Under ideal conditions - for example via Drush -   100                                                      OK
                                 the capacity would be 100.
  Page cache max age             Consider increasing your TTL to over a year, the better your site will perform!                                                                      1 month                                                  OK
  Processors                     You have multiple processors working the queue.                                                                                                      Cron processor, Late runtime processor, Purge block(s)   OK
  Purgers                        Purger configured.                                                                                                                                   Acquia Cloud                                             OK
  Queue size                     Your queue is empty!                                                                                                                                 0                                                        OK
  Queuers                        You have multiple queuers populating the queue.                                                                                                      Core tags queuer, Purge block(s)                         OK
 ------------------------------ ---------------------------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------- ----------

Your should primarily focus on the suggestions provided in the Acquia Purge Recommendations, where it should consistently indicate 'Nothing to recommend!'.
You will get a WARNING here if Acquia Purge is not setup properly.

For example, if your site is behind HTTP authentication, you will see this warning message:

Acquia Purge detected that you are protecting your website with basic HTTP authentication. However, on Acquia Cloud all HTTP responses with access authentication deliberately MISS cache to prevent sensitive content from getting served to prying eyes. Acquia Purge cannot detect if specific parts of the site are protected or all pages, but does recommend you to temporarily disable invalidating caches if indeed your full site is protected. Please wipe Drupal’s “default” cache bin when this warning persists after you updated your .htaccess file or uninstalled the Shield module!

To resolve this you should consider removing basic HTTP authentication from your site.

Note: Due to a bug in acquia_purge versions before 8.x-1.4, Acquia Purge assumed the Shield module was configured even if it was not. This bug has been fixed in version 8.x-1.4 and later releases.

If you have any questions regarding the setup, please contact the Acquia Support Team.

Optimizing your Settings

The suite of default settings that we share above are helpful for most sites, but some high traffic sites may find that the lateruntime purge queue processor sends too many BAN requests to Varnish, which can create another type of performance issue.

We recommend that you visit the Drupal.org Purge module documentation for the latest information about queue processors, and use the one that is most appropriate for you site's traffic patterns.  See https://git.drupalcode.org/project/purge#processors for more information.
 

Adding processors to invalidate cache from the queue

You may find that when using tools to import content from other sites, for example, you may find that you want to add a scheduled job to process the items in the queue until every item in the queue can have the cache cleared in Varnish.  This is an example of a drush command using drush 9 that will process the queue safely.

drush  p:queue-work --finish

--finish is designed to process a queue in a sustainable way but also as fast as possible, balancing resource consumption stability over speed. Long story short, it spawns processes until its finished, so you'll see Drush procs being serially-instantiated until it manages to clear the queue.

How to invalidate tags with your CDN?

First, make sure your CDN provider supports tag-based invalidation. It's a pretty innovative technology and not all vendors are compatible just yet.

Cloudflare

• Purging by tags is available for Enterprise only
• Cloudflare has a daily upstream API rate limit.
• Because of that API rate limit, and depending upon the number of requests for invalidation your site makes, cache invalidation might "stop" multiple times a day, letting Purge keep things in its queue until CloudFlare's API limit for the day has expired.
• To work around this issue, you could ask Cloudflare to increase the API limit, but please keep in mind the Drupal Cloudflare module depends on the cloudflarephpsdk SDK and API call limits are hardcoded there so even if the limits were increased at Cloudflare, you would be limited by the SDK regardless.
  • We are working with the SDK maintainer to remove this limitation
• As an interim solution, we advise to default to a short TTL at the edge while Cloudflare is evaluating higher defaults for Drupal 8+ sites. For this, you can create a Cloudflare Page Rule; see their article on the Edge Cache TTL Settings.

Akamai

Unfortunately, Akamai doesn't yet support tag-based invalidation. This can cause issues and even bring your site down when sending too many URLs for invalidation at once. Please carefully evaluate your caching strategy with this limitation in mind.

To still benefit from fresh content with Akamai in front, you can:

• Set a low TTL on the edge (e.g. 5 min) - Recommended
• Leverage the URLs queuer module to trigger legacy URL or path based invalidation - Not recommended! Read why .

Akamai is in the process of testing tag-based invalidation and should be able to support it sometimes in (but no earlier than) Q1 2018. Please get in touch with them to know more about their readiness status.
 

How do you take advantage of cache tags invalidation?

Cache tags invalidation works like magic. There’s no cache you need to rebuild yourself, nor any additional complex set-up or cache invalidation strategy you need to implement. This is why a high TTL is actually a good thing with the current Drupal version. Your newly configured purger will take care of automatically invalidating the obsoleted cache tags on your site.