Why do I see increased “Request Queuing” times on New Relic graphs for my Cloud Platform applications?

The Queue Time metric indicates the duration for which incoming requests to PHP/Drupal had to wait when all PHP workers in a webserver were busy. Queue Times larger than zero are a side effect of incoming requests to the application coming in at a higher rate than what the application’s available resources can handle without a wait.

Queue Time exists in Cloud Platform and Cloud Classic. However, the following explanation focuses on the Cloud Platform.

Cloud Platform provides a base number of PHP workers that are always available, usually 2 sets of 10 for production environments, and 1 set of 10 for non-production. If incoming web requests consistently use 40% or more of all the available workers, then we add another 10 workers (this is Auto-scaling, or scale up) until there are enough free workers.

Each set of these 10 PHP workers is known as a Drupal Pod. You can think of a pod as a virtual machine that runs services like Apache, PHP FPM, and mcrouter. The Drupal code of an application executes in this environment to serve web requests. Each incoming web request is assigned a random destination pod so it can be served.

Sometimes, the combination of incoming web requests, multiplied by the time the Drupal/PHP application can respond to them, causes pods to become completely busy as all PHP workers in the pod are processing previous requests. When this happens, a new incoming request is put into the Queue on that pod. The first request that is still in that Queue will then be served when the next PHP process becomes free.
For clarity, we can imagine a real-world scenario. Picture a bank with 10 tellers who are sitting idle, waiting for customers. If 10 customers come into the bank at the same time, they may need to wait in a queue (a line) for a small amount of time while they are assigned to their teller, and the queue becomes empty immediately. But if 50 customers come into the bank, this queue will grow, and they will need to wait. The time each customer had to wait in line before getting to a teller is known as their Queue Time.

Similarly, if a request is assigned to that pod’s PHP queue, under optimal conditions, it would wait around for very little time (a fraction of a second or even milliseconds). You can visualize the average queue time through the New Relic Application monitoring add-on included with your subscription.

However, sometimes certain extraordinary conditions or events could cause you to see queue times grow, sometimes uncontrollably:

• The rate of incoming traffic has grown enough to exceed the available free workers, which causes queuing.
• The average time your PHP application takes to service requests becomes slower, which may, in turn, cause queuing due to:
  • Factors internal to your application like empty caches, deploying features/code that demands more processing power or places higher loads on CPU, MySQL, Memcache, and more.
  • Factors external to the application, like dependency on external services, such as APIs, that are currently slow, causing your application to wait longer for responses.
  • A combination of both. More incoming traffic requires more resources, which themselves can become slower due to the extra load, locking or concurrency issues, stampedes, and more.
    
    As mentioned before, situations such as these might trigger auto-scaling under Cloud Platform. As more Drupal Pods are added, they will help spread the load onto more PHP workers. Usually, these help the queue times go down, except for certain cases. For example:
• Dependencies on external services that have become slow
• High amounts of complex MySQL queries slowing down the database due to internal locking/processing of data
• An application-level lock causing requests to wait
• Single requests that require high amounts of CPU processing power (since auto-scaling does not increase CPU power, another pod with similar power is added)
• Other factors
  Typically, diagnostics for the aforementioned situations can be carried out using New Relic.