Basic DevOps Tooling for Drupal

Background:

A Drupal project requires a basic devops framework the commands to Build, Validate, Run Tests, and Deploy.

Some of these start simple at the start of a project Build may be just composer install but become complicated as your project evolves over time, for example as themes are added that have their own more complex build step.

The Acquia project BLT previously provided a good starting point for these tools and operations, but is being deprecated in favor of newer, more modern, and more widespread tools.

Let's look at the basic scripts and tools you would use in starting a modern Drupal project, and how to implement them using Composer Scripts. ( If you have an existing codebase and you are removing BLT from it, this blog post will help, and you should also refer to Dane Powell's post You don't need BLT on Acquia Cloud .)

A second part of a DevOps framework would be calling those composer scripts or steps automatically on the appropriate events such as running tests after a code merge in a repo, or running database updates after a code tag deployment we'll touch on that as well. Common ways to do this use Cloud Hooks or (for Site Factory projects) Factory Hooks.

Steps Needed

We need steps or scripts for the following:

• Build - For the Drupal backend, this will most often be a simple composer install. However, most Drupal projects involve a theme that has some build steps itself something like npm install; npm build in the appropriate directory, and a multisite or Site Factory project might have a number of themes that need to be built. This is usually triggered by a pull request preparing for Tests, or when building an artifact for deployment.
• Validate - When changes are made, we want to check code syntax, code style rules, and etc. This usually applies to more than just php twig and yml also have validators. This can be triggered on a git commit with git hooks, or on a pull request, or both.
• Tests - Running tests is a step above validation, but it implies an installed and running system. Unit tests for individual pieces of code, and end-to-end tests in the form of behat or nightwatch.js tests that load web pages, should be able to be run here. This would usually be triggered by a pull request.
• Deploy - This task needs to run the Build step, but put the results in a separate build artifact, i.e. a separate branch or tag. This is usually then pushed to a separate git repository, the git repo that is tied to the hosting environment. This step will exclude directories that are created as part of the build process but not needed to run the applications, such as node_modules in the theme directory, needed for theme building steps only.

Let's look at an example of each of these in turn.

We will make a composer script for each of these tasks, and have that script call a bash or php script when it is too complex to manage well in composer.json.

Build

This example is a Drupal 10 codebase that has a custom theme named mytheme, that is a sub-theme of bootstrap_sass. This is a good example because bootstrap_sass requires a build step.

In composer.json we have the following, in the composer scripts section:

"scripts": {
 . . . 
 "drupal:build": [ 
 "./scripts/build.sh" 
 ]
}

Note - the "scripts" section of the composer.json file is at the same hierarchical level as "extra", not inside "extra".

In /scripts/build.sh we have the following:

#!/bin/sh

set -e

composer install

# If managing a codebase with multiple themes, consider looping
# over them here and running a ./build.sh script you place in
# each one.

cd docroot/themes/custom/mytheme

npm install

gulp

cd ../../../..

This is a very minimal setup, and for many projects it will become more complex as your project evolves however it is sufficient, that you can now type composer build and it will build your system.

Validate

In our sample Drupal 10 codebase we wish to put in a basic validation step.

In our example codebase we have installed grumphp and the twigcs with the commands:

composer require --dev phpro/grumphp php-parallel-lint/php-parallel-lint

composer require --dev "friendsoftwig/twigcs:>=4"

We used a simple gumphp configuration file grumphp.yml in the root directory:

grumphp:
 tasks: { composer: null, phplint: null, yamllint: null }

This should allow you to run the command grumphp run and it will run the checks.

We now will make that command runnable from composer adding it to the drupal:build step already created above:

 "scripts": {
. . .
 "drupal:build": [
 "./scripts/build.sh"
 ],
 "drupal:validate": [
 "grumphp run"
 ]
 }

Typing composer drupal:validate will now run our validation command. In this case we did not make a separate script, we just put the one command needed in composer.json; if our validation step evolved over time as the project grew, we might end up with a number of commands, and want to consolidate them into a scripts/validate.sh file, and then call that from composer. ( Maybe later in the project we might add phpcs and Drupal-specific rules, for example. )

Tests

There are many different methods and frameworks for running automated tests. Our main goal here is to provide an example that can be expanded upon.

We'll make a scripts/test.sh file, for now that will just launch phpunit; but ultimately we'll want it to run more sophisticated end-to-end tests:

#!/bin/sh

set -e

phpunit

And also add to the composer.json scripts section:

 "scripts": {
 . . .
 "drupal:build": [
 "./scripts/build.sh"
 ],
 "drupal:validate": [
 "grumphp run"
 ],
 "drupal:test": [
 "./scripts/test.sh"
 ]
 }

Now, typing composer drupal:test will run the scripts/test.sh file.

Deploy

This is usually one of the more complex steps, at least at first ( mature projects may have more complicated testing steps ).

A lot of the heavy lifting here is done by the acli push:artifact command: https://docs.acquia.com/acquia-cloud-platform/add-ons/acquia-cli/commands/push:artifact

The composer file scripts section calls a shell script here, as with the Test and Build steps:

 "scripts": {
. . . 
 "drupal:build": [
 "./scripts/build.sh"
 ],
 "drupal:validate": [
 "grumphp run"
 ],
 "drupal:test": [
 "./scripts/test.sh"
 ],
 "drupal:deploy": [
 "./scripts/deploy.sh"
 ]
 }

In the scripts/deploy.sh file:

#!/bin/sh

set -e

#TODO: Check if we are on a branch or tag and handle separately
acli push:artifact --destination-git-branch=develop-build

Note, the scripts/deploy.sh file can be extended it can check if the current working directory is a git branch or git tag, and automatically build the deploy branch or tag by appending -build, for example; or post notifications to Slack or elsewhere.

Cloud Hooks and Site Factory Hooks.

These are scripts that are run on various events on the Acquia platform after the deployment of a new code tag, or after a new database is loaded, for example.

( In many cases the Cloud Hooks may be replaced with Code Studio Actions. )

These scripts can be written, or modified from their defaults, to use the composer commands we have created above.

In cases where the hooks are already set up with BLT, or if you are modifying examples that presume the use of BLT, make modifications to replace BLT commands:

Notes on Acquia Site Factory and Multisite

Multisite and Acquia Site Factory will work similarly for all the basic steps; for the Test step, when running that in the CI/CD environment, you may wish to pick a particular site or sites to use, instead of an empty install of the application.

The acsf-tools module should be installed, as a DevOps aid to running drush commands on multiple sites.

References:

https://getcomposer.org/doc/articles/scripts.md

https://dev.acquia.com/tutorial/you-dont-need-blt-acquia-cloud

https://docs.acquia.com/acquia-cloud-platform/add-ons/acquia-cli/commands/push:artifact