Cloud Platform
Automated Accessibility Testing: Using Pa11y, BLT, and Continuous Integration
Accessibility is increasingly critical for many organizations as they re-evaluate their web presences. There are many open source tools available to you on a Drupal platform to perform accessibility testing. There are some challenges though! This guide is all about using tools from Acquia and the Drupal community to automate accessibility testing for your platform.
What is needed
Pa11y (and other accessibility testing tools) requires markup to test output. This means that if you install a clean version of Drupal (or any other website) with no content you won't get a true representation of the accessibility of your website! Out of the box Pa11y tests for WCAG 2.0 standards with configuration available to specify A, AA, or AAA.
In order to get started, here are a few quick considerations:
Have I completed enough development to have anything to test?
Unlike most test driven development (TDD) paradigms, Accessibility testing in Drupal requires that site building (specifically, configuration), backend development (especially theme functions), and front end (both the Twig templates and styling) to be completed as well as realistic content to be created before you can effectively "test" anything. As a result, we typically recommend avoiding enabling Pa11y until a feature has been built and is passing other manual accessibility testing (e.g. Lighthouse extension in Google Chrome).
Did I build a component-based site?
If you website / platform is built as a series of common components that rely on the Drupal entity system to manage your display, theoretically this means a few things:
If these statements hold true, you shouldn't need to test "every single" hero banner on your website. If you build an accessible hero banner that renders consistently, one or two hero banners should be sufficient to ascertain if the hero banner remains accessible (or not).
Do I have content?
In most CI builds, "real" content isn't present. BLT (by default) will do a setup during CI (which installs a clean database with no content, users, taxonomy, etc.). This means that to consistent test a given component we either need to import it or create it as part of the build!
When do I run Pa11y?
Once a feature has been built and made accessible, Pa11y should automatically test it every time you change your code. In other words, every pull request should trigger a CI build and that build should run Pa11y. This helps ensure if a change is being made that might impact the accessibility of a feature, the build will fail and you will be notified.
Pa11y is typically added to projects as a Node.js dependency (e.g. using NVM / NPM). As a result of this, it usually isn't added at the "project" level and is instead integrated as part of a Drupal theme. Acquia's COG starterkit includes an example of how to use Pa11y in a theme!
TLDR; Pa11y must be added to the theme's NPM file as a dependency and a Gulp task must be created to run it. Finally, Pa11y must be configured (either in the Gulpfile or in a separate config file) to tell it what standards to test for, what URLs to test, etc. See the Pa11y docs for more info.
Once Pa11y is in the codebase and configured to run, the next step is automating this process. BLT includes several command hooks for front-end behaviors. For this activity, we will want to include Pa11y as part of the command blt tests, so we'll configure the command hook tests:frontend:run to run Pa11y automatically.
To add the task to the command hook, open the blt.yml file and add:
command-hooks:
frontend-test:
dir: '${repo.root}/docroot/themes/custom/<your theme>'
command: 'gulp pa11y'Your command may be slightly different if you have other frontend tests (or are using npm test or other commands), and obviously replace <your theme> with the name of your custom theme. Once this snippet of YAML is added to blt.yml, anytime blt tests is run, it will automatically include your frontend tests!
Note: if you have multiple themes or even some modules, you can use a custom shell command to execute multiple commands. For example,
command-hooks:
frontend-test:
dir: '${repo.root}/blt/scripts'
command: './<your-script>.sh'Then, in the <your-script.sh> file
#!/usr/bin/env bash
echo "running theme frontend tests"
(cd docroot/themes/custom/<your theme one>; gulp pa11y)
echo "running react frontend tests"
(cd docroot/modules/custom/<your module one>/js; gulp pa11y)The final component of this is content. At this point, Pa11y should work both on manual execution (in the context of a theme) and automatically using BLT during CI. The last missing piece is the content.
Both the Scenarios and Default Content module provide methodologies for importing content into Drupal sites. Our recommended approach is to create a number of pages in Drupal (potentially landing pages with placed blocks) and then capture this content using one of these modules. The config split module can then be used to define a CI specific split that will be enabled during blt setup during CI. The content will be created. If Pa11y is configured properly, it will then crawl the pre-defined pages (that just got created) and test the markup of real content!
A final warning, remember that tools that test markup (like Pa11y and Lighthouse) only test a fraction of the true spectrum of testing required to make a website "fully" accessible. Do you want your site to work well with a screen reader? What about tabbing / keyboard navigation? Pa11y can test "some" of the required aspects of these features, but doesn't replace the need for actually testing with a screen reader and/or a keyboard.
The take away: YES, you should absolutely be using an automated testing tool like Pa11y to regularly test and confirm the accessibility of your website. NO, this shouldn't be the only accessibility testing you're doing on your platform.
Cloud Platform