Integrating Lighthouse Suite in Code Studio

Goal

Learn how to use Acquia Cloud, our CDEs, and Code Studio to run Lighthouse CI automated tests in our Drupal specific CI/CD build process.

Prerequisites

• Basic knowledge of NPM, CI/CD workflows, automated testing, and the Acquia Cloud.

Overview

What is Lighthouse

Lighthouse CI (LHCI) is an open-source, automated tool for improving the quality of web pages. Built by Google, LHCI makes it easier to test, archive, and browse reports on SEO, performance, and accessibility. Developers and site owners use these reports to take action against and improve their applications.

Features

• Run LHCI reports alongside every PR.
• Prevent regressions in accessibility, SEO, offline support, and performance best practices.
• Track performance metrics and LHCI scores over time.
• Set and keep performance budgets on scripts and images.
• Run LHCI many times to reduce variance.
• Compare two versions of your site to find improvements and regressions of individual resources.

Where should I run my Lighthouse tests

When testing web applications there are multiple areas to consider, functional vs non-functional, code standards, visual regression, performance, SEO, accessibility, and more. Taking a step back, we first need to determine which test should run in which environment and scenario.

In the CI/CD build, DEV or PROD

Testing in the CI/CD build

Whether you are using Jenkins, Travis, CircleCI, Bitbucket, Gitlab or Acquia’s Code Studio, our Drupal-optimized CI/CD and code collaboration solution built around Gitlab. If we look at CI/CD, focused on compiling and testing the code then there are a AutoDevOps you can run in the CI from code standards, PHPcompatibility, Semgrep, SAST to unit testing, these are all good examples of the kinds of tests you would run in the CI build process.

Using the build process to focus on testing the code will ensure your team that all code on its way to PROD has been tested and validated. Another consideration is the time it takes for each build to run. Taking into account that you might have a team of developers pushing code, how much total time do you want between pushing and the availability to deploy?

As mentioned above, the build process can play a valuable piece of your DevOps puzzle. Further in this article, I’ll explain how to use CI/CD to set up, and archive LHCI tests to run on different environments.

Testing in a temporary environment or DEV server

At this point, the code has already been deployed and is on one of the servers, DEV, Non-PROD or PROD. Having the code tested prior to this stage ensures that your team is testing a validated artifact of the application. This is a good time to start testing other aspects of the application, like Accessibility and SEO.

Running your LHCI test here makes sense, as you have the Database, Files and Configuration ready to go. That and the fact that it's an environment with known resources to benchmark against.

At Acquia, we have deployed CDEs, or Continuous Development Environments that are integrated into our Code Studio product to spin up new CDE environments on any new merge requests. This existing process gives us some interesting tools to build our LHCI testing into.

When thinking about performance testing, some customers choose to rerun these tests in PROD to benchmark against. When doing this, consider that the performance tests will not be comparable, as the Non-PROD environments usually do not have the same allocated resources as PROD environments.

Are these real tests if I’m not running them in PROD?

What is real? Does your UAT or TEST server have the same allocated resources as PROD? Can you observe the traffic and load on the server while maintaining efficient page load speeds.

There are things to think about when it comes to the different kinds of tests, benchmarking, measuring, and iterative. For all these examples, the goal is to improve and mitigate risk. If you keep that in mind, you can develop your own testing procedures that will lead to a successful DevOps process.

While Lighthouse’s primary focus is performance, it can assess other things like:

• Accessibility: Recommendations on making content more accessible. This ensures that the page is optimized for screen readers, all elements have labels, or the site is browsable with the keyboard.
• Best practices: Scans the page issues not following best practices and can improve performance and security.
• SEO: Runs different checks to ensure that the page is SEO-friendly

• The objective is to spin up a CDE and run the test each time a merge request is made.

  The goal of this is to have a new feature branch get created, which automatically spins up a new CDE in the Acquia Cloud, connected to that merge request. This allows us to write the LHCI tests around the merge request event type.

• #.gitlab-ci.yml

  If you have not done so already, create a `.giltab-ci.yml` file in the root of your project. To enable it, go to https://code.acquia.com///-/settings/ci_cd and expand the General pipelines section. Change the CI/CD configuration file to `.giltab-ci.yml` .

  The code below adds a new job to the `Deploy Drupal` stage called `Create Lighthouse Test`. The job has a dependency on the `Create Acquia CDE` job and runs only on merge requests. We are allowing this test to fail and archiving the reports for a week in a folder call ./report.

  copy

  include:
   - project: 'acquia/standard-template'
   file:
   - '/gitlab-ci/Auto-DevOps.acquia.gitlab-ci.yml'
  
  variables:
   REPORT: '$CI_PROJECT_DIR/report'
  
  "Create Lighthouse Test":
   stage: "Deploy Drupal"
   extends: .cache_strategy_push
   needs: ["Create Acquia CDE"]
   before_script:
   - acli update
   rules:
   - if: ($CI_PIPELINE_SOURCE == "merge_request_event" && $ACQUIA_JOBS_CREATE_CDE == "true")
   when: on_success
   allow_failure: true
   artifacts:
   when: always
   expose_as: 'Lighthouse'
   paths:
   - "$REPORT/lighthouse/"
   reports:
   junit: $REPORT/lighthouse/summary.json
   expire_in: 1 week
   services:
   - selenium/standalone-chrome
   script:
   - source scripts/ci/lighthouseci.sh
  
  

• #lighthouseci.sh

Downloading the results

In the Code Studio UI, CI/CD > Pipelines page, you can review build jobs and download the artifacts that Code Studio saves. Using the steps above, we’ve added a new archive named `Create Lighthouse Test:archive` . Clicking on that link will download a zip with HTML and JSON files for each page in the list above.

Example Lighthouse Test

Looking at one of the HTML files, you can see that lighthouse renders each page and runs the Performance, Accessibility, Best Practices, SEO and Progressive Web Apps tests.

Conclusion

The steps above can get you started on developing a clean DevOps process and ensuring you run the right tests at the right times in the right environment.

• Build a strategy around testing, where, when and what
• Implement a POC and run process tests around that to determine if this fits into your existing workflow
• Talk to an Acquia Professional Services Solutions member to hear more about how our PS team can help your team build a refined DevOps process using our best in class tools built for Drupal