Value of Testing in Drupal

Drupal8 beta is Out!

You probably know by now that Drupal 8 beta was launched and that it took some time. This was a huge effort made by the community. Congratulations to everyone!
Now we need to look back and see what we can do better. What can we improve to reduce the time to release?

source: https://groups.drupal.org/core/updates

If you are a Drupal contributor, one of the things that immediately pops into your mind is the time that takes to test your patch. Testing happens throughout the whole workflow of getting a change in Drupal core and contrib modules. Testing is what keeps you awake to see if your patch failed or passed. Testing is what makes you pull out your hair or jump with joy.

You will probably be happy to know that, at the Drupal testing team, we are working hard to make sure the process is improved.

Testing needs to be faster

Drupal testing was announced by Dries in April 2008 and back then there were only a few tests to run. Right now Drupal7 testing takes only 20 minutes, while Drupal 8 is taking much more time:

Drupal has increased the amount of all tests to be performed and some of them are even slower than before. A testbot (Job Runner) is what we call a computer that performs automated testing and reports back the results. During Drupal 7 development, 3 Job Runners were enough, but right now we are running on 8 Job Runners. This needs to be improved!

More agile

Last year I gave a talk at Drupalcon Prague about Docker, that covered how docker's container tech can improve testing performance, which got some traction by the community and generated more awareness towards adoption. I believe, even more now, that it could make Drupal testing and developing process more agile. Eventually, some time later I met Jeremy Thorson, who was searching for devops for the testing project and had the same goals of making the process easier, faster and scalable.

We finally joined forces at DrupalDevDays Szeged where the Drupal Docker testing Proof of Concept was created.

Drupalci testbots will play a very important role on automating the runs:

In short, as we envision it at the DrupalCI project, Drupal testing should be:

• Faster
• Lighter
• Automated (using travisCI files for eg.)
• Pluggable
• Easier to understand codebase and to contribute to!

This increases the value of Drupal work and decreases the time to market.

Progress

At Drupalcon Amsterdam we made progress. One of the things we are most proud of achieving is the bootable testbot, enabling developers to run the test stack anywhere, the coreresults site and the dedicated Postgresql testing site (still in Alpha):

1 - http://coreresults.drupal-pt.org/

Every 2 hours we test a different combination of PHP (5,3, 5.4, 5.5, 5.6), Drupal (D7 or D8) and DB (mysql, postgresql, and sqllite)

2 - http://testbot-pgsql.drupal-pt.org/

Dedicated test runner for postgresql running a few times every day:

Contribute

This is the current list of sub-projects that we, the Drupal testing team, are working on at the moment:

• DrupalCI: Drupal.org Integration

• Integration between drupal.org and the 'Modernizing Testbot' infrastructure.

• DrupalCI: Dispatcher

• Configuration files used on the jenkins master/slave servers which serve as the central job dispatcher for the environment.

• DrupalCI: Test Runner

• The core test runners being used on the infrastructure.

• DrupalCI: Results

• Long-term archival, storage, and exposure of build artifacts and job/test results; similar to the (non-dispatch related) functions performed by qa.d.o today.

• DrupalCI: Puppet Scripts

• The puppet configuration scripts used to build out the various servers which make up the environment.

Any new volunteers are welcome to help with the design and implementation of the new testing environment. We have several issues to be picked up from https://www.drupal.org/project/issues/drupalci_testbot?categories=All

You can also file an issue indicating your interest or reach all the amazing contributors for this project via the irc channel #drupal-testing and we will bring you up to speed!