Using Selenium for Platform Testing

One of the key aspects of successful project delivery is creating an effective quality assurance (QA) process. Functional testing of the application is usually performed by a separate team than the developers who typically do not know how to program. A challenge that I have found on projects is providing a good open source QA tool to the the QA team that does not require programming knowledge. Selenium has been the tool of choice and works wonderfully, but we have found a couple of issues when integrating it into our continuous integration (CI) processes.

The first challenge that we have run into when using Selenium is exporting the test case from the Selenium IDE to JUnit/Java. We export the test cases to Java to take advantage of the Selenium2 Grid capabilities. This is an issue because the test does not export perfectly from Selenese to Java and requires a developer to fix the test. Obviously it would be better not to involve a developer. Thankfully the fixes to the scripts follow the same patterns so they are quickly resolved.

The times when the conversion needs to be corrected are:

• when setting up the test, one must create a RemoteWebDriver rather than a FirefoxDriver.
• when using the JQUery Datepicker, one must create the date programmatically rather than trying to use the actions created by the script.
• when moving multiple items in one box to another box, one must first use the Select object to select the item rather than using the findElement method from the driver.
• when asserting an item on the screen, it is necessary to stop the test (as often times the test will continue to run and keep failing).

Once the test conversion is complete, the test case is added into a test suite and all changes are committed to the source repository for the automated test runs.

Because we convert from Selense to Java, the test case that is maintained by the QA team becomes disconnected from the test case that is run in the CI tool. Currently this is more of a communication issue for the teams. Since projects usually consist of a small number of team members with daily scrum calls, any coordination between QA and development to synchronize the Selense and Java test cases is easy. One way to solve this problem could be to use something like SauceLabs to run the tests, but budget or security requirements of the project often do not allow use of a third party provider. At one time I was hopeful that the Bromine project (http://www.brominefoundation.org/) would be a solution, but they stopped development back in April 2012. I am hopeful that 2013 will bring better solutions to this problem.

Though we have had challenges using Selenium, overall it has been a key tool to the delivery of a quality project. I am confident that, as we refine our processes and expand our tool sets, we will find better ways to utilize Selenium on our projects.