Acceptance Test Driven Development with Keyword Driven Test Automation Framework

Agile software development uses iterative development allowing changes and updates periodically to the software requirements. In agile software development methods, customer-defined tests have an important role in assuring that the software fulfills the customer's needs. These tests can be defined before implementation to establish a clear goal for the development team. This is called acceptance test driven development (ATDD).

With ATDD the acceptance tests are usually automated. Keyword-driven testing is the latest evolution in test automation approaches. In keyword-driven testing, instructions, inputs, and expected outputs are defined in separate test data. A test automation framework tests the software accordingly and reports the results.

In this thesis, the use of acceptance test-driven development with the keyword-driven test automation framework is studied in a real-world agile software development project. The study was conducted using action research during a four-month period. The main methods used were observations and interviews.

It was noticed that the keyword-driven test automation framework can be used in acceptance test-driven development. However, there were some limitations preventing the implementation of all the test cases before the software implementation started. It was also noticed that the test automation framework used to implement the acceptance test cases is not in a crucial role in acceptance test driven development. The biggest benefits were gained from the detailed planning done before the software implementation at the beginning of the iterations.

Based on the results, acceptance test-driven development improves communication and cooperation, and gives a common understanding about the details of the software's features. These improvements help the development team to implement the wanted features. Therefore, the risk of building incom-plete software decreases. The improvements also help to implement the features more efficiently as the features are more likely to be implemented correctly at the first time. Also remarkable changes to the test engineers' role were noticed as the test engineers are more involved in the detailed planning. It seems that the biggest challenge in acceptance test driven development is creating tests on right test levels and in a right scope.