Treat Test Automation Like a Project

Treating test automation like a project helps ensure the success of your automation initiative. Let’s look at how applying Agile project principles – along with preparing appropriate documentation – helps with your timeline and your productivity.

Tests automated during an Agile project follow the schedule of the project. Within a Sprint, tests are developed and executed for the user stories planned for the Sprint. When there is an initiative to automate a manual regression test suite composed of hundreds of manual tests, it’s critical the automation initiative is treated like a standalone project to meet cost, time, and quality expectations.

Projects typically start with a Project Charter or Statement of Work. Your test automation initiative should clearly identify the goals and scope for automation and define ‘done’ as part of that charter or statement.  Priorities and dependencies should be set to identify the most important tests to automate first, such as your smoke tests. These automated smoke tests can be used immediately to start paying back their automation cost and contribute value with each execution cycle.

Agile projects start with a Sprint 0 to establish the architecture before coding of user stories begins. Your automation initiative also needs a Sprint 0 to build out or extend your automation framework, regardless of the automation tool you are using.

For example, we have a base automation framework for various technologies that we ‘lift’ and ‘load’ during Sprint 0 for new automation solutions. This saves us time by not reinventing the wheel and provides consistency across our automation engineers. Also in Sprint 0 an automation design specification should be prepared. This specification provides non-technical stakeholders an understanding of the automation solution without the need to understand software architectures or read program code. The specification includes design components such as:

• how input and output data will be handled
• data cleanup after execution
• exception handling
• logging
• reporting
• archiving results
• execution cycles
• naming conventions
• linkage of automated to manual tests

A Test Automation Architect or Senior Automation Engineer plays a key role in the automation initiative’s Sprint 0.

Now the fun begins with coding automated scripts, using the base automation framework and following the automation design specification. Well, not quite yet. First the manual tests to be automated should be groomed in your backlog, and manual regression tests selected for automation within each two-week Sprint. The definition of ‘done’ is when these automated scripts are ready to be run within an automated test cycle.

Also before coding, the selected manual test case (or test scenario) needs a detailed specification. For UI automation, this specification defines the screens involved, the screen objects that are needed, the test case steps with expected results, input data files content, output data files content, pre-requisite tests, and data clean-up for the script’s re-use.

Stakeholders who are knowledgeable about the application under test – such as manual testers, business analysts, or product owners – should review the detailed specification for completeness and accuracy before coding begins. This minimizes coding rework due to the automation engineer’s misunderstanding of the manual test and data. It also increases velocity during the automation initiative.

During each Sprint, as the automated script is developed and debugged, it should be run through a code analyzer. This will enforce sound coding practices as well as undergo a code review by the Automation Architect. Automated scripts are assets just like the application’s program code and should be placed under rigorous version control. When the automated scripts are completed, a demo should be held with the automation initiative’s sponsor and other stakeholders for sign-off as ‘done’. This approach provides continuous delivery of automated scripts during each Sprint. The scripts demonstrate the progress made in automating the regression test suite.

Treating test automation like a project will prove successful if you apply agile principles and prepare the right documentation. Doing so will keep your automation activities on schedule and produce a maintainable automation solution.