In a “textbook” Agile development process, there are defined roles for a Product Owner (responsible for system direction), a Scrum Master (leads and guides the team through the process), and a Team Member (creates and delivers the system). There is no role specifically defined for a “Tester.” For better or worse, it is thought that when the time comes in each sprint, everyone will take responsibility for handling all of the software verification and validation necessary to put out the highest-quality product possible.
This approach makes assumptions about each particular role that are rarely true in real-world Agile implementations:
ASSUMPTION: The Product Owner (and other people with a business interest in the software product) is technically involved on a daily basis. They attend daily standup meetings, review tasks, and define the acceptance criteria by which each User Story can be “completed”.
REALITY: The primary responsibility of every business person, every Product Owner…is growing the business. Sure, they will provide as much input as they can, and attend demos, retrospectives and other meetings whenever possible. But they are also responsible for supporting the current customer base, and expanding into new markets. They work with sales, marketing, and finance organizations to provide input to the business forecast. The Product Owner’s focus is split 50/50…at best.
The role of a software test organization is to be the internal voice of the customer. Before a customer gets the first beta version of a product, prior to any chance to “kick the tires” to see if it works correctly and does what they need…it is the software testers who understand what the product is supposed to do, write acceptance test cases to prove it, and find the bugs and validation issues. The tester fills in gaps in the Product Owner’s defined responsibilities.
ASSUMPTION: The Scrum Master has sufficient technical input, balanced between development and test acumen, for all of the story points and task estimation to be as accurate as possible. The project staff level allows for programming pairs, test-driven development, and the ability to handle last-minute changes with ease.
REALITY: A 1:1 ratio between development time and testing time has never happened, and never will, for a number of reasons beyond anyone’s control, and are nobody’s fault. Every team has more developers than testers, so when the team does story point estimation in an Agile process, the estimate will tilt in the direction of development. If there is a slip in development because a story is more complex than expected, it is an issue to be tracked and managed. If there is a slip in testing that holds up the release of a product to market, it is a disaster.
The role of a software test organization is to keep things balanced, so the Scrum Master gets the most accurate estimates, task definitions and progress updates available from a testing perspective. Bugs are tracked, prioritized correctly, and verified within the context of other testing efforts.
ASSUMPTION: Team Members are equally adept at designing and writing new production code as they are at designing and writing an automation test platform. It is fairly simple for team members to switch between a focus on “making” a system and “breaking” it.
REALITY: What software engineers like to do best is to take on challenges, develop algorithms and data methods, and write new code. Their focus will always be on making it work instead of testing it, stressing it, breaking it. If you ask them to estimate the complexity of a User Story, they will do so from the perspective of “how much work is it to write this code?” not “how difficult will it be to define and execute a test environment?” To ask them to keep an even split between the two is an impossible exercise in double-think.
The role of a software test organization is to determine the best approach to testing each feature, in each sprint. The complexities of selecting the right programming approach to test automation are often equivalent, and always different, than developing new code. Asking software developers to switch between the two is inviting one area to be neglected.
This is not to say that developers do not provide a valuable foundation into the testing process. Thorough unit testing is the bedrock of quality software, and the development team is in the best position to make that happen. Only if the conditions of each module of code are tested with enough detail to provide confidence that changes to the latest build did not break the previous one can the Agile process move forward with the velocity expected of the team. There is rarely time for looking back built into each sprint.
Beyond unit testing, however, paradigms and concepts in test automation change dramatically. Even the most straightforward automation platform, such as Selenium, requires combined experience in coding, test architecture, and metrics reporting. Getting the most out of a business-driven test platform such as Cucumber, or an integration environment such as Microsoft’s Coded UI, takes an automation engineer that is a unique breed of programmer.
Not having someone focused on manual test planning and execution is another issue with “textbook” Agile. There is a specific skill, borne from years of experience, in taking a User Story and being able to write the “correct” number of tests to sufficiently cover positive and negative conditions, without wasting time by repeating tests or having too much coverage of low-risk areas. Not all tests can be automated, and it takes a well-thought out manual test plan to fill in the gaps. System performance, scalability, usability, security…all specialties within the software profession. The assumption that team members within an Agile team can switch into these roles and do what needs to be done is simply untrue.
Within the Testing Practice at SPR Consulting, we embrace the Agile Manifesto, and the philosophy of integrating within a team that can rapidly turn out quality software. But first and foremost, we believe in the value of having software test experts, not interchangeable cogs.