Putting Automation Solutions to the Test

Challenges abound with data. Data can be sensitive and must be handled appropriately, and identifying and obfuscating sensitive data can be difficult and time consuming. In many cases, only a few people understand the data and it can be hard to determine which data is the right data. Waiting for proper data can take weeks, and masking and stripping PII from sensitive data can add to that timing.  

The attendees then brainstormed solutions, and one solution rose to the top: data governance. Setting rules who can touch which data, data segregation, appropriately reverting data at the end of an automated test, and data assignment or reservation can help make test automation run more smoothly. Attendees agreed that a mindset shift is critical, and that upskilling and training are often necessary.  

Automation faces even greater challenges when only Quality professionals own automation. For success with automation, a whole-team approach to ownership is vital. Share on X Attendees highlighted struggles with lack of up-to-date documentation, lack of ownership around application and code quality, and lack of well-defined acceptance criteria for releasing a new build to QA or production.  

One resounding challenge we heard again and again was time. The race against the sprint cycle makes achieving quality software difficult, with quality often sacrificed to achieve a greater speed to market.  

When brainstorming solutions, it became clear that team building and leadership were essential in tackling this challenge. A top-down quality culture and mindset is critical to holding all parties accountable for quality, while bottom-up team building can foster a shared ownership of quality and greater respect between team members. Getting developers and testers to tackle problems together can go a long way. Setting metrics and goals can also drive accountability and entrench quality in the team culture.  

Attendees also participated in a short-form solutioning activity where they could volunteer to sit in the hot seat (a bar stool) and ask the group for help with particular challenges. We covered three challenges during this activity.   

Increasing Test Coverage 

Attendees suggested documenting where you’re currently at in coverage and setting a target or goal. Not everything can and should be automated, so determine what level of automation you want to reach. Always keep the user in mind when determining coverage, and lean into test driven development through pair developers and testers together for technical and business expertise. Finally, define meaningful metrics to gauge how automation improves coverage and decreases defects.  

Experts in Everything 

The QA professionals in attendance brought up a growing trend: QA resources are expected to be experts in everything. Key solutions were to build a hierarchy of subject matter experts and go-to resources in different topics, have SMEs share their expertise in Lunch and Learns, and to build new requirements for testing (e.g. accessibility, mobility, performance) in as user stories so they get prioritized along with functions and features.  

Onboarding Contractors 

In situations where a vast amount of domain knowledge exists, onboarding new employees and contractors can be especially challenging. Documentation is critical to achieving an efficient onboarding process. Domain knowledge only goes so far and should be shared through short introductory videos or other documentation, knowledge transfer activities, and pairing developers and testers who have the domain knowledge with new automation contractors to help quickly transfer that knowledge.  

Throughout the afternoon, testing professionals engaged with each other, working together to solve the common problems most are facing. Not only did the attendees leave with inspiration and solutions to try, they also left with new connections within their profession, enabling them to continue the dialogue long beyond our testing leadership event.