SPR Builds the Right Test Automation Strategy and Framework for SAP Upgrades

A construction firm underwent a significant regression testing effort each time they upgraded SAP, which they used to support business operations on projects. Regression testing took approximately 3 weeks back-to-back and required time from business analysts and other business operation SMEs. The firm needed a more efficient testing method and engaged SPR Consulting to develop an automated testing strategy and framework.

It wasn’t that the company didn’t have the tools for testing – they owned licenses of HP QC/ALM, QTP and Panaya. What they needed was a clear automated testing strategy, based upon their specific needs, future automation goals and resource constraints.

SPR Consulting conducted a test automation assessment that included CRM, MM, PS, PY, FICO, SD and HCM modules within SAP. The goal was to determine the optimal test automation framework and tool, implementation priority, timeline, and resources to build a test automation practice.  Our automation architect analyzed detailed information about the firm’s test cycles, level of testing, type of testing, test data, test scenarios, coverage needed, and resource skill sets for creation, execution, and maintenance of automated tests.

SPR recommended that the firm leverage test automation to close gaps in their smoke tests, which were being conducted in an ad-hoc manner due to limited time and resources. In some situations, this led to defects escaping into production. As an alternative, automated tests could cover core critical functional areas and transactions such as creating a project, creating a PO, hiring a new employee, etc. These automated smoke tests could be run every time there is code deployed to the QA environment. If any smoke tests fail, the root cause could be investigated – prior to critical defects going to production.

For integration regression tests, SPR recommended that the firm inventory their manual integration tests and prioritize each test case. Not all test cases are good candidates for test automation. Some test cases require too much effort and time to automate and provide little to no return on investment. SPR recommended that the automation effort be based on the test case priority with high priority tests first, then medium priority and finally, low priority test cases.

During the assessment, SPR determined that a modular/data-driven automation framework was optimal for SAP. Using this approach, the application functionalities were broken down into modules based on SAP transaction codes, e.g., VA01, P30, FV50, etc. Each module contains the necessary steps within a SAP transaction code and can pull data from a shared global data table, which is a centralized place to store all test data for the automated test cases.  Additionally, each module can run as an individual automated test or be combined with other modules to form integration tests or business process flow tests.

Advantages of using this modular/data-driven approach include:

1. Automated tests can be developed after the modules are built
2. Automated test cases can mimic business users’ activities
3. Modules and data can be reused across multiple automated test cases
4. Maintenance efforts are reduced and expansion of automation is easier

One of the most important decisions in developing a test automation practice is choosing the right test automation tool that will reduce execution time, increase coverage, and make test automation expansion easier. SPR conducted a proof-of-concept using two automation tools that work well with SAP. HP Unified Functional Testing (UFT) requires programming skills to write scripts, while WorkSoft Certify, a ‘scriptless’ tool, can be used by individuals regardless of their technical abilities. We prepared and demonstrated one functional test to the construction firm using each of these tools, pointing out the advantages and disadvantages of each. WorkSoft Certify was selected since the firm wanted non-technical resources within the automation practice.

The assessment and proof-of-concept established the automation test strategy and framework and selected the best tool for the firm’s SAP automation goals. The next steps were for the construction firm to build out (with SPR’s Consulting’s assistance) a data-driven, module automation framework along with ten end-to-end automation tests.  These ten tests serve as an example for the firm’s automation team to follow, enabling them to build additional test cases quickly and consistently.