That automating testing is a good thing to do is a truism that simply doesn’t need to be justified anymore and barely a week goes by when we’re not discussing with one organisation or another their journey to automated testing. One of the most eye-opening revelations of those discussions is when we talk tooling. It turns out that a small majority of the organisation’s we talk to have already bought all the automation products they need, in many cases they’ve even initially implemented them successfully with a whole mass of automated scripting – but initial success turns to frustration as error and redundancy creeps in and soon there’s a steady shift from automation back to manual.
P2 Consulting believes that ultimately the challenge for sustaining automation is one of reusability – ensuring that no matter what changes are made to your application, infrastructure or business processes your automated systems keep on working with minimum fuss and overhead.
The devil’s in the detail:
To tackle this problem, it’s important to understand how it develops. We believe there are 3 main failure points for automated script execution:
1. Change has been made to the application under test – this is particularly relevant for GUI scripts
2. Automated scripts have not been maintained – leading to error and redundancy
3. The application has been re-platformed – automation tools are generally configured to work with the specific platform on which the application operates.
None of these challenges is insurmountable but to tackle them you need a good understanding of the automated testing approach that is selected. Generally, there two known and accepted approaches used to develop automated test scripts:
Native Scripting: Native scripting is where the selected test automation tool is used in its scripting format to deliver test automation. This may deliver scripts quickly but is generally written by an automation resource that has the ability to script (write code) using the tool selected.
When building automated tests using a native script approach a test automation tool is selected (such as Selenium or UFT). Using the associated scripting language such as Java automated test scripts are created.
Test Framework: Here a test framework (sometimes called a wrapper) replaces the native scripting approach. Critically here a specific syntax is used that is readable to aid test automation (ensuring coding experience is not necessary).
A test framework is far more ‘user friendly’ as the automated script is readable and, once the syntax is built, manual test resources can be trained to both read and deliver automated test cases.
The tooling that underpins a framework approach is more complex than for native scripting. Two tools are required – a test automation tool (which becomes the script driver) and a test framework (such as Specflow or AXE). Critically the automation test team then develops an ‘automation function library’ within the test framework that then utilises the automation tool to drive execution in the AUT.
A framework for success
So, if we return to our major areas of problem and redundancy we see that the solution lies in employing an effective and user friendly test framework as an intermediary between the applications to be tested and the test automation tool (driver).
1. Change has been made to the application under test – where only an automated testing tool is used a change to, for instance, the ‘login screen’ will mean all relevant testing scripts containing ‘login’ will need to be laboriously updated. If a framework approach is used then only the automated function that executes ‘Login’ needs to be change
2. Automated scripts have not been maintained – Most automation is completed by test automation experts who are employed for a finite time to deliver a project automation pack, where a native script approach has been used and if those scripts are not well notated it is difficult to understand how those scripts have been created, what they will test, how they should be executed. With a framework approach the native scripts do not need to be maintained as the function library is called to execute the test scripts. In other words, the functions need to be maintained – not the scripts. A far more manageable activity with a dedicated internal team
3. The application has been re-platformed – As mentioned selected automation tools are often configured for a specific platform so re-platforming can have a significant effect on their operation. In this case the framework approach may not take away the need for the automation tool to be changed but the function library may have some reuse by integrating with the newly selected automation tool (driver).
The benefits of an effective test framework
So, for us, the solution to ensuring your automation tool does not become shelfware is the implementation and sensible maintenance of an effective test framework. The benefits are obvious:
1. Automated test cases that are easier to read and can be created and executed by manual test resources
2. Automated test cases that are easier to maintain by making changes to functions rather than automated scripts
3. Test automation resources can maintain a well-documented automation framework rather than a myriad of test scripts
4. Re-platformed applications may be able to use the exiting framework function library employed by a new automation tool rather than needing to fully re-script all automation test cases.
If you would like to learn more about test automation and the advantages of a framework approach, we at P2 would be happy to work with you to select the best tools that deliver real benefits and return on investment.