I recently saw an article that questioned whether automated testing was a good investment. They essentially laid out an argument that automated testing tools are too expensive, too fragile, and never give a worthwhile ROI. I am not a tester or QA specialist. Nevertheless, this seemed a gross assumption to make. I have worked with testing automation projects of various sorts and found a huge benefit from them at times. There are some excellent points made in the article. However, I think we need to look at how to achieve value from automated testing rather than throw it in the trash bin.
Regression Testing
The first thing that comes to mind when discussing automated testing is meeting regression needs. This not always a need for a project. However, any application that is regularly being changed will benefit noticeably from regression testing. I find that solutions that are regularly changing and being enhanced will run into quality problems if testing does not keep pace. When you throw in modern high-velocity approaches like Agile sprints, then automated testing becomes something you cannot do without.
A manual approach to testing is ok but think about taking the same steps every few weeks (to match pace with sprints) and the likelihood that errors will be made while executing the scripts. This is truly the sweet spot for automated testing. Any situation where you know that there will be testing performed in a large number of iterations will see the ROI from automation.
Find A Balance
It is common sense. If you do a series of tests and then they all change before they are executed again, then automation is useless. On the other hand, tests that will be repeated will save time on each iteration when automated. These statements assume that you take an all or nothing approach to testing automation. There are other options. Also, this is where we can find testing automation to be not only useful but arguably invaluable.
The best approaches to automated testing I have seen are built slowly and through good design. The goal of automated testing to cover growing percentages of the solution is perfect for the Agile process. Instead of completing a solution and then starting into an automated testing tool, the automation is built piece by piece and enhanced with each sprint. This can be accomplished through as little as a few automated tests with each cycle.
Find The Volatility
The critical point of the anti-automation article is a focus on how software changes. That is where we need to focus to achieve our goal of reproducible testing. There are usually going to be features and facets of a solution that will be stable through most releases. These are the areas we want to address in our automation. The challenge is that the static areas often can only be determined after the fact.
This is where we can take advantage of the repetitive nature of sprints. The testing team and related resources tend to have periods of a sprint where they are not utilized fully. There is coding being done, but nothing is in a state that can be tested. Instead of leaving those QA resources to twiddle their thumbs, these periods can be used to automate tests from the prior sprints. When you do this, you create a situation where every sprint effectively creates a backlog of testing for the next one. Every test completed becomes a candidate for automation. That leaves it up to the QA team to decide which tests are going to be the best options for automation.
Roll The Dice
There is no question that testing automation is a gamble. It is also an advanced skill to ask out of your team. I have come across very few people that can do this type of work. Also, they often are much more expensive than a typical tester. Nevertheless, adding a tool and even a single test automation specialist to your team can make noticeable inroads on your quality initiatives. When you avoid the all or nothing approach you open up options that can make you look like a hero.
