Software Testing and the hammer and nail approach

If all you have is a hammer, every problem looks like a nail.

It is sometimes a similar situation with software testing and specifically with test automation. This may not be much of an issue for single-product companies but when your organization produces a range of products and has multiple teams of testers handling these different products then the likelihood of hitting the hammer and nail problem arises.

One might say that the problem is prevalent more in a centralized testing structure than in a decentralized structure. The issue I am alluding to is the mandate to use a common test tool, mostly for automation since that is an area that everyone would like to see standardized but poses most challenges to standardization.

I have come across many instances where there is the attempt to identify a common tool that can satisfy all the requirements for automating various products being tested by different test teams. In some cases, the products would be as different as chalk and cheese. For example, there was this situation where we had a suite of products addressing different customer needs. This suite has a thick client application written in Java, a Win 32 application, a complex Ajax based web application, a few server products that you interact with using CLI and APIs, some middle ware products and few mobile applications.

Trying to find a common tool to handle all of the varied requirements for automating the different products could lead us to three possible ways of solving them.

1. One, you talk to a few tool vendors who would naturally promise the moon and claim that whatever tool they are selling can automate every kind of application that ever existed and will come into existence in the undetermined future. You could take this option like many folks do and purchase expensive tools and licenses and then mandate your testing teams to go use these (and only these) to automate in a standardized manner. Simple solution using a brute-force approach.

2. The second approach may be to take the compromise route. Here, you realize that a single tool may not be able to handle all the unique requirements of automating each product and go ahead to procure a tool that probably results in being the lowest common denominator solution. The tool ends up being  a good enough solution which essentially means that it is neither good nor enough from a testing perspective. However, the organization gets a standardized jack-of-all-trades kind of tool that all groups could use with varying degrees of effectiveness.

3. A third approach could be to take the time to understand the specific requirements and needs for automating each product, perform due diligence in identifying & then evaluating the tools that best fit the specific automation requirements before deciding on what tools to procure. This may result in having to procure more than one tool. If your products have similar automation requirements you may end up with needing just one tool but in cases where you have products with differing needs similar to what I stated as an example earlier, you might realize that more than one tool is needed to perform effective test automation.

Effectiveness , is the key here. Ultimately you automate not for the sake of automating but to support your testing campaign and deliver value. Trying to force teams to use a specific tool that does not fully support their needs is akin to the hammer and nail solution. When all you have is a hammer, then every problem is dealt with as you would a nail. Sometimes it is the right thing to do and in many cases it may not be the optimal approach to follow. In testing, such an approach could lead to teams bending their testing and automation practices around what the tool is capable of while ignoring other possibly important areas of the product which are harder to automate using the tool. The automation tool should not dictate how & what you test. It should truly be a tool (amongst other tools and techniques) in your arsenal enabling you to tackle the challenges of testing your software.