How do we measure the value of testing - coderskichen

How do we measure the value of testing?

I was asked recently if the number of bugs found is the best measure of the value of testing.  Honestly, I had to think about this for a bit. But after a few minutes I decided that counting the number of bugs found is actually a really poor way to measure the value of testing.  

Consider low-level testing (unit and API testing).  When your developers write tests do you want them to log formal bug reports for every error they find?  Probably not, this is a development activity, we want them to deliver a complete set of passing tests for each change they make while fixing the bugs they find along the way.  How about for the functional testing on the integrated product. Are you happier if that testing finds 100 bugs with a new feature or 0?  Well, 100 might mean that the feature was poorly tested before it was sent to QA, but what does 0 mean.  Does it mean that the testers did a bad job, or that the quality of the feature was high before it went to QA?

Measuring the value of testing by the number of bugs found is the completely wrong metric.  The true value of testing is to formalize correct behavior in a repeatable way.  When someone creates a set of tests with automatically checked expected results, that can be run every time the code changes, they are contributing to the infrastructure of quality.  A well-designed test will provide value for years, and pay for itself many times over.

Every time a test fails it provides value, it prevents a bug from getting into the code base. 

Share:

Share on linkedin
Share on twitter
Share on whatsapp

Leave a Comment

Related Posts

Testing AI-based systems
Martin Heininger

Testing AI-based systems

AI-based systems are becoming popular. However, the broad acceptance of such systems is still quite low in our society. Trust in and acceptance of AI-based

System testing in virtual environments
Marcus Eggenberger

System testing in virtual environments

When it comes to system level testing, virtual environments offer several strong benefits for you. In this blog post, I want to highlight some of

Improving test efficiency
John Paliotta

Improving test efficiency

Effective software teams are always looking for ways to make their members more efficient. They realize that improving the development process is an good way

Debugging without Security risks
Niroshan Rajadurai

Debugging without security risks

Security researcher Michael Myng found the keylogging code in software drivers that were preinstalled on HP laptops to make the keyboard work. He discovered the

Quantifying the cost of fixing bugs
Lynda Gaines

Cost of fixing vs. preventing bugs

When you think about improving software quality, your initial thoughts might be the cost of investing in new tools and engineering labor to implement them.

Hey there!

Subscribe and get an email every time we’ve got a new quality piece on here.