How did I miss that bug?
No matter what software testing techniques we apply, we have all asked ourselves that question.
Here’s a surprising answer: As software testers, we miss bugs — even those that become glaringly obvious after the fact — because we are hampered by our own biases. Our biases affect what we look for; how we go about designing, setting up and executing tests and how we interpret test results.
These biases can result in poor test performance and incorrect data interpretation. What we measure and what decisions we base on the data are driven by our own understandings and misunderstandings of the project and the goals of testing.
In this this tip, I explore the concept of biases and offer some software testing techniques to help overcome our individual biases. Let’s review some biases that especially impact testers.
The representative bias
The term ‘representative bias’ describes what happens when we judge the likelihood of an occurrence in a particular situation by how closely the situation resembles similar situations. Testers may be influenced by this bias when designing data matrices. As a result, they may not test data in all states or not test enough types of data. For example, if the code works for a new customer order as well as an order in process, why bother testing the code to make sure it works with a completed order?
The congruence bias
The congruence bias is at work when testers test only the "happy" path and miss the negative test cases. We fall prey to this bias when we underestimate the time needed for a test cycle. Often, when we test applications on which we are "experts," we are limited by the "curse of knowledge" bias and miss defects that we would have seen if we approached the application from the perspective of a new user.
But what is going on when we miss the obvious bugs, the ones that are literally staring us in the face? This, too, is attributable to a bias known as inattentional blindness.
Don’t miss the Invisible Gorilla
But how do we make sure we see the "Invisible Gorilla?" As testers, we need to approach our testing holistically; we need to focus our attention on determining if this is a quality product, as opposed to just tracing our test cases to requirements and executing all of our test cases.
In addition to executing test cases, we should do exploratory testing. Conducting exploratory testing prior to running test cases is useful because we have yet to make any assessments, and potentially developed biases, about the quality of the product. Lifeguards keep themselves alert and focused by switching stations during their shifts. Exploratory testing can be used in this way by working it in between our test runs.
Finally, we can use oracles, such as "What would someone who doesn’t know how this application works do?" And when we see something we don’t believe we are seeing, we should believe it and repeat it.
Source: http://searchsoftwarequality.techtarget.com/tip/Software-testing-techniques-Overcoming-biases