Many testers have observed that defects tend to cluster. This can happen because an area of the code is complex and tricky.
Testers will often use this information when making their risk assessment for planning the tests, and will focus on known Area called as 'Hot spot'.
There is also a contradictory concept to the defect clustering called pesticide paradox is that if the same tests are repeated over and over again, the same set of test cases will no longer find any new defect,hence test cases need to be revised.
We need to consider both of the above concepts depending on the situation.
Let us discuss both the phenomenon-
Defect clustering is based on the Pareto principle, that is 80-20 rule. It can be stated that approximately 80 per cent of the problems are caused by 20 per cent of the modules.
When we are testing a new software against the user requirements,defects will be finding at a large numbers in certain flow or area of the code which is more complex or critical. When the same software or code is to be tested again & again,after some changes or modifications,we will find the defects in initial few iterations of testing by identifying & concentrating on the 'Hot spot'.
Tester can focus the same area in order to find the more number of defects. It will reduce the time & cost of finding defects as by focusing on certain areas more bugs can be find in order to improve quality of the software.
But after certain number of iterations of testing,as the testing improves we see the defect numbers dropping, the most of the bugs will be fixed & the 'Hot spot' area will be cleaned up. Developers will be extra careful in a places where testers found more defects & might not look in other areas. Hence by executing the same test cases will no longer help to find the more defects. The test cases need to be revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.
Now we have two choices.
1.To write whole new set of test cases to exercise different parts of the software.
2.To prepare new test cases & add them to the existing test cases.
In the first case,we will be finding more potential defects in an area where we did not focused earlier or the area at which developer was not extra cautious as the tester was not raising the defects from these areas. But by neglecting earlier identified defect cluster, we are certainly taking a risk of giving less importance to the area which used be very critical in order to find the more defects in earlier iterations of testing.
In the second case,we can find the new potential defects in the new area as well as we can focus on the earlier identified defect cluster. But on the other hand the number of test case will become so large that it will increase the testing time & in turn increase the cost of testing. Hence too many useless tests may be an overhead.
To deal with this we can approach the middle path for balancing the disadvantages of both the listed choices. We can identify & remove the test cases that are not much important & have failed to detect any defect in certain number of test cycles.
For example, if you have 10 tests that cover the same area and none of them have detected bugs in number of cycles (for example,5 test cycles), then we should review them and reduce the number of test cases.
If test has not reported a bug in the last 5 runs, review it and verify its importance and weighting whether we should keep or archive it.
In this way we can keep the check on number of useless test cases without compromising with the quality as all the test cases covering important areas of the software will be retained. We have to follow this method regularly & keep updating/modifying our test cases whenever there is a change/modification occurs in the software.
In the initial iterations of testing,identifying the defect cluster is useful but it is not a good practice to assume that we can create a final test cases that will discover all the defects for once & for all.
Even though the created test cases have a very high coverage percentage & high rate of finding defects we still need to keep reviewing test cases regularly.
Add a Comment