Missed bugs...

Replies are closed for this discussion.

Replies to This Discussion

Permalink Reply by mak on March 12, 2009 at 8:10pm

Hi Rosie,

It has happened to me once and it was *very* visible inside the company. It has happened to any QA I know that takes ownership of their work. Our QA director (easily the best QA I have seen) once shipped a release on a corrupted CD :) that was found only after we mass copied the CDs and shipped them out. Another time one of our QAs misconfigured one of HP's network management tools and it went about trying to 'discover' some machines in the Pentagon :). Out IT director was contacted. Stuff happens.

My management is very understanding. The last time it happened my manager told me that if a goal is scored it is not *only* the goalie's fault.

Some things that should come out of this:
a) Move on.
b) Improve the *process*. Most QAs would love to have a better process ... the aftermath of a blooper is the best time to make a case for it. In my example, I wanted more rigorous code reviews, which at that point in my company, were a joke. I made a case for it and got it! Things are a lot better
c) Ask for a Xtreme Testing (similar to extreme programming) period. This should happen once the feature is about to be tested. The developer and QA sit together for a day or so and just use the feature. The QA then goes into testing. This helps on many levels including getting developers appreciate the difficulties in setting up test cases, hearing their thoughts as to what could be better (they are usually candid in this phase), getting new code added to the feature because the developer suddenly finds validating 700 unsorted metrics no fun ;), etc.

If your company has a "whom should I blame" attitude, then look out for other companies. I think software companies in general are improving when it comes to evaluating what are QA successes and measure them against the failures. E.g.: In my company the QA director (apart from the regular responsibilities) was instrumental in building us an excellent testing lab by clustering old developer machines. He also added some very cool hacks to ClearCase to make it simpler and better for us. It saves us a lot of money each year.

BTW, if you are young and have not faced this issue, pre-empt it by adding the below as your signature ;):
"If it works its the developer. If it fails it is the QA."

Permalink Reply by Tanya Martin-McClellan on April 8, 2009 at 9:30pm

First, cop to the fact that you missed it.

Next, trace it back to its point of origin and point out what kind of test activity might have caught it.

For example, if it was a problem with the requirements, maybe a requirements review would have caught it.

Instead of trying to defend, explain, and excuse the problem, help solve it.

I've found that this approach works very well, especially when you do it with a spirit of curiosity and cooperation within the team.

Permalink Reply by Mike Lockwood on April 9, 2009 at 11:32am

Couldn't agree more Tanya.

Admitting to the fact that the bug was genuinely missed but that you have done all that you can to highlight why it happened is very important to help build a strong relationship between you and the project team as a whole.

The important thing is being able to show that you want to learn from the situation and that you are implementing steps to ensure that similar bugs in the future will be caught.

Permalink Reply by phil kirkham on April 9, 2009 at 12:08pm

even better if similar bugs are never written again and are caught before the programmers fingers touch the keyboard...

Permalink Reply by Tanya Martin-McClellan on April 14, 2009 at 12:05pm

Agreed! Good point, Phil!

Permalink Reply by Tanya Martin-McClellan on April 14, 2009 at 12:08pm

Thanks, Mike!

When a team does this, they not only fix the problem at hand, but they also get better at solving problems together. Those relationships are very important to the team's ability to function.

Permalink Reply by Dustin Andrews on April 28, 2009 at 6:21pm

I often share this example. A 10 digit calculator has about 40 billion possible test cases. However, only about 100 of those tests are very interesting. So, I can find, say, 99% of the bugs running 0.00000025% or so of all possible test cases.

The cost to find the 1% of the remaining bugs is just staggering! So when someone askes "how did you miss that bug" you should be able to answer one of two ways. "It was a blind spot in the tests, we have fixed it." or "It would not have been cost effective to find that bug. The sun would burn out before we ran enough test cases."

Permalink Reply by Michael Bolton on May 10, 2009 at 7:32am

Actually, if you try to hang a number on the count of possible test cases, you're missing something.

- How many calculations of N steps?
- How many variations of timing between any pair of steps?
- How many successive tests before resetting the system?
- How many times could you create a loop in a test by reversing an action and then repeating it?

The sun would certainly burn out before we finished--but so would the rest of the suns in the universe. In addition, we don't know in advance what tests out of the infinite set are interesting or not (even if "interesting" meant only "finding a bug", which of course it doesn't.

All this reminds me of the fact that three weeks ago, I was at the QAI QUEST Conference in Chicago. As a speaker, I received a small gift--a pocket calculator. I entered 99999 X 99999, and pressed =. An overflow--just as expected. I pressed the ÷ key. Nothing happened. I pressed it again. The calculator switched off. I tried all this again; it switched off after the second press of the divide key.

Interestingly, another calculator--same model, same manufacturer--didn't exhibit the bug. So even after you had your infinite number of tests, armed with the information I've just provided, you might want to multiply that infinity by a number representing a sampling from the production run.

So tell me again... why haven't you found that bug yet? :)

---Michael B.

Permalink Reply by Amit V. Kulkarni on May 9, 2009 at 10:39am

I totally agree with Dustin.....

In such situation, I accept it by saying it was my mistake. Sometimes, its important as some of the missed bugs leads to many other bugs. It is not possible to find out all the bugs because of N number of reasons. I used to write it down the scenario so in next release if I have to check the same thing I know if I miss on this I will be hang :(

To encounter this:
1. Make sure that you have all the material available with you....all the scenario to re-create the missed bug
2. To convey the message loud n clear in a meeting with developers to let them know very politely that this part you need to code very carefully as there are some chances of bug may occur again....in this area....so they will take care of that from their end.
3. Visualize the worst case scenario....try to update your test cases to make sure that NOT TO MISS THIS TIME.
4. Need to be more attentive - ( ) -
5. The missed bugs if they are obvious then that is not a good sign....so be more careful buddy!!!

Ams: passion test

• ‹ Previous
• 1
• …
• 4
• 5
• 6
• Next ›
• Page