Which is more useful - detailed or general test cases?

Replies to This Discussion

Permalink Reply by Linda Wilkinson on June 22, 2009 at 1:18pm

When I'm writing a test case, I assume I'm writing for someone that has never worked on the app before.

I do not specify keystroke-by-keystroke conditions, but my test steps will help someone navigate through a workflow they may not understand.

In your case, if it were in my environment. I'd write a separate test case for each configuration, since the flow is different depending on configuration, and my pre-reqs (machine is configured to do X) would be different. I'd likely keep the steps as general as I could, while still providing enough information for someone new to navigate through the differing flow and verify some important checkpoints.

I was careful here to say "in my environment". If everyone on your team already understands the basics and doesn't need much help to navigate through the workflow or understand what needs to be verified, I'd go as general as I could. The more specific the testcase, the more maintenance is required when app changes come through.

- Linda

▶ Reply to This

Permalink Reply by Michael Persson on June 22, 2009 at 1:33pm

Yeah, the thing is, I was using very specific test cases at first, like this:

1. Press !-button. Info window appears.
2. Press something else. Info window disappears.

Then I realized that I know quite much about the machine, so why have these detailed steps? So I did some rewriting of test cases, simplified some of them and changed the order of them. But now I don't know which way to use...

Another aspect of not having to much information in the test cases is that I most of the time need to print out the whole sheet of test cases. It's great having them on paper when working, but even better would be to never print them at all. Just use a laptop or something...

To summarize this, I like the detailed version, since it's possible to give it to anyone and they can follow the test cases, just as Linda says. But at the same time I want it to be easy to work with...

- Michael

▶ Reply to This

Permalink Reply by phil kirkham on June 22, 2009 at 1:46pm

Try reading What Is a Good Test Case ? by Cem Kaner

or this blog post How Much Detail Should I Write In My Test Case - I saw this talk at a recent SIGIST and it gave me some food for thought

▶ Reply to This

Permalink Reply by Michael Persson on June 23, 2009 at 6:56am

Those sources were great! The blog post was what I thought about, but in a much more structured way. I think I know how to do it now. And one shouldn't be afraid of mixing levels of detail in a test suit I guess...

▶ Reply to This

Permalink Reply by Michael Bolton on June 23, 2009 at 8:08am

I like the detailed version, since it's possible to give it to anyone and they can follow the test cases

What are the chances that the test case will be given to anyone?

Would the test ideas be given to anyone, or to someone with sufficient skill to test well? Testing well involves the capacity to learn rapidly, after all. Is the test case (or, more likely, many, many cases) the best place to instill that learning? Or might it be just as productive (and much less expensive) to produce concise documents that guide the tester?

Might the excessive detail lead to inattentional blindness?

Might the natural variation of humans (combined with skilled testing) an asset, rather than a risk?

Might it be the case that you don't have to provide anything but the most general advance guidance, and instead focus on recording test results?

Might it be an even better idea to automate the tests that you're describing, such that you can provide enormous variation and long sequences of tests?

Note I don't have answers. Only you can supply those. But these are questions that you might like to consider.

---Michael B.

▶ Reply to This

Permalink Reply by Joe Strazzere on June 22, 2009 at 1:57pm

As with all writing, you need to understand your audience before you write.

Who is the audience for your written test cases?
What do they know?
Would they be best served by specifics? Or by more general test cases?

I like to make tests cases as general as I can, and only be specific when actually needed.
But I can't know the "actually needed" part, unless I know my audience.

▶ Reply to This