Heads up on something interesting I saw just yesterday on twitter: James Bach said: "For testers: what is an alternative to test cases and procedures? Answer: a testing playbook."

Oh. What's that? I read on. And smacked my forehead. Go and read the discussion to see why. But to give some idea:
a testing playbook is a tool for sapient testing"
" I want something much less expensive to produce and maintain, while making my testing way better."
"A set of lists, tables, flows, combos-- a compact reference that allows me to do thorough exploratory testing."
It is an aid to test design as well as test performance."

It's one of those ideas that makes perfect sense as soon as you hear it - and I wish I'd heard it about 7 months ago. It would have helped so much with the current MIS project - so much information, scattered across dozens of different specs, so much new to us, so much left unsaid...

Writing test scripts hasn't really helped so much to clarify things for us, as when you're working on a project where the devil is in the detail - and the detail is buried in hundreds of procedural test scripts - it becomes hard to see where there are areas you have missed. Hard to see how you came to the decision to choose this combination instead of that, write three tests on this flow, and none on another. I'd tried to pull some of that decision making out into a single document by putting together a high level test design, but with only partial success, I feel. We'd made some steps towards a playbook - creating mappings, state diagrams, sketches of business scenarios and what data was generated and by which feeds. But how I wish we'd had this idea, to encourage us to pull it all together.

You live and learn. This is definitely an idea I'll be using in the future.

Views: 888

Add a Comment

You need to be a member of Software Testing Club - An Online Software Testing Community to add comments!

Join Software Testing Club - An Online Software Testing Community

Comment by Chad Patrick on February 19, 2010 at 22:55
In regards to Michael's comment, I'm not sure why you would use 'alas'. I see little difference between what I've seen described as ET and how I manage our scripted test cases. The 'playbook' I use is my Test Strategy document. It lists the key areas of code, architecture, environment, etc... and what I deem are the critical paths as well as observations about other areas at risk that may not be clearly defined in the requirements.

On most engagements I've worked on, we've been fortunate to either have documented requirements or at least been able to assist the client with what they feel are the functional requirements. Scripted testing is generally for what you know during the analysis phase. If I know the client expects a certain behavior, why not develop scripted test cases to test that?

In the case where we're testing an upgrade, it's a little smoother because we have access to the software. In the case where it's a new product or a heavily upgraded product there will always be things that the development team throws in there that are undocumented. Ideally your tester(s), scripted or exploratory, will be observing the software's behavior during execution and I encourage mine to make observations on things that are outside the scope of the script. These are consolidated and presented to the client as risks and when possible, this leads to the generation of additional scripts. If time is a factor, then we simply test them and document the software behavior for a final review.

I think the prompt book is a great comparison and I'm not so certain it breaks down regardless of if you're scripting or using the ET approach. Many great one-liners were not originally part of the playwright's original script, rather they come naturallly to the players during rehearsal because it feels right. But then again, there are some great one-liners that are in the script.

Thanks for your comments. They're always thought provoking.
Comment by Rosie Sherry on February 19, 2010 at 20:23
I like the sound of a 'play book'. Makes it sound like fun and interesting, which is a good thing. Quite the opposite when I think of Test Cases ( = boring).
Comment by Michael Bolton on February 19, 2010 at 8:02
There's a similar concept in theatre, the prompt book. This is the stage manager's book, a kind of production manual for the show. It contains all the sound, lighting, and effects cues; prop lists; lighting plots; diagrams of the stage and the set, in plan and in elevation; the roster of people working on the show with their contact information; everything that one might want to know about the show. I've maintained such a document; in an earlier career, I was

The comparison breaks down, alas, because the biggest part of the prompt book is the heavily annotated script for the show, which is by design not very exploratory in nature.

---Michael B.


© 2017   Created by Rosie Sherry.   Powered by

Badges  |  Report an Issue  |  Terms of Service