Requirements Based Exploratory Test Management: overview
So as a tester I’m a big proponent of Exploratory Testing. But as a test manager I know it’s hard to manage. Even harder to describe what’s done. Yet harder: to understand and describe what’s left. The hardest: to do it so that both developers and customers would understand. I don’t fool myself anymore hoping they understand the QA language. I know: everyone have their own interpretation of test case and bug statistics I’m providing. I've realized that my real problem is translation between business and development languages. I don't want to be lost in translation :)
RBETM: requirement based ET management
So what WE did is quite simple on a surface. So simple that it may seem dumb at the first glace. I beg you don’t judge too quickly and read details carefully. We have developed a new way to trace “Test Case”, “requirements item” and “implemented feature” to each other. I actually wanted to call my approach Traceability-based but changed my mind. Not yet decided.
My Context (in which RBETM works well)
1)I prefer Exploratory testing: where Test Cases are updated, created and removed on the fly: during test execution.
2)Developers have “Continuous integration”, but not vertical development (no user stories): somehow testable (probably with stubs and workaround) features are added, reworked and sometimes even removed on a daily basis, and developer want us to test them the next day
3)We (our management) are willing to follow agile principles, including: responding to change and customer collaboration which means requirements changes even all the time (not only once per sprint). But (management) don’t insist on any specific engineering practices (like XP).
Solving the traceability problem
The problem is well known: business, developers and testers have each their own “language”. I know at least two ways solving this issue: ensuring traceability (i.e. creating dictionaries) or vertical implementation by “User Stories” (i.e. creating an universal language, like Esperanto). I assert we have developed a different approach. We did something similar to what happens at border regions between two countries .
The benefits, or why we did so
In the current project my stakeholders are both development management and customers. It seems to me that biggest issue is progress reporting. Both of them want to understand when will be done with testing (and developers with bug-fixing)?
Test case statistics (pass/fail/not executed) did the work for us 5 years ago. It was when we had waterfall-like approach. It just don’t work in new (agile) era. On the other side we don’t have vertical development, i.e. there are no user stories that everyone (business, developers and tester) could set values to (priority, effort, risk). Our technologies are so complex that we have specialized team – each developer (and a DBA) knows and implements his own layer according to his own schedule.
So how do we keep our testing schedule in sync with horizontal development, and report progress against with vertical requirements (well actually there are horizontal requirements as well called non-functional requirements). It is not so simple – it requires each tester to make smart decision when deciding if a feature pass, fail or is not yet implemented. Right “Passed” don’t mean there are no bugs. It means good enough, though some non-critical bugs may be pending. Details? You have to wait a bit
Issues, Details and further conversation
I will describe technical details of my current approach in the next blog. However I want to admit that the approach is still under construction and I have several problems right now:
It appears I will have to create a few “Tester’s Test Cases”. I may them either “Advanced” or “System” or “Integration” or “Scenario” tests. I’ve not yet decided if I have to map them with requirements, but they are described in business language, so that stakeholders could review them. I’m not sure if I have to develop special pass/fail criteria to them or even different life-cycle.
I also have problem with non-functional requirements. Some of them (such as browser and screen resolution requirements) are covered by functional tests, other will be covered by performance tests.
RBETM: requirement based ET management
So what WE did is quite simple on a surface. So simple that it may seem dumb at the first glace. I beg you don’t judge too quickly and read details carefully. We have developed a new way to trace “Test Case”, “requirements item” and “implemented feature” to each other. I actually wanted to call my approach Traceability-based but changed my mind. Not yet decided.
My Context (in which RBETM works well)
1)I prefer Exploratory testing: where Test Cases are updated, created and removed on the fly: during test execution.
2)Developers have “Continuous integration”, but not vertical development (no user stories): somehow testable (probably with stubs and workaround) features are added, reworked and sometimes even removed on a daily basis, and developer want us to test them the next day
3)We (our management) are willing to follow agile principles, including: responding to change and customer collaboration which means requirements changes even all the time (not only once per sprint). But (management) don’t insist on any specific engineering practices (like XP).
Solving the traceability problem
The problem is well known: business, developers and testers have each their own “language”. I know at least two ways solving this issue: ensuring traceability (i.e. creating dictionaries) or vertical implementation by “User Stories” (i.e. creating an universal language, like Esperanto). I assert we have developed a different approach. We did something similar to what happens at border regions between two countries .
The benefits, or why we did so
In the current project my stakeholders are both development management and customers. It seems to me that biggest issue is progress reporting. Both of them want to understand when will be done with testing (and developers with bug-fixing)?
Test case statistics (pass/fail/not executed) did the work for us 5 years ago. It was when we had waterfall-like approach. It just don’t work in new (agile) era. On the other side we don’t have vertical development, i.e. there are no user stories that everyone (business, developers and tester) could set values to (priority, effort, risk). Our technologies are so complex that we have specialized team – each developer (and a DBA) knows and implements his own layer according to his own schedule.
So how do we keep our testing schedule in sync with horizontal development, and report progress against with vertical requirements (well actually there are horizontal requirements as well called non-functional requirements). It is not so simple – it requires each tester to make smart decision when deciding if a feature pass, fail or is not yet implemented. Right “Passed” don’t mean there are no bugs. It means good enough, though some non-critical bugs may be pending. Details? You have to wait a bit
Issues, Details and further conversation
I will describe technical details of my current approach in the next blog. However I want to admit that the approach is still under construction and I have several problems right now:
It appears I will have to create a few “Tester’s Test Cases”. I may them either “Advanced” or “System” or “Integration” or “Scenario” tests. I’ve not yet decided if I have to map them with requirements, but they are described in business language, so that stakeholders could review them. I’m not sure if I have to develop special pass/fail criteria to them or even different life-cycle.
I also have problem with non-functional requirements. Some of them (such as browser and screen resolution requirements) are covered by functional tests, other will be covered by performance tests.
Add a Comment
Find Us Elsewhere
Nice Things
Bloggers
© 2010 Created by Rosie Sherry
You need to be a member of Software Testing Club - An Online Software Testing Community to add comments!
Join Software Testing Club - An Online Software Testing Community