There’s been a lot of talk in the testing community about ‘Best Practice’. From my perspective the conversation goes along the lines of how best practice is usually something promoted by tools vendors or consulting companies. That, in practice, there is no one size fits all best practice so anyone touting best practice is something to be wary of.

I can understand the argument. Vendors selling tools that work a certain way, will of course (need to) say that it does it that way, because it’s the best way of doing it. The steps, processes, data, reporting and whatever else the tool does is designed to represent best practice. It’s hard for vendors not to say this, what’s the alternative? They have a tool that’s pretty nifty but isn’t based on insightful research or unique experience, but will do the job, just not perhaps in a way that gives you an edge. They’re not likely to say that are they.

Likewise, consulting companies need a unique message, they generally want to say they also have insight and experience that no one else has, and so their way of doing things is the best of current practice. It gives them the air of success, of being the go-to crew for solving your testing problems.

But there’s a problem especially in our profession - the problems that we try to solve with these tools or consulting approaches are not static, but the ‘best practice’ they publish, promote and enforce is. Static, codified, best practice can’t take into account the unique context of the testing problems we face project to project, day to day.

The only practical way to address the change and unexpected circumstances we encounter on projects is to adapt the best practice. That means making a context sensitive application of the practices used. Now in doing so we acknowledge there’s not really a single definition of best practice. That there is only what we've decided to call at Test Hats (the consultancy I own and work in) - Best in-Context Practices.

Best in-Context Practices
Let’s briefly explore what best in-context practices look like in the wild. Every time we engage in a project we discuss with the Client their specific needs for their project. Sure, they come to us for testing services and consultancy, but the question is what do they need those services for? Typical things we need to consider include; what is it that needs testing, what constraints or risks is the project under, how confident are they about the level of quality of the development?

Do these seem obvious? They are. Consider the static and enforced nature of best practices and how they would address the above considerations. In summary they can’t, it's mostly just interesting 'project management' stuff yet it directly impacts how we approach testing. So, we can see that best practice is at best reduced to a framework around which the real best in-context practice is developed. The tools become just what they are, tools that form an element of the best in-context practices that are shaped and agreed upon to meet the unique needs of the project.

Best practice as it is often thought of doesn’t exist, context is everything and rightly so. The professional testing consultancy (and competent individual professional) combines proven practices, techniques and approaches, blended with their experience, utilising vendor tools, all in the context of the unique needs of the Client - to produce an approach that is by definition, “best in-context”.



Views: 308

Add a Comment

You need to be a member of Software Testing Club - An Online Software Testing Community to add comments!

Join Software Testing Club - An Online Software Testing Community

Comment by Scott Barber on November 24, 2011 at 20:35

As far as I'm concerned "best" is a superlative & as such needs to be based on something & supportable. In the US at least, all marketing claims that could be used by a potential buyer to make a decision must be both supportable and supported. Therefor, as far as I'm concerned, if "best" is not preceded by a statement like "in our opinion" or "according to X" it *must* be supportable with real & valid research that can be replicated by others. Interestingly, this is consistent with US law... more interestingly, very few companies get prosecuted under the "truth in marketing" act because the vendors have lawyers on staff & consumers don't, so unless enough consumers complain to get the attention of an attorney specializing in class-action suits the apparent "best practice" of deliberately committing and getting away with what I consider to be knowingly fraudulent acts by having more $ and better lawyers on staff/retainer will continue.


More of my thoughts on this term can be found on my blog, specifically:

Comment by Bj Rollison on November 4, 2011 at 15:18

Hi Mark, Perhaps I didn't express myself well. My statement was in reference to people in the trade who refute the notion of "best practice" based on the argument that some people assume a practice that has worked well in a given situation applies universally to all contexts (e.g. some people lack the capacity to determine when a specific practice applies and when it may not apply). I agree with you that what is more common is that sometimes people are convinced into buying magic elixer by snake oil salesmen.

So, IMHO its not necessarily the notion of "best practices" that is bad, it is the issue that some people drink whatever kool-aid is placed in front of them and blindly buy into something without critical thought or experience.

Comment by phil kirkham on November 3, 2011 at 20:07
thanks for the explanation, Mark - yeh, I've been on the wrong end of a Big Consultancy coming in and everything had to be done Their Way. ( which didn't work - but that's part of the plan, then they get extra work... )
Comment by Chad Patrick on November 3, 2011 at 13:06

Mark, you make my point for me.  STC is a great place for ideas and discussion.  I find it ironic that people bash the term 'best practices' because they ignore context.  When in reality, what you mean is that in YOUR context, you had a consultant sales team come in and try to force them down your throat.  In MY world, the term is thrown around a bit more loosely and basically means, "Here is an organic list of ideas that we learned in our experience, do with them what you will."

I guess to sum it up, with all of the issues I have in the testing world, having someone refer to something as a "Best Practice" isn't really on the top of my list.  Now that you've explained yourself, I can better understand your situation.  An overpaid consulting company came in, spent a week at  your organization and sold your executive branch on a solution they used on another client.  I think making that clear, elaborating on it and going into more detail on how you dealt with it would be a valuable bit of information for the rest of us.

I apologize if my post was discouraging, rather it was intended to prod you to go a little deeper.

Best regards.

Comment by Mark Crowther on November 3, 2011 at 10:17

Hey all,

Thanks for the replies.

The main ‘event’ that triggered this discussion in-house and the blog post was attending a presentation on a potential client site, who had been visited by a large consultancy beforehand. They’re basically had the consultancy pin them down with the our-approach-is-best and if you don’t realise it you’re morons (or have no common sense?)

We spent about 2 hours unpicking what had been proposed, which essentially didn’t help evolve a lot of the existing process and practice that the client had in place. It was intended to replace it for the convenience of the consultancy, that’s my bug bear with best practice touting. That alongside the fact that the work and review needed to declare something best practice is rarely done, certainly not by the organisations adopting it.

Chad, best practice is rarely touted as a simple list, apologies if I suggested that was what I take issue with. If it was just ‘best practice is always to write test cases and research has proved it’ the rolling debate on this topic might end. The best practice I’m against is that presented in terms of a full scale approaches wrapped around inseparable practices and then mandated as the way forward. Something that has little or no relevance to evolving  the way a client works to bring real improvements in their business. That's not consultancy.

BJ, I never said people (or my potential clients) have no common sense nor would I suggest that. Any potential client I go to see who has been visited by a big consultancy, are very often convinced of the ‘best practice’ they’re espousing due to their lack of experience in our field.  What’s more common is people assuming the ideas they’re getting off whichever consultant(cy) is far superior to their own. You would hope that people in organisations we encounter have the critical ability and necessary experience to assess the unique needs of their project against the approaches being pushed, but often they don’t, they often don;t have the time or authority either. A lack of common sense or otherwise has nothing to do with it.

Comment by Bj Rollison on November 3, 2011 at 2:14
Chad, you have hit the nail on the head. The folks who continually argue against the notion of "best practices" also seem to think people don't have an ounce of common sense. The general concept of a best pracitce is that it helps solve some problems in some situations,and that best practices can always be improved. What is funny is that the same people who denounce the concept of best practices rely and promote heuristic methods; which are in essence a best practice!
Comment by Chad Patrick on October 31, 2011 at 14:03
Could you list some specific "best practices" that you are referring to?  In my experience, the topic comes up when one person or group is working in a new area and we're looking to benefit from some other group's experience.  I've never seen anyone take the list and apply them without assessing whether or not they make sense.  I imagine if I did meet someone who applied someone elses practices without applying common sense, it wouldn't matter whether I called them 'best practices', 'good practices' or 'super fun magic juice'.
Comment by phil kirkham on October 28, 2011 at 14:34

"There are good practices in context, but there are no best practices "

- 7 basic principles of context driven school


© 2017   Created by Rosie Sherry.   Powered by

Badges  |  Report an Issue  |  Terms of Service