Replies to This Discussion

Permalink Reply by Iain McCowatt on January 4, 2012 at 0:48

How about: If deployed today, could the test team learn something by using the software?

Depending on your mission and approach, this could be more important that any of the items above.

• ▶ Reply

Permalink Reply by Marco on January 4, 2012 at 6:42

Sorry, I don't get it. Probably my English is not good enough. Could you pls explain what do you mean?

• ▶ Reply

Permalink Reply by Iain McCowatt on January 4, 2012 at 23:43

Stop for breath, try again.

You test to learn something. Either you are testing to learn something that will help you test, to learn about the quality of the software and if/how it adds value, and to learn about how it fails.

The development may not be complete, formal unit tests may not have been run and results documented, reviews may not have been held, the software may not be ready for full system testing (there will be blocking bugs, so that it won't be is guaranteed), there will be outstanding known bugs, and there will be missing documentation/help...BUT...if your mission involves learning (about the software, it's quality and it's bugs) AND it is both installable and executable...THEN there could well be value to starting to test.

The above list may or may not be worthy goals, but they should not be hard YES/NO excuses for not testing. The absence of any of the above (apart from installable and under some kind of configuration management) do not prevent testing from taking place, but their abscence may change HOW you test - for example if a particular component is still evolving you may choose to test a little more sympathetically and work more closely with the developer rather than simply throwing bug reports over the cubicle wall.

To summarize: if starting to test would add value, start to test. If not, don't.

• ▶ Reply

Permalink Reply by Torsten J. Zelger on January 4, 2012 at 10:00

I think what Iain tried to say is, that your checklist are good points to verify, but you will learn what you have missed in your SDLC checkpoints when the customer starts using your software.It is always a learning process "in context".

Your points are great for an overall strategy to make sure things are done in a structured way. Anyhow, when it comes to the question whether or not a release is ready to ship, then you may have to break down your topics into more details.

For instance, what does it mean "ready for the system test". You probably need to specify exactly when a system is ready for system test in your or the team's eyes. People have different meanings of when a system is ready. For instance, we once had a developer who said to me "if it compiles, I'm done. Testing is your job"....=;O)

When it finaally comes to shipping/deploying, then my hottest question would be this here

• Have our regression tests covered all the major/most important and known use-cases that the customers are using to make sure we didn't break anything.
  
  

Cheers

TOJZ

• ▶ Reply

Permalink Reply by Marco on January 4, 2012 at 10:10

Thanks for clarifying Iain's post.

Of course I have to break down the topics and specify more specific, what it means in detail.

And regarding the stone age developesr: they are still around us - believe it or not ;-)

Cheers

• ▶ Reply

Permalink Reply by Iain McCowatt on January 4, 2012 at 23:44

Nope that wasn't it, see above. Sorry for the drive by lack of clarity.

• ▶ Reply

Permalink Reply by John Stevenson on January 4, 2012 at 10:55

Have you thought about using some/all of the Software Quality Characteristics checklist created by the test eye people?

 

http://thetesteye.com/posters/TheTestEye_SoftwareQualityCharacteris...

 

Or use the sofwtare testing club wiki

 

http://wiki.softwaretestingclub.com/w/page/34704040/Checklists

 

• ▶ Reply

Permalink Reply by Stephen Janaway on January 4, 2012 at 14:09

I think the key activities of a Quality Gate should be checking that the results of the activities have met the required criteria, rather than checking that the activities themselves have been completed. So you should be creating yourself a results dashboard which can show the test results, status of the code check in, completeness of the helpfiles, number of open defects, etc. You can then set acceptable upper and lower limits, or just a single set of binary criteria which you use to decide if the particular user story, product release, etc has passed the Quality Gate.

Also be careful about using criteria which are too qualitative. Obviously some will need to be, but you should be looking at making them as quantified as possible.  And ensure that you have some way of checking later that they have been met (for example it's very easy to tick a box saying 'the software is ready for system test' since it's open to individual interpretation as to what 'ready' actually is). Better to think in terms of coverage, test results, run rates and pass rates, etc.

• ▶ Reply

Permalink Reply by James Christie on January 4, 2012 at 20:08

I'd be careful about asking for advice on a forum about the criteria that should be applied in a quality gate at your company.

Firstly, I hate quality gates. In my experience they suck. They apply arbitrary and unhelpful conditions that become a ticklist. People confuse governance with management and they focus on the process rather than the underlying reality.

My second concern is that the criteria that should matter are those that will help the testers to do a better job, and ensure that the developers do their work in such a way that the product can be evaluated effectively and efficiently in the particular context of your development shop in your company.

Of course there is no necessary reason why quality gates have to be an unhelpful bureaucratic, CYA style charade in the way I describe in my first objection. Let's leave aside the bigger question of whether testing should be integrated so closely with development that the notion of a hand-off through a quality gate becomes redundant. It is possible to set up quality gates so that they do add value, but you need to make sure that you take account of my second objection.

If you ask for advice from people with no experience of your company or development shop then the danger is that you will fall into the first trap. You will apply checks that look good, but are not necessarily useful to your people.

So the question you should be asking yourself is; what do I need to do here and now to ensure that we do the job properly? Not what works for other people, elsewhere. What works for you?

• ▶ Reply

Permalink Reply by Jeff Lucas on January 5, 2012 at 15:33

James points out a key concept - the team is responsible for quality, not the testers. I work in a small team and the coders create a build when they have finished something in the dev environment and want input on how it works in a deployed environment. In that case, my initial responsibilities become:

• Rapidly install and load baseline data (through a script or a database migration)
• Run a quick smoke test to verify core operation
• Send an announcement to the entire team that the deployed app is ready for general testing
• Run verification on the specific functionality the coders identified for feedback

Beyond those steps, it enters the standard exploratory process. The purpose of the above is quickly get feedback to the coders and minimise wasting anyone's time. If we start getting repeated failures (e.g. on install), we stop the process and work out how to avoid the failures in the future.

Also note that I didn't specify how the feedback occurs. It is up to the coders - word or mouth, email, Jira issues, or carrier pigeon. It depends on the context of the company, team, and where you are in development.

• ▶ Reply

Permalink Reply by Del Dewar on January 6, 2012 at 17:40

Hi Marco,

James has hit the proverbial nail on the head, so in that spirit instead of asking this forum what your quality gate should contain is like asking which flavour of ice cream is best. 

Your first question should be 'what do I need?' followed closely by 'how do I get that?'. There are other ways of finding out what's coming your way (and in what state) by other means than quality gates so don't limit your thinking to this vehicle alone. For instance, I have setup (relatively informal) product demos for handover purposes in the past which I found to be much more lightweight and informative than a bulky, time-consuming quality gate processes. 

In short, horses for courses.

Good luck

Del.

• ▶ Reply

Permalink Reply by James Christie on January 6, 2012 at 18:15

Best ice cream? My mother's rum and raisin (with a quite ridiculous amount of rum).

But then not everyone has a mother so generous with the rum. So ice cream's all about the context too. :-)

• ▶ Reply

• ‹ Previous
• 1
• 2
• Next ›
• Page