Add a Comment

Comment by Anna Baik on March 19, 2009 at 21:02

Hmm. Can I ask whether people are working on legacy systems, or new systems? I think that makes a difference to how useful screenshots are, as legacy systems have often settled into a relatively static design.

Also, it depends on the nature of the project, surely: if the project is "introduce a new product" then it's a lot more likely to involve visual design changes than "upgrade our database/billing system without any visible impact to customers".

Comment by Jamie Gagnon on March 19, 2009 at 8:35

Hi Ainars

Why would we update scren shots?

- i feel your test cases should be uopdated to reflect changes to the design.
- i feel that adding screen shots to test cases even for expected results can give a level of disassociation from the design. This could be dangerous in the event someone else not knowing, executres a test and looks at the test case, sees the screen shots and then ignores the design assuming the test cases and screenshots there in are up to date. You could end up logging needless defects.
-Why re-invent the wheel when the design document should have it all and be up to date?

If screen layouts change then yes instructions on how it works can change. but i still feel changing a few words is easier than changing screen shots.


You are right the human brain is a marvelous test tool. But with the human brain comes a problem. it doesn't really have an autopilot. Its not programmed to perform one specific task to the letter. Each individual thinks and performs actions differently and assumes things differently too.

Comment by Rob Lambert on March 18, 2009 at 19:58

Good follow up question and I guess it could be about the same to update both. I personally find it unnecessarily long winded to capture the screen shot, name it sensibly, archive it (store it), insert to script, check it is up to date and then to replace etc.

With scripts it seems somewhat easier. Whether it is or not would probably need some scientific experiment. Ainars, if you find it useful to use screenshots then I don't think any of us are saying not too. It's what works well for you and your team.

I personally (unless required by contract) don't write test cases anymore. I write guidance level test ideas (http://tinyurl.com/dbkcfj). These to me, encourage exploration. They also give you the metrics and they can also be expanded for those times when you need a fully blown test case.

I reached a point where I was so fed up with re-writing test cases that were out of date and never having up to date tests (unless I expelled huge amounts of effort keeping them tidy) that I stopped updating the incorrect parts and instead deleted the detail. What I was left with, was essentially a test title with some guidance. It works for me and the team perfectly. My blog link above covers this.

Comment by Ainars Galvans on March 18, 2009 at 17:34

Jamie, Rob, Ale: a follow-up question - so you say that maintenance costs for screen-shots attached are bigger than for step-by-step instructions? Why so? Because instructions are easier to rewrite or because instructions are still applicable even if UI layout changes? Why should we update screen-shots if it is only a layout that changed - I believe human brain is a perfect tool to ignore a small layout difference... (unless we direct the brain not to).

Comment by Jamie Gagnon on March 18, 2009 at 14:26

Oops sorry Rob I accidently deleted my original comment. but absolutely agree. If you are using the requirements and design spec during test execution as a reference point then (So long as you are aware that the current version of design and requirements is in your possession) you limit miscommunication and you also pic up any variants to the design.

When developers code, some have a tendency to look at a design and say " well I think it looks better this way" (not saying all do but there are some). If you take screens shots say from a prototype developed system. you them run the risk of saying "well since the screen in the test case is the same as what is in the final developed product it must be right" but if the change was not agreed and the design not updated. Then you actually have a defect.

Comment by Jamie Gagnon on March 18, 2009 at 14:18

Personally, I never use screen shots in my test cases for the following reasons

1. The screens can change from the design
2. High level of maintence is required to your test cases if you have to constantly apend screen shots.

I believe screen shots are good when it comes to logging defects, but that is as far as it goes with me.

On the other hand I sometimes use UML diagrams to describe processes.

I normally stick to written step by steps that are followed by the testers and make sure they have the most up to date copy of requirements and design documentation to assist them with the testing.

Comment by Rob Lambert on March 18, 2009 at 14:12

Jamie,

Interesting point you make "All testers I work with have the requirements and designs to hand while executing the testing so that they can verify and validate against them."

This is the crucial part to me, in that you are using the most recent (up to date and accurate at that moment in time) details and information to test. You also only have the one place to update this information meaning synchronisation and miscommunication is limited.

Comment by Brent Strange on March 17, 2009 at 5:25

Yes, but only when paired (and duplicated) with a detailed description and steps to reproduce. A screenshot is supplemental IMHO.

Comment by mak on March 16, 2009 at 21:21

I very rarely attach screen shots for the reasons below:
- Unfortunately in my company the look of the software changes often enough to avoid them.
- We usually automate the testing of older more stable parts

I do however maintain a set of (around 8) tutorials (apart from the official product documentation) of common work flows and use cases to help Tech Support answer common questions. Tech Support engineers update the screen shots of these tutorials for each major release. The value they get out of these prepared tutorials outweighs the time they spend updating screen shots. One common problem is that I am sometimes needed when the changes are drastic e.g.: when the features involved are new. I have to agree with the others. Over the last three years the work load to maintain the documents has increased to a point where I am beginning to question the ROI on these documents.

The one solution I tried to get over this problem was using video. I used and abandoned very quickly was video recordings. I thought it would be a good idea to record the tests being executed with voice explaining the test plan. I tried using CamStudio. I found out that preparing a video is a *big* thing. Its very detailed and very tough to actually put together a good, informative video.

Comment by Ainars Galvans on March 14, 2009 at 6:38

Thanks for links. But it is not about sketching the application screens (like input forms), is it? It is about models, flows, etc. I planned to cover it in the last blog ""do you Draw your test ideas?" in series. So thanks for link again I'll use it in the next blog.

• ‹ Previous
• 1
• 2
• Next ›
• Page