What valuable tools are in your exploratory testing arsenal?

I have found that one of the most valuable tools I use while exploratory testing is a blank text document. I use this to create a map through the paths that I have selected to test during the current test cycle. This helps me in a few different ways.
The first way this document helps me is that it keeps me thinking about the logic of the program that I am testing. I may be asking myself questions such as “Is this step necessary?” “Is this confusing?” “Is the navigation user friendly?” “Is anything missing?” This enables me to question the programmers, product managers, and myself in regards to domain knowledge. “Do I need more domain knowledge in order to understand the business logic of this application?”
The second way this document helps me is that it shows me where I have already been in the program. It helps me determine the range of coverage that I have tested, and what areas need more testing or have yet to be tested.
The third way this document helps me is that it enables me to more easily reproduce difficult defects. When I ask myself, “what was I doing before this program crashed,” I can look at the document and have some idea of what to check into. This saves time.
The fourth way this document helps me is that when the people who matter on the project ask me what was tested, the information is all right there. I can easily inform others of what was tested during the test cycle.
The fifth way this document helps me is that it keeps me accountable to myself while in a test cycle. Exploratory testing does require self-discipline and documenting what I test provides me with the ability to self-manage my testing sessions.
What valuable tools are in your exploratory testing arsenal?

▶ Reply to This

Replies to This Discussion

Permalink Reply by Dylan Lane on August 12, 2008 at 6:12am: My most valuable tool when doing ET is my team. Together we constantly push each other to critically analyse our assumptions and results.

Aside from that, I like to use Python to generate quick throwaway test scripts to test cases which would be difficult to do manually.; ▶ Reply to This

Permalink Reply by Michele Smith on August 12, 2008 at 9:18am: Testing as a team is definitely valuable. Diversity in technical skills, testing experience, product knowledge, domain knowledge, and each tester's thinking processes will expose more bugs quickly. Team testing allows testers to learn new things from each other. This sharpens individual skills. And let's not forget how much fun it can be. The excitement over finding a big bug when in a group setting is much more heightened than when it is found alone.; ▶ Reply to This

Permalink Reply by jaffamonkey on August 12, 2008 at 1:42pm: The mind is a no-brainer - of course, a good logical (but open) mind is essential tool for testing. Exploratory testing can be necessary for a number of reasons, but the two main reasons for me are: learning the system and understanding the constrictions/limitation. To that end, documenting your progress and findings is key, but then it always should be during testing. For test tools, I find Selenium great web recorder tool, for quickly generating automated test scripts. And a blogging tool (like Wordpress) to use as a jotter to takes notes. I like this method, as then I can seperate out information and relevancy using tagging.; ▶ Reply to This

Permalink Reply by Simon Godfrey on August 13, 2008 at 11:17am: http://portableapps.com/; ▶ Reply to This

Permalink Reply by Michele Smith on August 13, 2008 at 8:15pm: Thanks Simon!; ▶ Reply to This

Permalink Reply by Tony Bruce on August 14, 2008 at 10:28am: I second that, a blank text document is my best friend, Notepad is always open on my machine. I've used it since my days on help desks for taking notes while on the phone to my current days as a tester and still using it to make notes, documenting steps etc.; ▶ Reply to This

Permalink Reply by James Lyndsay on November 20, 2008 at 11:40pm: Here's an idea of my current set:

- for recording visual stuff: I most often use a camera, which allows screensnaps and brief audio+video without being platform-dependent. For screenshots, on a given platform, I use SnapzPro on Macintosh/OS X, and HyperSnap on Windows. I've been know to rig up a video camera to watch keystrokes and screen. I also use Mouseposé, which makes clicks obvious and shows keystrokes (including modifiers) in a big translucent bezel.
- an aside: I also use Mouseposé when I'm showing people what I'm doing, and even to help me get visual feedback on the buttons I'm pressing. Butterfingers = happy accidents.

- for tools that specifically help with recording manual exploratory testing (ie records keystrokes and timings, allows annotation of screen movies etc.), I've used SiriusSQA's TestExplorer and BlueBerry's BBTestAssistant. For my purposes, they're often a bit heavy and windows-centric, but you may find one or the other is just what you need. You can try them yourselves, as both have a great attitude to limited-use trials. Spector - the outrageously intrusive spy-software - was often put forward as an alternative before tester-oriented tools became available. It seems un-disruptive, and offers comprehensive monitoring, but I found it hard to use for testing. You may find the terms of the license prevent you from using it well in a test environment, and there are difficulties in saving the information you've captured. Finally, it's so very much oriented to sneaking on spouses and employees that having the license makes me feel slightly creepy myself.

- for recording what I do, I was pretty-much paper based until a couple of years ago. I now use a dual system, keeping most of my scribbles and diagrams on paper as an extension of my thought processes, but using OmniOutliner to keep track of bugs found, to put timestamps on observations, arrange stuff hierarchically / foldably, and to allow me to search my notes.

- for monitoring system activity, I use "top" on UNIX-based systems. I've recently discovered the joy of Process Explorer and perfmon (when I got the logging finally turned on) on windows. On the Mac, Activity Monitor is reasonable, but Instruments (a wrapper for DTrace) is for serious work. Crash monitors for specific applications/OSes are good, too, as is a knowledge of where the logs are hidden (and how to extract them).

- for looking at differences, I use unix/windows tools comm and diff. Unix tools can be introduced to windows environments with Cygwin, UnixtUtils or many others. You could also try kdiff, which is pretty comprehensive and runs on everything, or FileMerge (in xCode) on the Mac. It's good to know one's way around regular expressions, so here's a link to ilovejackdaniels' great Regular Expressions cheat sheet. If you need to do specific Windows Registry checks, Process Monitor and TestExplorer will help you out.

- for influencing system activity, it's got to be Holodeck - the freebie on the back of "how to break software security" is less buggy than the version on "how to break software", and (being free) is $$$ less than the retail version. Roll on the day when developers expect this sort of capability to be built into OSs.

- for input data, I often have a library of useful data about the place - large files, empty files, different sizes of different formats of picture, files that have meaning to something I'm testing (ie an Excel file for testing within MS Word). I sometimes use Bach's perlclip for creating data. I use Excel for creating test data to fit requirements, and load it into SQL tables with CSV

- for output data, I primarily use Excel for most parsing/analysis work - with a little bit of fiddling to get in amongst datasets that break Excel's 65K row limit. I very much prefer DataGraph for exploring datasets graphically. It's Mac-only, but IMHO worth buying a mac for.

- for automating , I use whatever's lying about at the client's site. Last time I bought a tool, it was Vermont High Test, so my licence is a bit out of date these days.

- for web testing, I push hard for Firefox to be within the browser gamut, and use FireBug. iMacros was in my toolkit for a while, and I wasn't over-familiar with it, but I've recently tried it out for loading web forms, and it's fine. The firefox version is still free (as in beer).

- for cross-browser testing, I use the free web service netrenderer if I've not got my kit with me. It opens a publicly-accessible web page in the browser of your choice, takes a screensnap, and shows it to you.

- for putting load into an app and measuring output (although not graphing it, because Excel's far better, and DataGraph better still), I use (cross-platform) JMeter. You might want to consider The Grinder.

- for emulators (dead handy), I've had great results with VirtualPC on PCs. The field is moving fast, and these days I'd look specifically at VMWare and Parallels. I've used Parallels on the Mac lots, but not for testing (yet).

I'm sure you use different tools. When I teach Exploratory Testing, the tools workshop is often most eye-opening in closed teams, where one person is often sitting on a tool whose use is immediately apparent to other participants - yet they've never shared their toolsets until the class. (Disclaimer - I've already blogged most of this at http://www.workroom-productions.com/blog/2008/06/tools-for-explorat... , so apologies if it's not new to you)

Cheers - James; ▶ Reply to This

Permalink Reply by Rosie Sherry on November 21, 2008 at 12:30am: Silverback is worth a look at - http://silverbackapp.com/

It's relatively new, meant for usability testing, but could probably easily be used to record other testing stuff. Haven't used it yet, but will do soon. It's for Macs only.; ▶ Reply to This

Permalink Reply by Mark Crowther on November 21, 2008 at 8:35pm: Hiya Michele,

For testing of digitial media work (websites, etc.) I tend not to use Test Cases but Mind-maps. I get them reviewed by the client before we execute and as we execute and learn they get added to so we can track the path of testing.

The tool in this case being Free Mind, free mind-mapping software... who'da thunk it.

Mark.; ▶ Reply to This

Permalink Reply by Michele Smith on November 27, 2008 at 11:42am: Hi Mark,

I checked out the link for Free Mind and it actually looks like it could very well be worth trying out for at least one of the applications that I test. It could possibly save me time in documenting my Exploratory Testing on this specific product, which would, of course, leave more time for testing the actual application.

I have heard about mind mapping software before, but I would not have thought of this use if you had not listed a valid example of its use in your comment. Thanks for doing that!; ▶ Reply to This

Permalink Reply by Harinder on November 27, 2008 at 12:48pm: I have used Free Mind tool and its pretty useful tool. I have used during my functional testing days (helped me in the exploratory testing) and I am also using it (along with plane paper) when I am working out the performance scenarios.; ▶ Reply to This

Permalink Reply by Bob French on November 26, 2008 at 6:54pm: My most valuable tool...? My whiteboard. I'm very picture orientated (to the point my team accuses me of flowcharting EVERYTHING!).

My most valuable tool allows me to use my brain, which is my next most valuable tool.; ▶ Reply to This