Do You Build Your Own Test Tools?

Replies to This Discussion

Permalink Reply by Jake Brake on March 7, 2009 at 1:21pm

Build own? Yes - many. Here is one. One of the application architectures I performance test uses basic base-64 authentication (yeah - a real security hole - but in a DMZ behind a firewall). When a user accesses the site page, a header must be supplied with encrypted userid and password. The performance testing of this app requires large numbers of simulated users. Since our large pool of simulated user's test user account passwords change periodically, so must the encrypted string. To regenerate the basic auth strings, I have a .bat file call a VBScript utility. This utility reads each userid/password; calls a base64 encoder; captures the output; VBscript takes that and writes the new into a spreadsheet; iterating through all userid/password pairs until complete.

Favo(u)rite language = assembler or FORTRAN-77. I understand per Phil that "Limerick" language is gathering momentum. I have not yet written a line of code in that language but I am contemplating a unit.

▶ Reply to This

Permalink Reply by Michele Smith on March 7, 2009 at 7:10pm

Hi Jake,

I admit I had to google your favo(u)rites. Those two appear to be way out of my league, but VB is one of my considerations. I think getting to the point where I can find areas to test "smarter and not harder" is what I am hoping to get out of this eventually. Your example of a tool you built seems to have done just that for testing your application.

As for the "Limerick" language, I have been contemplating that one as well.... ;)

▶ Reply to This

Permalink Reply by phil kirkham on March 7, 2009 at 1:23pm

I've used Ruby quite a bit - started off because of Watir then realised how much more I could do after reading Everyday Scripting with Ruby: For Teams, Testers, and You ( that book also gave me other ideas for test tools and not just ones in Ruby )

also dabbled with Perl and used VB to quickly make something with a front end, used .Net for web service testing
Some of it depends on the app being tested as it can often be easy to grab routines the devs are using and use them in your test tools

Keep meaning to give Python a try sometime and I would also like to brush up on my Ruby, been quite some time since I last used it

▶ Reply to This

Permalink Reply by Sharath Byregowda on March 9, 2009 at 4:55am

I agree with Phil, the book ""Everyday Scripting with Ruby: For Teams, Testers, and You"" did give me a lot of ideas and have helped me learn Ruby as well.

I have off late started writing small scripts, which could help me in testing, but no a tool yet.

-Sharath.B

▶ Reply to This

Permalink Reply by Michele Smith on March 9, 2009 at 9:12am

Thanks for the book suggestion. I checked it out and decided to order it.

▶ Reply to This

Permalink Reply by Brent Strange on March 8, 2009 at 4:22pm

So I was wondering, do you build your own tools to help you in your testing efforts? Yes!

If so, do you have a favorite programming language that you use that has benefitted you in building tools for your own personal testing arsenal? .NET and C#

I second Phil when he says "Some of it depends on the app being tested". Before choosing a language I always consider the app under test, the language the app developers are using, where I'm going to get support from when I'm stuck, and where the tests will be executed from.

This is how strong I feel about .NET and C#:

http://qainsight.net/2009/03/08/20ReasonsToUseVSTS2008AsYourAutomat...

However, it's not a silver bullet in all situations. C# is probably overkill for doing simple things like creating random data, but you never know what you're going to automate next!

▶ Reply to This

Permalink Reply by Michele Smith on March 9, 2009 at 9:29am

Hi Brent,
Thanks for the reply. I read your blog and while I found it an interesting read, it seems to be about automating testing. That is a bit more than what I am thinking about at this point. But, who knows what tomorrow brings...

▶ Reply to This

Permalink Reply by Tony Bruce on March 9, 2009 at 10:54am

What about starting with things like these?
http://msdn.microsoft.com/en-us/devlabs/cc950524.aspx
http://blogs.msdn.com/coding4fun/pages/1994662.aspx

▶ Reply to This

Permalink Reply by Gil Bloom on March 9, 2009 at 1:35pm

Your question can go further to a more general one - do you build your own applications for your specific needs? testing? other? my approach is to let the experts (in whatever field) do what they know best and keep doing what I (try) to do best.
Find the best tools available for your purposes, otherwise you will find yourself too busy with programming, maintaining (and testing :-) your self-built applications instead of testing what you need to be testing.

▶ Reply to This

Permalink Reply by phil kirkham on March 9, 2009 at 1:52pm

But in Michelle's case, wasn't knowledge of a programming language to allow some unique ID's to be generated the best tool ? What would your advice to her be ?

Point taken that people can get distracted by building tools and often dont consider the maintenance of them but even if you get the best tools in written by experts you still find you need to spend time tweaking them so they can be used for your needs. Add in the fact that there will be testers working with no budget to buy tools nor time to evaluate them...

▶ Reply to This

Permalink Reply by Michele Smith on March 9, 2009 at 2:04pm

Thanks for clarifying this some, Phil.

There is also more that can be learned through what I am hoping to accomplish.

1. Learning programming languages helps testers to better understand where to look for bugs.

2. Learning programming languages helps testers have a bit more understanding of the development processes.

3. Learning programming languages helps testers to have a better understanding of and a more open line of communication with developers themselves.

I am sure there are more benefits than these as well.

▶ Reply to This

Permalink Reply by Anna Baik on March 9, 2009 at 9:13pm

Absolutely agree - although I very much believe that you do not need to be able to code to be an outstanding tester, I do believe that for an outstanding test team (disclaimer: in my area of work!) it helps to have some members who are testers who code (even just a tiny bit), and who can understand the application under test at a more technical level. I don't think that outsourcing this thinking to people who do not have the tester mindset is ever a good idea. There is always an overlap in responsibilities, and deliberately pushing out that overlap area so none of it is within your team and you're always asking other people to do stuff that really ought to lie at least partly within test, is a great way to create a tester ghetto where you're perceived as low skill button monkeys.

Where do you draw the line? Is an SQL query a tool? What about having a nice editor that allows you to do block edit or column edit on datasets? What about being familiar enough with the Unix toolset that you can extract the error messages related to your failed order from a logfile whilst excluding the noise? What about when you want to do some data analysis to spark off some more test ideas and you need to munge together a quick shell script to compare different columns in two different files and output the matches? What about when you want to track your defects against the tests they're blocking and you want to construct an Excel spreadsheet which will do the job?

None of this is rocket science, none of this would take all that long to teach someone who has the intelligence and motivation to be a good tester. And none of it is really stuff that's worth bothering with the overhead of booking development time with another team - you could probably do it in the time that it takes to raise the request. You don't need to have everyone in your team capable of doing everything, you can split the load.

At the end of the day I think it's really important to be able to play with data. You get ideas when you're playing that you wouldn't get otherwise. And testers usually care more about data and its quirks than anyone else.

▶ Reply to This

• ‹ Previous
• 1
• 2
• 3
• Next ›
• Page