Replies to This Discussion

Permalink Reply by Joe Strazzere on November 2, 2009 at 20:52

I'm not sure what "best" means here, but WinTask works well for us at this time.

Still need to write/modify the occasional script, though.

Permalink Reply by Peter L on November 2, 2009 at 21:03

I think it depends what you are trying to test, your application and what you trying to achieve through using a tool?

Permalink Reply by Jonathan Kohl on November 2, 2009 at 21:32

I'm biased towards Watir. Try installing Ruby and the Watir gem, and try the examples here and see if they do the trick for you or not. I've used Watir on all sorts of web projects.

One question: what language are your programmers using?

-Jonathan

Permalink Reply by Tony Simms on November 6, 2009 at 17:47

Sorry, this is not a very helpful post but I do like the idea of Jewell using ruby and gem :0)

Permalink Reply by Rosie Sherry on November 2, 2009 at 21:38

I came across http://go-test.it/ recently. Haven't tried it though.

Permalink Reply by Albert Gareev on November 3, 2009 at 13:33

Sorry, click-based automated testing can never be "the best". No matter what tool.

Permalink Reply by Jewell on November 3, 2009 at 16:41

You know, that's just not really a helpful response given the parameters of my question.

Permalink Reply by Albert Gareev on November 3, 2009 at 18:01

Some problems have no solution, you know.

Your parameters:
* Interaction through UI only.
It's not sufficient enough even for _good_ coverage in automated testing, and you want the best.

* Test creation: Click-and-record only, no programming.
It has been proven long time ago and in so many ways that the only practical outcome of click-and-record is sale presentation which has nothing to do with testing.
Please do some research. One article to start with. Bret Pettichord, November 1999.
A lot of useful links at the end of the article.

* Environment: Web-based/AJAX
Web 1.0 applications have plain logic. Nothing happens until the page is submitted and reloaded.
Web 2.0 applications are different. AJAX and Flash technologies make the flow event-driven and render GUI beyond the browser window.

Function Dictates Structure.
On top of GUI recognition successful test automation demands built-in functionalities for data management, reporting, verification & validation, event- and error-handling, synchronization and transparency. All the above is encapsulated in a framework.

There are some powerful and successful commercial solutions, like QTP + QC + BPT, and there are many companies that use open-source or in-house created frameworks. Model-based hybrid keyword/data driven would be on top but even plain keyword-driven table-based allows achieving most common business needs.

Thank you,
Albert Gareev
http://automationbeyond.wordpress.com/

Permalink Reply by Rob Lambert on November 7, 2009 at 19:38

Albert,

Interesting points but somewhat context based. I'm a firm believer that record and playback is the most expensive and heavy to maintain form of automation there is. I'm also a firm believer that well scripted, well thought out and framework led automation is also the "best" way to automate.

However - context is King. And record and playback (R&P) does very much have a time and a place in some contexts. To blindly dismiss R&P as a waste of time and to claim "It has been proven long time ago and in so many ways that the only practical outcome of click-and-record is sale presentation which has nothing to do with testing." is to dismiss potential contexts in which it is useful.

I use R&P very lightly, but very successfully. On a project a few years back 75% of the tests were automated using R&P and it worked perfectly. It was a throw away project, didn't warrant investment in automation and the scripts were needed for none technical end users to use to load data. On many projects we have R&P tests that we use to simply manage state or load data through the UI. We could do it through the db or write a nice script but a simple R&P did the job in about 1 minute.

R&P is also a very good way of starting a test, learning how the code is generated and is often a superb introduction to automation - leading people down the route of more maintainable and usable scripting tools/techniques.

There are also many many tools that will work very well with R&P tests to give the management stats, screen shots, error reports and daily feedback. We have R&P test integrating to the CI build, running, reporting and storing within TFS. It works perfectly well - for that context.

But it is important to point out that R&P tests are expensive to maintain, brittle and often too high a level. But I know Jewell is aware of this and her context may demand/need something yours, mine or anyone elses context doesn't.

I do feel R&P has a bad reputation because people spend years using it to very little ROI but if used selectively, correctly and with the knowledge of it's limitation, R&P can be very successful.

http://thesocialtester.posterous.com/tag/recordandplayback

In my opinion I would avoid the QTP and main vendor tools. Watir, robot framework, fitness, ruby, python etc etc etc are all really powerful.......and free.

Rob..

Permalink Reply by Albert Gareev on November 8, 2009 at 17:04

Hi Rob,

Yes, you're right. It's all about context.
So let's look into yours.

> On a project a few years back 75% of the tests were automated using R&P
> and it worked perfectly. It was a throw away project, didn't warrant investment
> in automation and the scripts were needed for none technical
> end users to use to load data.
> On many projects we have R&P tests that we use to simply manage state
> or load data through the UI

- As I said before (see dialog with Jonathan on the next thread): data entry is not testing. Doesn't even matter whether you do it manually or by using any tool.
http://automationbeyond.wordpress.com/2009/05/30/test-automation-pr...


> R&P is also a very good way of starting a test,
> learning how the code is generated and
> is often a superb introduction to automation -
> leading people down the route of more maintainable and
> usable scripting tools/techniques.
None of the activities you enlisted are testing.
Additionally, learning programming through recording and code generation is a very bad approach. It DOES NOT teach people following good programming techniques (structure/layer/abstraction/naming convention/scope ... etc) but it DOES teach the most horrible things: raw coding, hard-coding, copy/paste coding, and absolutely no understanding of error handling.

As for test automation requirements, from the end-result and ROI perspective, I set it as the following matrices: Usability, Maintainability, Scalability, and Robustness.
As you may see there is even a small niche for GUI Data Entry which offers very little testing value though.

While being somewhat sometimes helpful as "quick-and-dirty" approach, record-playback still is a necessary evil that should be left behind on the automation evolution path.

Thanks,
Albert

Permalink Reply by Joe Strazzere on November 8, 2009 at 17:32

I never depend on Record-and-Playback as the sole method of developing a Script or a Test Suite. It's just not possible to develop robust, full-featured Test Suites that way.

On the other hand, a quick recording is very often a nice way to create Script code rapidly; code which can then be edited into the final form.

I've used Test Tools which didn't provide a recording capability. It's still usable, but not nearly as efficient.

http://www.sqablogs.com/jstrazzere/301/Things+I+Like+to+Have+in+my+...

"data entry is not testing"
Executing a hand-crafted script is not testing either, but data entry and script execution are both steps within many testing exercises.

Permalink Reply by Albert Gareev on November 8, 2009 at 18:17

Joe,

Data entry lacks in verification and reporting. At the end, who cares if the tool failed something somewhere but there is no clear way to reproduce it? Or worse, if it raises irrelevant fails due to legacy of dynamic data or GUI variation, it loses credibility. So testers either ignore execution results or redo testing manually to be sure.

A script, intended for test execution, and following the requirements described in your blog (btw, great article!), wouldn't get confused so easily.

Thanks,
Albert

• ‹ Previous
• 1
• 2
• 3
• …
• 5
• Next ›
• Page