I made a brief and rather dismissive comment about uTest on Twitter, and Rosie Sherry asked me to expand.
Firstly, my comment was a first impression, which I admit wasn't very good. However, I do want uTest to be a good idea, and I want to be persuaded that it's going to be of value to experienced professional testers. It would be great if something like this could really take off. However, I'm sceptical.
I can see how uTest would be valuable to development companies, especially for web applications. It will introduce another layer of testing that they'd have difficulty providing in house.
However, it strikes me that such testing is more suited to exploratory testing at a late stage in the lifecycle, after more formal testing has been completed in house. In effect, the uTest cycles look like a sort of beta release.
Exploratory testing is surely performed better by experienced testers, otherwise it can be pretty random button pressing. The uTest model, ie payment by bugs detected, and at a low rate, isn't likely to attract many experienced professional testers, especially if applications really are at a beta release stage, and most of the bugs have been detected.
Also, at this stage many of the problems would be related to usability, and this can be subjective. Testers could work on a defect report and find it's not accepted, because the client's subjective judgement is different, or because the defect is dismissed as cosmetic and of no great significance.
This leads on to another concern. The possibility of spending time detecting and reporting defects, then finding that many are not accepted, or that they have already been reported. No payment in such cases.
Testers would presumably have to invest some time in establishing how the application is supposed to work, and checking defects that have already been reported. They could then find that they find few defects, and that not all of these are accepted. Given that the rate per defect is fairly low it's hard to see that there's much money to be made for the time that would be required.
The majority of testers could therefore be inexperienced, even beginners. If the quality of testers is low then that forces the payment level down, and you've got a vicious circle. Or is my assessment of the payments based on UK levels, and are they set at a level that is going to get good, experienced testers from Asia?
When I was at school I used to work at a big racecourse, selling papers at big race meetings. We worked on a commission-only basis. We got 30% of the price of each paper we sold. I packed it in because I realised that the newspaper company had every incentive to take on as many sellers as possible, and that was what they were doing, making it extremely difficult for any seller to get a worthwhile wage for their day.
I fear that uTest could go the same way, and that the temptation for uTest and the clients would be to throw as many bodies as possible at the testing because there's no marginal cost to using an additional tester. If payment is only by results, and if the skill level of individual testers is low then that might seem a rational, though short term approach.
This isn't a considered essay on the subject, just a quick explanation of my fears based on my first impression. Like I said earlier, I want to be wrong! This could be great for contractors in between contracts, and testers who would really prefer to work from home if that's possible.
Is someone going to pick these sketchy thoughts apart and show why they're wrong?
Tags:
Share
Facebook
-
▶ Reply to This