Developers as Testers ?

The following (somewhat provocative) idea is circulating around my workplace: Since we don't have enough testers, let's give the developers do the testing phase. The idea is that each iteration will have a "freeze" phase, where no development is done, only QC by the developers themselves.
In this phase, the developers would test the product .Ideally a developer would only test code that his colleague wrote and not his own.

BTW, apparently this scheme is applied successfully in the OpenBSD community, where the OpenBSD developers test the code before end of each iteration and there is no QC phase.

Pros:
1. Developers know better what goes under the hood so they would find more bugs
2. No need for tester position
3. Less people involved in the project -which is always good
4. Fast testing cycle, since arguably it would take faster for an insider developer to start testing

Cons:
1. It is beneficial that a professional test engineer would test, using his years of experience to find bugs
2. Developers can be engaged in fixing bugs and development instead of testing.
3. Developers might ignore bugs that interfere to much with their plans

To be honest, this idea appeals to me, but I would like your opinions.

Gabi

▶ Reply to This

Replies to This Discussion

Permalink Reply by Alex Sergeev on November 2, 2009 at 10:08am: It looks like you talking about "pair programming".; ▶ Reply to This

Permalink Reply by Gabi on November 2, 2009 at 10:24am: No. I don't think it has anything to do with XP or agile or whatever new buzzword they invent next :)
All it says is that developers would do their own testing instead of testers.; ▶ Reply to This

Permalink Reply by Alex Sergeev on November 2, 2009 at 10:44am: Well, in 'pair programming' we have two devs - one types in code, other review it from strategic position (architect, business, code artifacts, etc). Roles are changed periodically.

From your post I understand (sorry, my English is not good enough) that you talking about testing in the end on development period, but exluding process details (in pair programming - review during development, in your case - after development) it's really seems like some kind of 'pair programming'. Where I'm wrong?; ▶ Reply to This

Permalink Reply by Gabi on November 2, 2009 at 10:53am: Well, it doesn't have to be in pairs. The idea is that the team of developers develop together as usual (no pairs), and at the end of iteration, they all take the roles of testers and start testing. So you are right, the testing is not done during the development time but at at the end of the iteration, but it has nothing to do with pair programming in particular; ▶ Reply to This

Permalink Reply by Alex Sergeev on November 2, 2009 at 12:56pm: Okey, I'm not see big difference between because 'pair' means 'someone another'. You propose that all switched from development to testing. Ok, will think that this makes it special (possible I'm really not understand... no matter).

I think that it can be useful (and usually happened) in small or new projects, when testing base is not defined and hangs are admissible.

If we are talking about theoretical good [sapience] developer (that types code w/o newby mistakes, like to test and responsible) then your proposal will work, but... What about real environment where not all developers like testing, and not all can test. What to do in this situation in the end of iteration?; ▶ Reply to This

Permalink Reply by Joel Montvelisky on November 2, 2009 at 11:16am: Hi Gabi,

I haven't seen this idea implemented but I have seen phases when the regular QA team "invites" developer to perform pair testing sessions where they test together to find non-trivial issues that up to now where not discovered by the regular testing activities.
It is very good to do it, but mainly because you get the best of both worlds having the development insights with the testing perspective and perspicacity.

I would personally not go for the approach of having only developers do the testing, and I think that I mainly because I disagree with some of your assumptions:

(Taken from your post above)
Pros:
1. Developers know better what goes under the hood so they would find more bugs
-> Knowing what "goes under the hood" will not allow you to find more bugs, it is the way we challenge the product and bring it to multiple and complex end-user scenarios that finds more bugs. It is true to understanding the code will allow to find bugs differently, but more or better bugs is something that I don't agree (at least from my experience)
3. Less people involved in the project -which is always good
->On the other hand the project would take more time to be delivered, so remember that you should not only think in man in the project, but man-days/weeks/months in the project.
4. Fast testing cycle, since arguably it would take faster for an insider developer to start testing
-> Again I don't agree with you here, a tester can start testing and finding bugs faster than a developer if he/she is experienced enough and understand his job correctly.

To summarize, I agree that developers can gain a lot from doing testing and the project can also gain from this, but relying solely in developers to do the testing is something that I find risky at best and wasteful (time and resources) in most occasions.

Don't get me wrong, some of my best friends are developers, I just find they do a lousy job when it comes to testing...; ▶ Reply to This

Permalink Reply by Michele Smith on November 2, 2009 at 12:20pm: 1. Developers know better what goes under the hood so they would find more bugs

Find more bugs than who?

2. No need for tester position

Developers testing does not mean there is no "need" for a tester

3. Less people involved in the project - which is always good

Always good? Not necessarily.

4. Fast testing cycle, since arguable it would take faster for an insider developer to start testing

Is it faster to switch from writing code to testing than it is to go from testing to testing?

Cons:
1. It is beneficial that a professional test engineer would test, using his years of experience to find bugs

Experience does not necessarily equate to finding bugs.

2. Developers can be engaged in fixing bugs and development instead of testing

Some developers also test their own code while they develop. The issue is that they do not usually think the same way a tester or a customer would, so they miss bugs.

3. Developers might ignore bugs that interfere to much with their plans

This is sometimes true with other stakeholders as well in order to ship a product on time.

I am not sure I understand what is appealing about this idea.; ▶ Reply to This

Permalink Reply by Brian Buckmaster on November 2, 2009 at 12:56pm: I'm not surprised at this. I'm sure that the suggestion of having developers to the testing was put forth by the pointy-haired types (managers), who have had no formal training in software testing (or possibly software development either). So let me dispel a couple of urban myths:

1. Developers know better what's under the hood and can find more bugs. Taken as a whole, this is statement is inherently false. While its true that they (better) know the details of their code, its also the case that they are too caught up in those details - the cliche about not seeing the forest for the trees applies here. They have made assumptions and will embark on the testing function with those same assumptions.

Developers have a creative mindset, while testers (necessarily) destructive mindset - its natural for a developer to want to show that their code works, while a tester wants to demonstrate that it doesn't.

There is no evidence whatsoever that developers can find more bugs than testers. But hey, don't take my word for it. Try an experiment - hire someone with software test training/experience and have him/her test an application; have the developer test the same application (both should use the same code) and see who does better.

2. No need for the test position. If you have no need for a test position then software quality is not a priority. What you want to do is get the product out the door on the cheap. If you want to do that, then why not simply outsource the development to, say India? While you're at it, why not consider outsourcing the pointy-hair (project management) function as well?

3. Fast cycle - Well if you want to shorten the development cycle, why not simply get rid of the test function altogether? Let the customer find the bugs.; ▶ Reply to This

Permalink Reply by Rob Lambert on November 2, 2009 at 1:00pm: Gabi,

Interesting idea and one I've seen only once before with limited success.

I would ask the following questions:

1. Why are the programmers not doing Test Driven Development or Pair Programming to negate much of the need for testing afterwards? It seems wasteful to carry on as is and then test after when TDD and PP work and get rid of the bugs early. It sounds like the idea is to simply remove a tester, extend the release process to allow developer to test and only be concerned at a code level.

2. Do your developers really have the critical, questioning and creative mentality that good testers have? If so, then it could be a good initiative but my inclin would be that they don't.

3. Is testing purely about what goes on under the hood? Most defects I find are on the surface. Would they be found under the hood? If so great. If not, then who does the surface testing?

4. Would less people being involved in the project be a good thing? There is a scale. Sure, 100,000 people sounds too much, but an extra few testers here and there sounds like a positive thing to me. Why not remove some developers, do some TDD and pair programming and keep the test team?

5. Who is responsible for accessibility, usability, relevance, fit for purpose, look and feel, conformance, performance and supporting documentation?

6. Who owns the whole? i.e. the integration, the release, the installers, the environment support, the documentation, the help etc? Someone must take ownership for this and if developers are only looking at peers code then this sounds too low level.

7. Is comparing commercial (I'm assuming it is commercial) delivery of software valid to OpenBSD or Open Source software in general? Open Source is fab, but its models of working are based on volunteers and open source concepts. These might not gel with the commercial world.

Rob..; ▶ Reply to This

Permalink Reply by Gabi on November 2, 2009 at 1:53pm: Thanks. Some strong points guys. I will try to compile an updated pros/cons here :

Cons:
1. Knowing what "goes under the hood" will not allow you to find more bugs, it is the way we challenge the product and bring it to multiple and complex end-user scenarios that finds more bugs. It is true to understanding the code will allow to find bugs differently, but more or better bugs is something that I don't agree (at least from my experience). Developers have made assumptions and will embark on the testing function with those same assumptions.There is no evidence whatsoever that developers can find more bugs than testers.

2. Who is responsible for accessibility, usability, relevance, fit for purpose, look and feel, conformance, performance and supporting documentation? Who owns the whole? i.e. the integration, the release, the installers, the environment support, the documentation, the help etc?

3. The same effect can be achieved using TDD or PP, but this doesn't mean that the role of the test engineer is not needed. These are different kind of tests, which require different skill set and mindset.

4. Developers can be engaged in fixing bugs and development instead of testing.

5. Developers might ignore bugs that interfere to much with their plans

6.This suggestion was probably put forth by the pointy-haired types (managers), who have had no formal training in software testing (or possibly software development either). Maybe we should get rid of few of them first.

Pros
1. Sometimes faster testing cycle, since arguably it would take faster for an insider developer to start testing

2. Some types of bugs are more easily found by a developer that knows what's going on in the code.

3. Less people involved in the project -which is sometimes good; ▶ Reply to This

Permalink Reply by Brian Buckmaster on November 2, 2009 at 2:11pm: What experience or evidence can you offer to support Pros, 1 or 2? For example, if there was any basis for #2 then why do we even have software testers, siince the developers can find their own bugs?; ▶ Reply to This

Permalink Reply by Gabi on November 2, 2009 at 2:23pm: #1 is arguable, but am I am convinced that #2 is true in many (though not most) cases. Take for example a thread deadlock bug. A developer who knows his way around the code have much higher chances to find such bug because of the simple fact that he knows there are threads, and he knows the logic of the program, and he knows where are the weak spots. This kind of bug is very difficult to find without a prior knowledge about the code. There are many other examples like this. Inside information is priceless; ▶ Reply to This