Testing Concepts: The Clumsiness of Agile
Introduction:
In 2001 the Agile Manifesto introduced a new paradigm in software development. Iterative Agile and Agile-like development have been widely adopted and have become a subject of interest for both the business and academic communities with dedicated conferences and an Alliance of practitioners. But is 8 year old technology a fad or is it a sustainable business practice? Does Agile maximise return on investment? Two scenarios of a are presented to investigate.
Scenario 1: Maximising Revenue.
Consider two projects on a backlog. Project A will result in an increase in revenues of £5000 per month, Project B will result in an increase of £1000 per month. As both are revenue generating both have been marked as critical. Both are identical in terms of the estimated amount of effort required to develop and test each project which equals total effort available for each of the coming development iterations. A decision must be made as to which project is completed first.
Whether this information is made available to the team seems to be related to the development type used. Scrum prioritises by both business value and development effort. Extreme Programming on the other hand will group both projects as being critical, information as to how critical is not suggested for use at iteration planning.
Scenario 2: Minimising Costs.
Consider two projects C and D that are identical in the return on investment and in the total estimate of effort required for completion. The team available consists of two developers and two testers. Project C is development intensive; two developers and two testers are required for it's completion. Project D has requires less development effort but more testing effort so that one developer is needed to code full time and the remaining team members will be required to plan and test. Again, a decision is required as to which of the two projects is to be attempted in the next iteration.
To decide which project should be prioritised, ceteris paribus, the costs must be considered. In particular since the salaries paid to complete either project will be the same, the opportunity cost is important. Currently the average annual salary for a tester in the UK is £35,000 and the average salary for Java Developer is £47,000. For a one month iteration the opportunity cost of using a developer to test software is £1,000.
Conclusion and discussion.
From the above scenarios it can be seen that Agile does not in all situations maximise revenues and minimise costs for each iteration and, by extension, each software release. Since profit equals revenues minus costs, it is clear that Agile does not ensure maximum profit.
But should the decision of which project should be completed when really matter? After all, as long as the critical projects are completed in a given release and the resources available are utilised should the order of completion and the team structure make a difference? The answer can be resolved by combining scenarios 1 and 2.
Consider two competing companies, Alpha and Omega, each have only one project in the current release. Company Alpha is in the fortunate position to have a project with the characteristics of A and C; that is their team will be operating at full capacity to produce high revenue generating software. Omega has a project that is a combination of projects B and D; the staff are overpaid compared to what they are capable of producing and the resulting software is worth less than Alpha's. After just one month Alpha is £6,000 richer than it's competitor. This gives an advantage that can be re-invested in the company. If Omega continues with sub-optimal strategies then it's long term survival is less than certain - project choices really do matter.
It can be said that Agile can lead to clumsy decisions.
Suggestions for future work.
A theoretical framework based on simple economics has been here presented. In real life applications projects are less likely to satisfy the "ceteris paribus" condition as they will differ on several variables such as complexity, risk, cost and return. How decisions can be optimised could be an interesting area of study.
Furthermore, project teams influence project selection. Group dynamics including majority and minority influence should be studied.
Lastly, experiments could be conducted to investigate the combination of superior project assessment tools such as Net Present Value (NPV) over the simple payback rule assumed throughout this report.
In 2001 the Agile Manifesto introduced a new paradigm in software development. Iterative Agile and Agile-like development have been widely adopted and have become a subject of interest for both the business and academic communities with dedicated conferences and an Alliance of practitioners. But is 8 year old technology a fad or is it a sustainable business practice? Does Agile maximise return on investment? Two scenarios of a are presented to investigate.
Scenario 1: Maximising Revenue.
Consider two projects on a backlog. Project A will result in an increase in revenues of £5000 per month, Project B will result in an increase of £1000 per month. As both are revenue generating both have been marked as critical. Both are identical in terms of the estimated amount of effort required to develop and test each project which equals total effort available for each of the coming development iterations. A decision must be made as to which project is completed first.
Whether this information is made available to the team seems to be related to the development type used. Scrum prioritises by both business value and development effort. Extreme Programming on the other hand will group both projects as being critical, information as to how critical is not suggested for use at iteration planning.
Scenario 2: Minimising Costs.
Consider two projects C and D that are identical in the return on investment and in the total estimate of effort required for completion. The team available consists of two developers and two testers. Project C is development intensive; two developers and two testers are required for it's completion. Project D has requires less development effort but more testing effort so that one developer is needed to code full time and the remaining team members will be required to plan and test. Again, a decision is required as to which of the two projects is to be attempted in the next iteration.
To decide which project should be prioritised, ceteris paribus, the costs must be considered. In particular since the salaries paid to complete either project will be the same, the opportunity cost is important. Currently the average annual salary for a tester in the UK is £35,000 and the average salary for Java Developer is £47,000. For a one month iteration the opportunity cost of using a developer to test software is £1,000.
Conclusion and discussion.
From the above scenarios it can be seen that Agile does not in all situations maximise revenues and minimise costs for each iteration and, by extension, each software release. Since profit equals revenues minus costs, it is clear that Agile does not ensure maximum profit.
But should the decision of which project should be completed when really matter? After all, as long as the critical projects are completed in a given release and the resources available are utilised should the order of completion and the team structure make a difference? The answer can be resolved by combining scenarios 1 and 2.
Consider two competing companies, Alpha and Omega, each have only one project in the current release. Company Alpha is in the fortunate position to have a project with the characteristics of A and C; that is their team will be operating at full capacity to produce high revenue generating software. Omega has a project that is a combination of projects B and D; the staff are overpaid compared to what they are capable of producing and the resulting software is worth less than Alpha's. After just one month Alpha is £6,000 richer than it's competitor. This gives an advantage that can be re-invested in the company. If Omega continues with sub-optimal strategies then it's long term survival is less than certain - project choices really do matter.
It can be said that Agile can lead to clumsy decisions.
Suggestions for future work.
A theoretical framework based on simple economics has been here presented. In real life applications projects are less likely to satisfy the "ceteris paribus" condition as they will differ on several variables such as complexity, risk, cost and return. How decisions can be optimised could be an interesting area of study.
Furthermore, project teams influence project selection. Group dynamics including majority and minority influence should be studied.
Lastly, experiments could be conducted to investigate the combination of superior project assessment tools such as Net Present Value (NPV) over the simple payback rule assumed throughout this report.
Replies to This Discussion
-
Permalink Reply by Robert Healy on -
Hi Joe,
Thank you for recognising the existance of arbitrage but far more importantly thank you for recognising that Agile as currently exists can cause sub-optimal decisions.
Some banker friends of mine ran a risk free casino for a while. They bet with everyone else's money and kept all profits (mostly) for themselves. They did so with the government covering all major losses and when it all went belly-up everyone but them seemed to lose their shirts. It's nice work, but I prefer testing.
- Rob
-
Permalink Reply by Joe Strazzere on -
"Some banker friends of mine ran a risk free casino for a while."
Good for them! You should ask them what development methodology they used. Perhaps they found a method for making optimal development decisions?
-
Permalink Reply by Ainars Galvans on -
Robert,
Just as James I feel somewhat wrong reading your conclusions. I feel like you are mixing up projects management (I'm not an expert but I've also seen term portfolio management) methodology with development methodology. Beside revenue there are such things as time-to-market (i.e. if you are too late you may loose all the revenue because your competitor beat you). and other things.,,
Here is what thinking provoked your post, though it is not direct answer: agile tries maximize revenue and minimize cost through the project. It is methodology that think about immediate ROI. For example if your projects A and B are expected to long for 1 year, agile provide you the benefit to deliver 20% of features within, let's say 3 months. Assuming that we be able to generate i.e. 1000 revenue on project A which would be 9000 in next 9 months compared to traditional approach would not allow any revenue all year long. More over the revenue will increate to let's say 2,500 beginning with moth 7, etc.
I believe agile is more suited for projects where ROI is dynamically changing and I believe most of the businesses are dynamic.
On the other hand If you know that software will generate you stable revenue for next 20 years (no support cost, etc.) ... well I have no experience with such a type of business, so no comments. I believe that this is not the context agile is optimized for.
P.S. SCRUM is more a management methodology while XP is pure development methodology that' why SCRUM include more details about investment management integration with development methodology. They are not different methodologies they are commonly used together.
-
Permalink Reply by Robert Healy on
-
Thank Ainars,
Hmm, maybe my terminology is off? My meaning of project is the opposite of yours, in a release there are several previews. In each Preview there are several iterations. In each iteration there are several user stories. These user stories are my projects. I don't mean the long-term product vision.
Your analysis seems reasonable. If we consider the time-value of money then money received in the near future is worth more than the same amount received further away. From this perspective Agile with it's release-able software philosophy offers definite financial benefits to a company.
You are dead right about agile offering distinct advantages to dynamic organisations, again Agile organisations should be able to react faster to changes in the marketplace, technology or competition.
Thanks,
- Rob
-
Permalink Reply by Dave Rooney on
-
Your terminology is off? Confusing User Story with Project?
I'm almost afraid to ask this question, but how big are your User Stories? Are you able to complete them within an Iteration?
Dave Rooney
The Agile Consortium
-
Permalink Reply by Robert Healy on
-
Hi Dave,
I am sorry that I used a term that was not straight from the www.userstories.com. I do recognise that this has caused confusion. I would point out, in an attempt at defence, that there is no restriction in the Agile manifesto as to what terminology would be used and that "user story" seems to stem from XP and scrum and need not apply to all agile methodologies.
It's a very reasonable question. For the sake of argument, the a-priori information suggests that each user story can be completed in any given iteration period but two cannot. Neither are epic.A decision is required between two mutually exclusive options as to how to employ development effort in the next iteration.
Does your question pertain to project selection or the implications of development overruns?
Thanks,
- Rob
-
Permalink Reply by Michael Bolton on -
Extreme Programming on the other hand will group both projects as being critical, information as to how critical is not suggested for use at iteration planning.
Huh? Where did you get that impression?
From the above scenarios it can be seen that Agile does not in all situations maximise revenues and minimise costs for each iteration and, by extension, each software release.
Agile doesn't in any situation do that stuff. People do that. Agile processes are intended to help them do that.
It can be said that Agile can lead to clumsy decisions.
Given an insupportable premise, any conclusion is possible.
---Michael B.
-
Permalink Reply by Robert Healy on
-
Hi Michael,
Thank you for taking the time for reading my post. I based that statement around personal experience and evidence from both http://en.wikipedia.org/wiki/Extreme_Programming_Practices#Planning... :
"
The business side sorts the user stories by business value. They will arrange them into three piles:
- Critical: stories without which the system cannot function or has no meaning.
- Significant Business Value: Non-critical user stories that have significant business value.
- Nice to have: User stories that do not have significant business value.
"
and http://www.extremeprogramming.org/rules/planninggame.html :
"
When the final release plan is created and is displeasing to management it is tempting to just change the estimates for the user stories. You must not do this. The estimates are valid and will be required as-is during the iteration planning meetings.
"
I agree that people are responsible for decisions and Agile is simply a tool for that purpose. My point is although Agile processes are intended to help, they fail to do so. They fail when they don't help the people to recognise the comparative benefits of one project over another and they fail when they don't help planners assess the comparative costs of one project over another. A bad tool may be better than no tool (or, as several others have highlighted, alternative tools) but this still doesn't make it a good tool. I hope you concur that we can re-engineer the tools available and hopefully make iterative improvements to them - my intention was to highlight one possible area of improvement.
Thanks for your feedback,
- Rob
-
-
Recognizing comparative benefits of one project over another has nothing to do with the development process. That's portfolio management! (Edit - I wrote this before reading your earlier clarification that you consider User Stories to be Projects.)
The Wikipedia article makes it sound like the Release Plan is a static entity when it is anything but. As teams build software, they learn more about the problem they're solving (including the business people!). As such, priorities may be adjusted every iteration. That's not a bad thing - in fact, it's why we have moved away from plan-based software delivery processes.
Don Wells' page describing the XP Planning Game is 10 years old. It was valid in 1999 (and still to some extent today) because in many cases the people making the estimates weren't always the same people who did the actual work. However, that's not the case as much as it once was. We have also learned a considerable amount about applying XP and Agile in those 10 years. When I'm coaching teams, I have them review the estimates during each Iteration Planning meeting. Quite often, estimates will have changed based on what the team has learned during previous iterations.
Dave Rooney
The Agile Consortium
-
Permalink Reply by Robert Healy on
-
All valid points Dave, well made.
-
Permalink Reply by Dave Whalen on -
Spot On!
Contrary to popuplar belief, Agile is NOT the absolute solution to everything related to software development. But like most major religions - if you dare to question Agile the "true believers" will rally and crucify you. Continue to speak up! I've got your back. Don't drink the Kool-Aid!
Agile is ONE method that works well in SOME situations - NOT ALL
-
-
Dave,
Agile is not even "one method". There are a number of forms of it such as Scrum, Lean Software Development, Kanban, Extreme Programming, Crystal, DSDM. They have common characteristics which fall under the umbrella of Agile as defined by the Agile Manifesto.
I agree that there are situations where none of these methods works, and I've seen them. Those are the places that don't value people, communication and delivering to the business.
As for "crucifying" people, I don't do that. When I see someone write things that don't match my experience of the last 9 years, I'm going to point it out. I'm concerned that in this blog entry, the original author has so misused the terminology that he either doesn't understand it, or hasn't even used an agile approach before. To me, it doesn't make sense to criticize something that you know nothing about. I'm not sure if that's the case here, which is why I'm trying to get some clarification.
Dave Rooney
The Agile Consortium
- ‹ Previous
- 1
- 2
- 3
- 4
- Next ›
Find Us Elsewhere
Nice Things
Bloggers
© 2010 Created by Rosie Sherry
