How many of you are stuck in the "never enough time/people/equipment to test because everyone is fixing the problems in live" trap ?
I heard from a few people at last weeks SIGIST who were in that vicious cycle of poor quality releases being sent out as everyone was too busy dealing with the problems of the last poor quality release to test the new release...
If you've been in that situation, how have you managed to break out of it ?
I've been there, and even have an ulcer to prove it (from my first job as a QA Manager a long time ago...)
In retrospective, our problem back there came from 2 main rookie mistakes:
1. Lack of (my) leadership.
As you wrote in your post, I was not strong enough or secure enough to turn a couple of tables and demand to get some work resources for my testing needs (and there are always solutions like outsourcing that can cost a bit more but provide a good patch in relatively low time).
2. I did not make an effort to "think outside the box" and look for alternative solutions.
Today I know that when we have an issue of this sort it is simpler to take 30 minutes (and everyone can spare half an hour no matter how busy they are!) and brainstorm for possible solutions that might not be perfect but definitely good enough for our current needs and issues.
This is not at all a good situation for a company to be in. If a poor quality release is a one release problem, then people have to be available to attend the higher number of support calls, diagnose and fix the problems, and test the fixes. However, if release after release is of a poor quality, then it implies that the releases are never properly tested and fixed. One probable cause could be a (temporary) shortage of testing staff i.e. resource planning problem and another one could be the lack of a test process before release i.e. a quality problem.
In order to break such a cycle, the following solutions might be required:
1. Better planning (release planning e.g. postpone poor releases/ merge two releases etc. as well as resource planning e.g. hire temporary staff/ train and re-deploy staff from other areas etc.)
2. Better compliance to the test process
No doubt, everything may happen. There are different circumstances and challenges during the work on the project.
But project manager should take the most important part in this process. If there's a threat of poor-quality release each team member should know his duties very well. Each team member should be responsible for some particular part of the project. And the team leader usually coordinates this process and defines the priority of the task. Even in the case of the lack of testing stuff there are possibilities to coordinate the work correctly and avoid obvious drawbacks.
In most cases it depends on the character and managing talents of the team leader.
I have certainly seen this scenario. The solution was agreed between development management and senior user management. There was a suspension of development work on the applications being developed and supported by the area concerned until they'd stabilised the live applications, then the development management re-planned once they had the situation under control. It took a fair bit of guts on the part of the developers' management to press for this if you ask me. The problem had been caused by unrealistic expectations of what was possible by user management who'd been pressuring the developers to do more than was realistically possible.
The organisation of development in that company at that time was that testers were lowly seconded user/clerical staff embedded in the development team and emphatically subordinate to the developers. Not a good idea. It took a few years more for the idea of a separate test team with professional level testers to be accepted.
In my experience a behaviour like this is typically symptomatic of even bigger problems. Robbing Peter to pay Paul is always on the cards and Paul (Production) will alway inevitably win. :)
The trick in my opinion when the world is spinning out of control is to stop and get your bearings. There's something to be said to getting back to basics and reviewing not just the negatives but the positives of what you're doing well and what you need to improve on. You break the cycle by making a conscious decision to stop the cycle. Typically this will involve a conscious decision to stop and get focussed on getting forward momentum going and not circular...
If you're working your butt off to fix defects and introducing new and 'improved' defects, the best thing you can do is to stop creating more defects, by stopping development. The worst that can happen is you'll plateau on defects being raised while you commence an investigation and then get a baseline of how stable the application under test is (holistically). Then plan a way forward. A mitigation strategy is that you decide that only Severity 1 defects (maybe Severity 2 as well) get fixed in production to keep the mission critical systems going but other than that you need to let the dust settle and understand how bad things really are. Given that you're introducing new defects, your application can't be 'life threatening', but you do need to identify, categorise, prioritise your risks from an organisational, project and product perspective.
Heroics (working 14 hour days, weekends etc) typically never deliver on the promise of having more time to do more. Stop, clear your head and work towards moving forwards. Best of luck to those caught in this. I've been there and lived. Great for war stories when you get together with other testers...
- Entry Criteria. Testers who are empowered to stop testing early and 'move on to the next priority'. This forces the Business (and their analysts) to re-think the solution constructively. It also prevents testing from becoming a 'workshop' session for incomplete solutions.
- Create a Change Control board. Leave the task of prioritizing what gets fixed first to the people with the financial and political clout to hire more resources or force acceptance of workarounds. This free's up Testers to focus on quantifying the impact, identifying solutions (or even testing!!)
-Improve your process, find out what what makes your customers happy! then change your process to set and deliver to those expectations. Review what production bugs were found and investigate what fault in the testing process allowed them to get through unnoticed.
Why the heck do we pay all this money for testers anyway? is a tough question to face, its the financial impact of the bugs we find that can justify the time we spend.
In a Service oriented world where the vendor holds the client to their contract (sometimes to ransom), it is the testers who have the detailed systems knowledge to empower the business to hold a vendor the contract as well.