Exit Criteria for Software Deployment

In all textbooks and colleages’ lessons on software engineering and testing theories, we all knew what are the needed requirements to complete the testing cycles.  All these are what we therorically desire to work towards when applying to working life.  

However these are not always what we expect in real working world.  Unexpected scope ‘creep’, unclear use cases’ definations contribute to lots of unexpected outcomes and various decision-makings to bring the projects back on track.  Leeways may have to be considered what can be accepted to move the project to go-live.  Testing criteria is one of the important factor to ensure what needs to be tested, to replicate the exact scenarios before cutover to Production.

We also have to set expectations to allow the project teams to decide how to make decisions to get the solution out, working towards Customer satisfactions.  We are aware how good agile project management or extreme programming do provide such benefits and tangible outcomes in software engineering, likewise for iterative developments and testings.

So what are the types of exit criterias to set?  Below are what I have been through since many years of software project management:

1. 95% of the test cases covered - This is up to the steering commitee on agreeing at how many test cases, especially the priorites ones to be resolved.
2. 90% of critical/high defects resolved – This is up to the steering commitee on agreeing at what level of defects to be resolved.
3. The defined deployment setups tested and completed successfully in the test environment.
4. User training – enable the users to have hands-on on the tested system and being familiar with them before going live.

Another Arguement on Test Plan

Today, we had a “destructive” discussions with the Client on leveraging the testing schedule, a project issue and a change request.  We still come back to the initial plan that the Customer will like to start the testing as planned for mid-February.  However, our development works cannot complete at that time due to certain factors like dependency on the webservices’ development, late sign-off/agreement on the requirements, and changes that come along the project phase.

Up till today, neither party agrees to the late testing schedule as in our initial plan.  Internally the team is trying their very best efforts to complete the work by end February but with the customer’s preference and persistence to get the testing to start next week we feels that we are facing project failure if no proper development is achieved just to ‘please’ the customer.  I fully understand the reason behind the early testing as due to the complexity of the scopes and more testing is required to ensure critical defects are identified and resolved prior to April go-live.

If the ultimate objective is to have low, minimum impacts for Operations during go-live, I feel that we need to do careful development works and proper testings to produce the quality works.  If we are to rush and care to get things done just to meet the go-live date we are actually facing the project failure and may result more issues when go-live.  We cannot achieve quality work in a short period, not to mention to work on the change requests that come along.  In simple words, we can’t have both worlds together.

Tomorrow I have to get back to the Customer if we can achieve the required testing scopes by end February.  I do know by now, we are not possible to make it.  Seems like the last resort I have is to escalate to the Management and postpone the go-live date may be the solution if to achieve our planned testing schedule and scopes with the Customer.

Just have to leave with it………