The stabilizing phase conducts testing of the complete solution. Testing during this phase emphasizes usage and operation under realistic environmental conditions.
The team focuses on resolving and prioritizing (in Team Foundation Server also called triaging) bugs and preparing the solution for release.
Early during this phase it is common for testing to report bugs at a rate faster than developers can fix them. There is no way to tell how many bugs there will be or how long it will take to fix them.
There are, however, a couple of statistical signposts known as bug convergence and zero-bug bounce that helps the team project when the solution will reach stability.
MSF avoids the terms “alpha” and “beta” version to describe the state of IT projects. These terms are widely used, but are interpreted in too many ways to be meaningful in industry. Teams can use these terms if desired, as long as they are defined clearly and the definitions understood among the team, customer, and stakeholders.
Once a build has been deemed stable enough to be a release candidate, the solution is deployed to a pilot group. The stabilizing phase culminates in the release readiness milestone. Once reviewed and approved, the solution is ready for full deployment to the live production environment.
Major Milestone
The release readiness milestone occurs at the point when the team has addressed all outstanding issues and has released the solution or placed it in service. At the release milestone, responsibility for ongoing management and support of the solution officially transfers from the project team to the operations and support teams MSF Process Model v. 3.1 35
Recommended Interim Milestones
Bug Convergence
Bug convergence is the point at which the team makes visible progress against the active bug count. That is, the rate of bugs resolved exceeds the rate of bugs found.
Figure 10 illustrates bug convergence.
Because the bug rate will still go up and down—even after it starts its overall decline—bug convergence usually manifests itself as a trend rather than a fixed point in time.
After bug convergence, the number of bugs should continue to decrease until zero-bug release. Bug convergence tells the team that the end is actually within reach.
Zero Bug Bounce
Zero-bug bounce is the point in the project when development finally catches up to testing and there are no active bugs—for the moment. Figure 11 illustrates a zero-bug bounce. After zero-bug bounce, the bug peaks should become noticeably smaller and should continue to decrease until the solution is stable enough for the team to build the first release candidate.
Careful bug triaging is vital because every bug that is fixed risks the creation of a new bug. Achieving zero-bug bounce is a clear sign that the team is in the endgame as it drives to a stable release candidate.
Note that new bugs will certainly be found after this milestone is reached. However, zero-bug bounce marks the first time when the team can honestly report that that there are no active bugs—even if it is only for the moment—and it focuses the team on working to stay at that point.
Release Candidates
A series of release candidates are prepared and released to the pilot group. Each release candidate is an interim milestone.
Other features of a release candidate are:
• Each release candidate has all the elements it needs to be released to production.
• Building a release candidate tests its fitness for release, that is, whether all
necessary pieces are present.
• The test period that follows generation of a release candidate determines whether
a release candidate is ready to release to production or whether the team must
generate a new release candidate with the appropriate fixes.
• Testing release candidates, which is done internally by the team, requires highly
focused and intensive efforts, and focuses heavily on flushing out showstopper
bugs.
• Testing requires a triage process for resolving any newly discovered bugs.
• It is unlikely that the first release candidate will be the one that is released.
Typically, show stopping bugs will be found during the intensive testing of a release
candidate.
Pre-Production Test Complete
The focus of this interim milestone is to prepare for a pilot release.
This interim milestone is important because the solution is about to “touch” the live production environment. For this reason the team must test as much of the entire solution as possible before the pilot test begins.
Activities that must be completed during this interim milestone are:
• Evaluate test results against success criteria.
• Complete site preparation checklist and procedures.
• Complete implementation procedures, scripts, and load sets.
• Complete training material.
• Resolve support issues.
• Complete and test the rollback plan.
The pre-production test complete interim milestone is not complete until the team ensures that everything developed to deploy the solution is fully tested and ready.
User Acceptance Testing Complete
User acceptance testing and usability studies begin during the development phase and continue during stabilization. These are conducted to ensure that the new system is able to successfully meet user and business needs. This is not to be confused with customer acceptance, which occurs at the end of the project.
When this milestone has been achieved, users have tested and accepted the release in a non-production environment and verified that the system integrates with existing business applications and the IT production environment. The rollout and backout procedures should also be confirmed during this period.
Upon approval of release management, software developed in-house and any purchased applications are migrated from secure storage to a pristine archive location. In MOF, this is known as the Definitive Software Library (DSL).
Release management is responsible for building releases (assembling the release components) in the test environment from the applications stored in the DSL.
User acceptance testing gives support personnel and users the opportunity to understand and practice the new technology through hands-on training. The process helps to identify areas where users have trouble understanding, learning, and using the solution. Release testing also gives release management the opportunity to identify issues that could prevent successful implementation.
Pilot Complete
During this interim milestone, the team will test as much of the entire solution in as true a production environment as possible. In MSF, a pilot release is a deployment to a subset of the live production environment or user group. Depending on the context of the project, a pilot release can take the following forms:
• In an enterprise, a pilot can be a group of users or a set of servers in a data
center.
• In Web development, pilot release takes the form of hosting site files on staging
server(s) or folders that are live on the Internet, only with a test Web address.
• Commercial software vendors, such as Microsoft, often release products to a
special group of early adopters prior to final release.
Deliverables
The deliverables of the stabilizing phase are:
- Golden release
- Release notes
- Performance support elements
- Test results and testing tools
- Source code and executables
- Project documents
- Milestone review
Team Focus
The following describes the focus and responsibility areas of each team role during the stabilizing phase.
Product Management
Communications plan execution; launch planning
Program Management
Project tracking; bug triage
Development
Bug resolution; code optimization
User Experience
Stabilization of user performance materials; training materials
Testing
Testing; bug reporting and status; configuration testing.
Release Management
Pilot setup and support; deployment planning; operations and support training
Posted
May 06 2006, 06:03 PM
by
Damir Dobric