Testing & Review

In this phase the developers tag the completed modules as being ready for testing and copy their product to a testing sandbox so that the Quality Assurance staff can execute the test plans that were written/updated during the development phase of the cycle.

After each test execution, a meeting is held to review the bugs that were found and prioritize them. The developers then fix the prioritized bugs and the tests are executed again. These test-fix cycles continue until the product is stable.

Once a stable product is ready for User testing, the developers tag the completed modules and copy their product to a sandbox that is accessible to the users. Not all users will test this product. The Project Manager will find a few Power Users who will take the time to review the software and report their conclusions. They might also log bugs into the bug tracking system which will be reviewed and most likely fixed by the development team (after being prioritized.)

Expectations

Remember that, during the Design and Implementation phases, the customers were discussing all sorts of great ideas between themselves - but the development team is not working on any of these ideas. It is very important to manage the expectations of the customers who see the software in this phase. Customers who will be involved in testing must understand that they will be looking at the results of the last Evaluation Phase.

To maintain end-user buy-in for the project, it is important that the users have a clear picture in their minds of what has been done with the ideas that they've been discussing. In practice their ideas may not have been passed along to the Project Manager - so the development team is often limited in their ability to discuss specific new ideas and how they are being integrated into future cycles. Nevertheless, the customers need to know that their ideas have been documented and will be considered.

End-users need to have a clear idea of what they will be testing in the current phase of the software development cycle. In later cycles, after the stakeholders have experienced a few cycles of regular software releases, expectations will become a much less important issue - everybody will know that their ideas will be passed along to the development team during the Evaluation phase in a future cycle and not the current one.

Failure to manage these expectations can result in a situation where the Design and Implementation phases of the current cycle are revisited without completing the current cycle. This prevents the building of a solid foundation on which the project can grow successfully. Over time, development will continue but there will be no way to determine if the project meets the real needs of the end-users. Perceived needs will be met but at a cost which can't be evaluated relative to the real value of the features produced. This stretching of the Implementation phase can seriously threaten the success of the project overall.

Acceptance Testing

If this cycle is expected to produce a result which is intended to satisfy a contractual obligation, a special test is executed. This test, called an Acceptance Test, contains only test steps designed to verify that the software meets the contractual obligation. Acceptance tests are usually simple and quick to execute. They are far less detailed than the majority of tests that are performed before the acceptance step. The customer may already have been using the software for some time prior to the acceptance test and is already prepared to accept the software. Therefore, the acceptance test is often omitted if the customer signs an acceptance statement or approves the software for release to a production server.

Go No-Go Approval

If this cycle is expected to produce a result which is to be posted to a production server, formal approval of the posting is required from the customer. To obtain this approval a Go-No-Go meeting is held. During this meeting the heads of any departments affected by the release of the software will review all bugs remaining in the Bug Tracking System. Department heads will be able to query the development team leaders to ensure that everybody has a clear idea of the risks, if any, involved in making the release. If all department heads approve the release, the package can be installed on the production server. Otherwise, the various managers must decide what they want to do and take action to ensure that it is done.

Unfortunately, many Go-No-Go meetings result in the software being approved but the release being deferred. There are many reasons for this:

• Many releases involve functionality changes that require employee training. If the employees have not been trained to use the new software, the release might have to wait for the training to be completed.
• Many releases are of no value to the customer if the necessary data is not available. If the customer needs time to prepare the necessary data, the release will be delayed.
• Many releases involve components that must interface with new features that are in external systems. If one of those external systems is not ready yet, the release of the current product will have to wait until it is.
• Many releases add complexity and overhead to existing software. This might necessitate increasing the amount of available memory or disk space in the production server. From time to time a new feature in the software to be released requires an Operating System or Component upgrade before it can be used on the production server. If these upgrades are held up for some reason, the release to production will have to wait for them to be completed.

Regardless of the results of a Go-No-Go meeting, the development team can simply branch the software in the Revision Control System and move forward with the Evaluation Phase of the next cycle. When the next Go-No-Go meeting is called, the software will still be available for posting to production.

At this point, we know that we have a solid product in our hands. It does what the customer wants it to do and it works correctly - thereby meeting the two main objectives of the Software Engineering process. When we start writing additional software, we know we won't have any problems with the software we've written already; we are confident that the work will go smoothly as we will be building on a solid foundation.

Finally, we close this Software Development Cycle with a review of the work that has been done in the current cycle. Are we on track? Did we learn anything in this cycle that we need to incorporate into our future cycles (good or bad?) Is there anything we need to change in our Project Roadmap?

Following this review, we start the Evaluation Phase of the next cycle and repeat these steps again.