From planning, through execution to release, UAT (User Acceptance Testing) is an intricate process that needs to be actively managed to avoid unnecessary costs and delays. UAT is used to validate that a system or application will meet its users’ needs in accordance with predefined business requirements. It is often the last functional testing cycle before a system goes live and, as such, is the primary opportunity for the end users to identify any defects.
Despite the reputational and financial consequences of releasing an unstable or flawed system there are several common pitfalls of UAT which all project managers should be aware of, but often overlook. Without taking these into account, and planning accordingly, a project manager may find the product does not fully meet its requirements resulting in the failure of the project.
- Insufficient planning – we are always ready until we are not
The lack of an appropriate, clear and signed off test plan can result in ambiguity as to the sequence of activities, responsibilities, scope and timings. Planning should always be founded upon an agreed set of business requirements with clear entry and exit criteria specifying what UAT is to achieve before the next phase of the project can commence. Agreeing the component and scenario scope with all signatories is the cornerstone of UAT planning activities.
A detailed single run book listing the sequence of activities and interdependencies can be invaluable for successful UAT. Yet, its effectiveness is dependent on communication. Clear planning and communication is particularly important where there is tight coupling of activities, meaning delays in one stage can have a knock-on effect on the whole testing cycle. Calling out, to enable heightened awareness of, critical path activities is essential to ensure the correct tasks have the appropriate focus.
- Lack of system availability and stability
Thorough effective System Integration Testing (SIT) should be performed before UAT commences. As users often have the most detailed system knowledge, it is tempting to start UAT even though SIT is not complete in order to save time and highlight issues early. The risk this carries is that UAT execution will be undermined by the inadequacy of the systems upon which it depends, requiring additional cycles of regression testing. Test environments can be notoriously unstable due to non-production infrastructure. Having adequate application alerts and/or performing daily smoke tests will help ensure systems are up and running during testing, leading to early identification and resolution of potential issues and reducing investigation time into cases failed due to environment issues.
- Lack of resource availability and ability
Having the right people to perform UAT is vital for its success. Few things are more frustrating than having testers without the required skills or business knowledge. Those with the best skill set are always in high demand and may not be available, thus striking a balance between knowledge and availability is the key challenge. Where resources are constrained, on boarding new resources to backfill an SME’s line role is recommended.
Identifying the best resources available is futile if they are not available when required. Very often resources will be shared across projects or work in production as well as UAT. While this can increase their flexibility and usability, it can also limit their availability during testing, impacting the overall test and project plan. Identifying a back up as contingency is advised.
- Poor communication channels
It is never safe to assume that key stakeholders, upon who success is dependent, fully understand and adhere to the agreed plan. Often testing is unsuccessful or delayed due to lack of communication among stakeholders. Regular pulse calls/meetings are essential to communicate updates, keep track of progress and escalate issues. While there is no set frequency for how often meetings should occur, it should reflect the complexity of testing and the stage in its lifecycle. More regular meetings are advisable at the start of UAT and through critical path or end-to-end test scenarios. Making sure signatories are kept informed throughout the journey will ensure a smoother and swifter sign off.
- Limited record keeping – keep your eyes on the essentials
UAT requirements are inevitably not static. While some of the less significant requirements may be de-scoped, given budget or time constraints, new requirements may also be requested. This requires effective change control procedures, assessing impact to time and cost. Changes should be brought to the attention of the business, reviewed and signed off through a formally agreed process. Additionally, an up-to-date record of requirements should be kept, as ultimately the success of UAT will be measured in accordance with agreed requirements.
UAT plays a pivotal role in every system release and should be proactively managed to avoid unwanted reputational or financial damage to a company. By being aware of these common pitfalls, a project manager should be able ensure a successful test phase. However, regardless of how much planning has taken place, unforeseen complications may still arise so it is important to build in adequate contingencies to guarantee a smooth transition to the next phase of the project.