08 September 2017

What is UAT?

User Acceptance Testing (UAT) is exactly what the definition implies; solution testing that must meet certain agreed criteria to be accepted by the end users. It constitutes the final and most important step of a software development process to allow the acceptance of the overall solution prior to a release in production. It is mandatory for UAT to be satisfactory and meet the business requirements, by testers/users before moving in Production Environment. During UAT, the end users successfully test the software, following explicit test scripts developed by the business to validate and verify that the software is fit for purpose and it can perform the required tasks in real-life case scenarios.
For those who have been involved in UAT, it is widely known that it is better to fix a bug in the beginning rather than having to fix it post-release, as the financial and operational impact is very high. That is why when users start UAT they want to find problems in the system and therefore, most of the time they use extreme test case scenarios to check the limitations and the workload the system can handle.

But this is the last part of the chain. There are many challenges that need to be overcome prior to UAT and many aspects of the business that need to work well together to ensure successful release. These might appear because of poor planning and requirements gathering, ineffective communication, delays in delivering the requirements or the fixes, poor training of the users/testers or even not making the right use of the available resources. This list is not exhaustive, but the aforementioned issues are the most important and will be further analysed.

Ineffective or non-existent communication

One of the most common denominators that almost all the implementations have, is poor communication between software developers, testers and project management team, especially when (as is common) the teams are based in different locations and even countries where they also have to overcome a language barrier. Time difference can delay the reporting of problems and the fixes. Lack of BA expertise in IT might also lead in poor requirements gathering which means IT resources (developers) proceed ahead without fully understanding the requirements of the end users which will use the actual solution. Hence, the team should log the issues online following strict guidelines agreed and followed by everyone. Also, recruiting the right resources with sufficient language skills and training them so that they acquire consistent understanding of the process flow is crucial at this point.

Industry aversion to change

Another major challenge that is ubiquitous, is the fact that some more traditional industries (e.g. insurance) are more adverse to change than others. Either it is the old fashioned way (such as lack of digitalisation) or the internal politics, there are cases that the business tries to find problems in every solution and hence, reasons to reject the new software and the suite of its offerings (e.g. logging in claims in real-time in case of an accident, ability to calculate the premium someone needs to pay without waiting for someone to process it etc). This is genuinely a very difficult problem to tackle and the key is to try to identify the business relationships as early as possible and construct a strong and positive relationship.

Scripta manent

While words can be easily said and forgotten, written information always provides proof of what has been agreed, what has been delivered and what is yet to be delivered. Change requests and general changes must be communicated through the appropriate channel and shared with the PMO, whilst making all the interested parties/stakeholders aware of the change and its implications. An orally-agreed over the phone change will complicate things and lead to irreversible changes that might affect the testing itself, the planning, the regression, but also may result in many team members not being aware of the change. It is highly important to cover all the requirements in a Business Requirements Document, as this is what the team of developers should adhere to and the document that will be revisited in case there is a change request or new requirements need to be added (something that is feasible when using the Agile methodology). Last but not least, documentation shouldn’t be detached from capturing the lessons learned, as this is what will prevent any future mistakes or help any new endeavours.

Make right use of your resources

When the software development has finally come to UAT and the right resources have been recruited for the testing, businesses have to dedicate adequate time and planning to ensure that the key deliverables are effectively communicated to the testers with the latter being sufficiently trained for the testing. The testing itself must comprise of test cases supported by the corresponding test scripts. However, most of the time, in real life, test scripts or manuals are very poor or non-existent and the dedicated resources spend most of the time trying to understand the test cases, without having clear steps to follow.

Also, the output end acceptance criteria have to be defined thoroughly in each test case, so that the end results of testing are consistent and in agreement with the set expectations. It is not wise having a team of testers unsure of what they are testing or not understanding the results they obtain.

Speaking about expectations, it would be a miss not to mention the expected time to correct each bug. Having a failing test not fixed, which impacts multiple subsequent tests, is detrimental to the whole UAT timescales. For this, a possible solution is to define the Service Level Agreements (SLAs) before UAT begins, so that the response time in fixing the right bugs doesn’t affect the whole testing.

In an ideal world, the rate problems are fixed should be faster than the rate they are created. However, testing could be described as a hurricane of bugs that wait to be fixed with unsatisfactory response times. In this case it is highly crucial that triage calls (calls that are arranged according to priority/severity of issues) are set up for P1 (high priority) problems at the end of the day, to set up the work and priorities for the next day. Testers should also be aware that once they start a test, this needs to be finished within the same day and has to be specific with the test case under investigation which also needs to finish within a reasonable amount of time. Keeping things tidy and finishing what has already started keeps everything more organised and on track.

Last but not least, the infrastructure and the standards of the UAT environment need to be similar to the production. Otherwise testers will be faced with the most common case of having to wait for hours to run a test that normally in a production environment take minutes, seconds or less.

UAT is not an easy task as the teams can face many unpredictable bugs and unforeseen problems while performing the testing. However, eliminating the aforementioned risks and issues will contribute to having fewer surprises and more chances of going live. It is critical that the above practices are adhered to when applied in an industry such as financial services given the pace of innovation and technology change and ultimately the benefits to companies if built, tested and deployed successfully.