PinPoint Performance Solutions Blog (Legacy Content)

Not often do I take on the role of participant in a UAT. Most of the time I am either an administrator or a moderator.

I'm deep into a project with a client and I decided it would be fun and informative to see the other side of things in their environment and I volunteered to take part in the testing of an application they are soon to launch. I also chose to participate because it would give me the end-user's perspective when it comes time to develop the training for the application.

The group administering the test is doing a good job, but I noticed a few things that could have made the UAT, overall, produce better results and be more objective.

1. Audience - The team did a good job picking the right people to fill the role of participant. These people, after all, are the target audience for the application. The only trouble was the limited schedules of the participants. The test was to be conducted over the course of a week, however it wasn't until the end of the week did most people begin participating in the test. When conducting a UAT, make sure the test participants have the time to conduct the tasks. You want to have a consistent, steady stream of test results that you can react to as soon as possible or uncover trends that may point to problems that can be resolved. If the issues or bugs discovered are "showstoppers", you can alert your developers so they can prepare to focus their attention to these issues and bugs once the test concludes.
2. Monitoring testing and recording results - As you probably guessed, the test participants were left to conduct testing on their own schedule, unmonitored. I feel most UAT's should be monitored to pick up on difficulties the test participants are experiencing. Most go undocumented by the participants. They may see them as minor inconveniences and not worthy of documenting, but in the long term they can cause usability issues that diminish the overall user experience. Watch the participants as they go through the test and solicit feedback from them. What you see and what you hear may reveal more insightful data.
3. Outline of test objectives and expectations - I think test participants should have some general idea of what is expected of them during the test. They don't need to know the overall test objective. If they do, it could be leading and might skew your test results. However, give them some checkpoints or milestones they need to reach for their tasks. This way, just like what was mentioned earlier, the test results are consistent and you can see trends emerge that you can alert your developers to ahead of time.

In all it was a a good UAT, but the lesson learned is that it is necessary while conducting any kind of testing to make sure you are consistent, objective and working toward specific goals. This way you can identify gaps in performance and establish a course of action to resolve the problems quickly and efficiently.