Time efficient user testing
– posted October 7th, 2009 by Ruairi Galavan No comments
Photo by photomastergreg
For the majority of projects, analysts rarely get to spend more than a day or two testing with users. This short period of time is both critical and costly, so it is very important to get it right. Over the years, we’ve gotten pretty good at it. Here are some tips on how best to use your time with your users.
Plan, pilot and prepare for the worst
There is nothing more detrimental and time consuming to a user testing session then a hardware breakdown. Set up your testing equipment the day before if possible, test it, and then test it again. On the day of the user testing, arrive at least an hour before the first participant, and test the equipment again.
Always have access to a spare machine, mouse, keyboard, monitor, camera, microphone, and plug board (Yes, we really do bring two of everything when we go on site).
Pilot your test with colleagues, friends, or family. This gives you an idea of how long your test will last with each user, helping you to avoid running over time on the day, or finishing up too early.
Silence is golden
Administering a user test without asking leading questions is a difficult skill. We all slip up from time to time, probing for information and actions we think we want. Most of the time, getting the best insights requires us as test administrators to remain silent.
After we have posed a task to a user, let them at it. If they get lost, try to remain silent. Try to ride out the awkward silences that often ensue. If they make a comment like “this is confusing,” try repeating it to them as a question; “confusing?”. Remember, your goal here is to observe natural behaviour, and while you are probing users with questions, you may actually be directing them around an application.
Effective note taking and scoring
As an observer of a test, it is important not to take too many notes. Taking too many notes into the analysis phase is daunting and time consuming. Usually, we find that more notes are required for the first couple of users on the day. But, as the same issues reoccur, we simply mark these again with a tick (to signify each additional time it occurred), or make short hand references to remind ourselves of the issue.
If you are scoring your users on task completion, try to establish a simplified scoring scheme. For example, fail = 0, success with difficulty = 1, and success without difficulty = 2. This is particularly useful in larger studies, when you are testing many tasks with many users.
Scheduling is key
One of the most important things you can do to utilise your time is to schedule breaks into the day. Ideally, we like to run no more than 6 tests in one day, with a 30 minute break between each one. There are valuable benefits to this schedule.
Firstly, anyone who has administered and/or observed a full day of user testing will tell you how tiring it is. The breaks from the tests help to keep you fresh for the next one.
More importantly however, the break allows you to collect your thoughts, and record the key findings from the previous test. Quite often, we even start constructing a mind map in between the first and second user (we like to use Mindmeister). As the day goes on, we add to it, and iterate it, until we have a rounded idea of our findings and the problem space.
When the last tests ends at 7pm, you’re probably exhausted want to go home. Don’t. Hang around for another 30 minutes with your colleague. Quickly run through all the key findings and discuss their weighting, relevance, and priority. Finally, make some last amendments to the mind map.
By scheduling breaks, and using them proactively, you can jump straight into analysis the following morning. This is time effective usability testing.
The alternative to all of this of course, is to schedule 6 tests back to back, an hour for lunch, duck home at 5pm, and come back in the next day for a full day of video play back, struggling to remember what happened, and which users did what.
Now it’s your turn
And there you have it: from planning to note taking to analysis. A little know-how will make sure you get the most out of your user tests.
Digg this post Submit to del.icio.us Submit to technorati
Categories Usability
