Lab-based usability testing is expensive and hard to set up, right? Wrong. You can fit all you need to get fantastic insights about your product, design or prototype into a bag that even the meanest budget airline will let you carry on to a plane.
Don’t get me wrong: lab-based usability testing in an actual lab can offer a surgically precise environment that can yield high-quality participants and great-quality outputs but it isn’t the only way to test with real people.
I’ve spent the last year working for a software company that has over a million users. When I started, I couldn’t believe my luck – all of those people to speak to, all of those insights to glean! Not only that, no-one had asked our customers whether they’d like to be involved in testing out new versions of our products before. When I did, the response was overwhelmingly positive. It showed the great depth of the relationships that we have with our customers. They wanted to help us to make our better products. From a UXer’s perspective, it doesn’t get much better than that.
Building my own lab
Once I had a prototype to test and it had been through a couple of days’ of usability testing in the lab (I chose to do this to get the harshly objective view of representative members of the public and to benchmark the prototype against the current version of the system), I took my own lab on the road.
Here’s what my lab was made up of:
- 1 x 15-inch Macbook Pro – I insist on this because the built-in camera and microphone are of superior quality, QuickTime is already installed, and the 15-inch screen is a bit more practical than its 11-inch sibling
- 1 x iPhone 5. I use this for testing mobile designs as the majority of our mobile customers are using iPhones and the iPhone 5 has the added advantage of offering a viewport width of 320px, which matches the narrowest Android and Windows phones used by our customers
- 1 x ‘standard’ mouse – don’t use the built-in trackpad or a magic mouse – these throw off non-Apple users (otherwise known as most people in the workplace!)
- 1 x printed copy of my testing tasks – I needed this because my laptop was being used for testing. I also prefer writing scores while testing because it shows the participant that I’m not distracted by another task on another device
- 1 x printed Excel spreadsheet to record my usability scores (I favour a 1-to-7 usability scale)
- 1 x ballpoint pen (don’t use a Sharpie as its nib is too big for writing scores on your sheet – plus they’re a bit whiffy at close quarters)
- 1 x copy of QuickTime (comes as standard on a Mac) to record the screen and the voice of the participant; or
- 1 x copy of Lookback. This product is truly awesome and can give you high-quality playback, Morae-style commentary, and clips that you can send links to and more. Check it out at: https://lookback.io
And, yes, all of that can fit in a bag that you can carry on to a plane.
5 is the magic number
As any good UXer knows, testing something with just 5 representative users should uncover around 85% of all major usability problems (https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/). You can test with more, of course, but you’re getting into the land of diminishing returns. What this means is that you’ll probably just waste time depressing yourself by discovering the same usability problems over and over again when you could be designing another iteration to test with another 5 users.
Failing fast and moving on is the way to go – not dwelling on what doesn’t work. Keeping things lean also lets you experiment (‘what if we got rid of that?’, ‘what if we totally reimagined this?’ – that sort of thing.). My mobile lab has let me do that, time and time again, without having to worry about the expense of hiring a real lab each time.
On the road…
So, once I’d created my mobile lab I just needed to ask customers if they were happy to lend me a meeting room, their wifi and 5 people for individual, 30-minute testing slots. I assured them that I would bring all of the equipment we’d need and would share my findings with them. When possible, I also promised doughnuts.
Using that approach, I’ve been able to notch up more than 50 hours of testing with around 90 people from 3 different countries. The results have been invaluable and have shaped the design of a new version of my company’s flagship product that went live late last year. Without my mobile usability lab there’s no way the product would have been as successful as it’s already been.
So, UXers, do you have your own mobile usability lab? If so, what’s in it?