UX testing isn’t just a natural fit for the Lean methodology: it’s an essential tool. UX testing gives you the “why” to explain the “what” provided by Google Analytics or similar measurement tools. Say you’ve just added a killer new feature, but your Google Analytics data shows that users are ignoring it. Why? Maybe it’s hard to find; maybe the link or button text is offputting; maybe the flow’s confusing. A quick burst of user testing can help find the answer(s), so you can make a targeted, effected response to the problem, rather than just guessing.
And yet too many people doing Lean ignore UX testing, because they think it’s a nice-to-have rather than a need-to-have, or that it’s hard to do, or expensive. That doesn’t have to be true. Here are five principles for successful integrating UX testing into your Lean process.
1. UX is not polish
Remember what UX stands for: user experience. It’s everywhere in your product, not just a layer on top. You could have the best idea in the world, but ignore the experience of using your site/app/new kitchen gadget, and it’s certainly going to fail.
From a Lean point of view, bad UX is noise masking the signal. If you want to test whether people are interested enough in your new product idea to sign up, spending a little time thinking about the UX means that your results will actually reflect people’s interest – not whether they can find the sign-up button or whether they can be bothered to fill in your long, confusing form.
Further reading: Aral Balkan’s excellent blog post on why design is not ‘veneer’, making similar points about the centrality of design to a product.
2. Test early and often
People often think UX testing requires a testing lab, dozens of test subjects, and weeks of work from a team of expert researchers. So they wait until they can afford all this stuff before running a test, and then they only test once.
This is a mistake.
In reality, a handful of users will be able to identify the majority of problems with your product; even one user will find a large percentage of the issues on their own. Some of the most revealing UX tests we’ve ever done took one user less than 5 minutes, and were completely free.
And, if you don’t need to wait til you can afford UX testing, why wait at all? Running a series of small, focused tests lets you can catch problems early, and steer away from the wrong path before you’ve gone too far down it. The longer you wait, the more time, money and momentum you’ll have wasted.
Further reading: UX guru Jakob Nielsen’s post on why you only need to test with 5 users
3. Choose your measurements carefully
This one should be familiar from other aspects of Lean. It may sound obvious, but every user test should be designed to answer a specific question/questions, and you should know before you start what sort of answer you’re looking for.
There’s a temptation with UX testing to just ask what users think of your site, and if they like it. But that results you’d get from this kind of test aren’t going to help you decide what you need to change, add, or remove. And when you use this kind of fuzzy data, you’re more likely to shape the ‘results’ into the answers you want.
Quantitive, observable data is your friend here. Don’t ask “do you like this site”; ask “on a scale of 1 to 10, how likely would you be to use this site”. Avoid asking people if they understand: you’ll only find out if they think they understand. Instead use measurements like the number of seconds it takes users to complete a task, or the number of unsuccessful attempts they make to discover whether they really understand.
Further reading: This video on the “Texas Sharpshooter fallacy” explains how easy it is to delude yourself into seeing the answers you want in your data.
4. Rethink the Minimum Testable Product
What if the product or feature you want to test isn’t finished yet? No problem: don’t test the product.
There are a dozen quick and dirty ways of testing without using your actual product. You can make paper prototypes, or turn Balsamiq wireframes into a clickable mock website, or upload images of your designs to InVision to make a more high-fidelity interactive prototype.
People often dismiss these methods (especially paper prototyping) because they don’t look and act exactly like the real website, and people assume this will distract users and distort testing results. But you’d be surprised how quickly people suspend their disbelief and start using the prototype just like they’d use a website.
All of these quick and dirty Minimum Testable Products/Features will tell you important things about whether users can navigate through your site, discover features, follow the flow you expect them too, and much more. They just won’t tell you if users like your colour scheme.
Read more: Here’s a great introduction to the leanest prototyping tool out there: paper prototypes.
5. Get out of the bubble
When people doing Lean do try UX testing, they often do what’s called “hallway user testing”: stopping someone walking past their desk, or their office, or making tea in the kitchen and getting them to quickly try out their product. This ‘guerrilla testing’ is fast, it’s easy and it’s free. What’s not to like?
The big disadvantage of guerilla testing is the sample bias. Think about who you share your office/building/kitchen with: it’s usually other people working in tech or start-ups. Unless this is your target audience, testing with them won’t give you all the information you need – and it’ll often lead you astray. For instance, tech folk are often power-users who’ll ask for loads more features than your average user. They often love novelty, so they’ll rave about a cutting-edge gestural interface that will confuse others.
By all means, use guerilla testing (especially if this gives you access to your target audience). But also step outside your filter bubble and test with other users. Cheap online tools like WhatUsersDo, ClickTest, ChalkMark, Five Second Test, Optimal Sort (and more) make it possible to get rapid feedback from different types of people.
UX testing is an essential part of your Lean toolbox – and it can be as fast, cheap and easy as you need it to be. So what are you waiting for?