UserTesting, UserZoom, Usabilla, and many other tools out there offer tools for UX researchers and designers to conduct remote UX testing. When we work with our clients and they want to use these tools to run a project, they always ask us for our recommendations on how to structure a survey or a navigation test. It really depends on the type of study you’re running and the results you’re looking to achieve. However, there are some general rules that apply across the board.
- Keep the study short. Research has shown that the attention span of a participant dips after the 20 minute mark. They get testing fatigue, so they’re more likely to drop out of your study after twenty minutes. That sounds bad, but what’s even worse is if they continue on to take your hour-long study and they progressively perform worse over time. This makes your data pretty much worthless. Generally speaking, aim for 10-15 minutes. If you can keep it under 10, then you’re golden.
- Make an effort to build questions that identify and remove cheaters from your study. If you’re running a study and you’ve enlisted the help of panel providers (companies that get participants to take your study), then you better make sure you have questions that test the participant to see if he/she really took the time and made an effort to go through your study. Panel participants are expert survey takers. I’ve seen people finish a 20-minute survey in 5 minutes. They’re just clicking away, so that they can get their cash/points rewards afterwards. Obviously, as a good researcher/designer, you should go through the data and clean it. But, just to make your life easier, you can ask a question about the content of whatever you’re testing or even ask a totally random question (e.g. Here is a number pattern. What number comes after the following pattern? 1, 3, 5, 7, …) to test if they’re really reading through the questions you’re asking them.
- Whittle down your basic demographic questions. You don’t need to ask about their whole life story. Figure out what demographics info you really need.
- Make sure your prototype works on whatever platforms and devices you’re using AND that it works with whatever testing tool you’re using. I can’t emphasize enough how important this is. There are technical limitations to these remote UX tools, and make sure you know about them before you start building your prototype. Sometimes, the tool can’t track InVision prototypes or the bottom half display of the prototype is cut off. If you want to be agile and rapid test your prototypes, you have to do the legwork first and figure out what works and doesn’t work with these UX testing tools. Don’t get caught in a situation where you have to keep modifying your prototype just to run the study.
- Keep open ended questions to a minimum. Save them for your in lab studies. Open ended feedback from remote testing are often not useful, because participants don’t want to write a paragraph about their experiences with your study. Also, you’re not there to probe them further to gain insights. So, you’re very likely to get superficial answers from open ended questions.
- Do a soft launch. Test your study out by getting 10-20 participants to take your study first. Make sure you set up your study correctly. A lot of clients forget about this and think that they can get an infinite amount of participants to take their study, because it’s an “online” study. No, this is not how it works. Think about your population size. Then, think about what percentage of those people who use your product. Then, think about the percentage of people who use your product and who are willing to take your study. On top of that, think about the additional criteria you have for this study. The number of potential participants keeps getting smaller and smaller. Don’t accidentally waste your sample because of a mistake you made. Test your study. Run a soft launch. Then, do a full launch.
- Don’t make your participants read too much. If you’re providing them with a scenario for your study, don’t write a novel. Try to describe it in 2-3 sentences. Anything longer is going to be hard for them to remember.
- Try not ask participants to write anything down as part of the study. A lot of clients want to ask participants to take down a fake credit card number or a coupon code that they have provided and use it to make a purchase on their website. Participants will forget or write the info down incorrectly. If you must have them take down some information, just realize that it may be more difficult for participants to complete your study.
- Think aloud studies are the rage these days, but think really hard about what exactly are you looking to achieve with think aloud studies. It’s not the most natural thing for people to talk aloud as they’re browsing through a website. The ones who do are usually trained to take these type of studies.
- If you’re running a mobile study, remember that the average attention span on the phone is even shorter than on the desktop. People don’t stay on any particular apps for too long. They may spend 20 minutes on your website with their undivided attention, but when they’re browsing your app, they may be watching TV, cooking, or doing a million other things. Tailor your test to fit mobile expectations.
There are a lot more to remote UX testing. If you’re starting out on adding remote testing into your toolkit, then these ten tips will save you a lot of time and will help you get quality data. Until next time, happy testing!