Close

Usability testing methods

Usability testing has the unfair reputation of being expensive and time-consuming, but that’s simply not the case. It’s even been found that usability testing can deliver significant Return on Investment (ROI), with studies showing that for every dollar an organisation spends to increase usability it receives between ten and 100 dollars in benefits, and that organisations which redesign for usability see an average improvement of 83% in key performance indicators. This evidence presents a strong argument that, in fact, you cannot afford not to conduct usability testing on your products and services as part of the product development lifecycle. With lots of testing methods out there, from quick, lightweight ‘guerrilla’ testing through to specialist lab-based services, usability testing can be employed at any point throughout a project. This post covers the strengths, and limitations, of each method to help you determine which is most appropriate for each stage of your project.


Guerrilla testing

Referring to its ‘out in the wild’ style, guerrilla testing is performed by User Experience (UX) consultants using a portable testing kit in order to gather feedback from users in their natural environment.

Benefits

  • Rapid feedback and insights
  • Reduced overheads, with no recruitment fees and small participant payment
  • Conducted in a natural environment
  • Full control over session timings and test plan
  • Captures facial expressions and body language
  • Perfect for running testing sessions throughout the project lifecycle

Limitations

  • Geographic diversity is limited to a single location
  • Risk of failing to find suitable participants
  • Reliance on third-party WiFi and battery life
  • With shorter sessions fewer tasks can be tested
  • No real-time observation
Our guerrilla testing unboxing post has further details about the approach as well as real-world examples.

Remote (moderated) testing

In the moderated remote method, testing takes place outside the lab with users participating in their own home using their own computer/device, and a UX consultant on-hand to moderate sessions via third-party software.

Benefits

  • Excellent qualitative results
  • Perfect when particular specialisms and/or experience are required
  • Can be performed in multiple locations
  • Full control over session timings and test plan
  • Sessions can be observed in real-time
  • Conducted in a natural environment

Limitations

  • More expensive than some other types of testing
  • Heavily reliant on internet connection and excellent audio quality

Remote (unmoderated) testing

Like moderated remote testing, unmoderated user testing takes place outside the lab with users participating in their own home using their own computer/device; however, unlike the moderated approach no third-party facilitator is present.

Benefits

  • Rapid feedback and insights
  • Cost-effective due to availability of large third-party participant databases
  • Conducted in a natural environment
  • Not limited to a single location
  • Flexibility of a browser test environment
  • Perfect for running testing sessions throughout the project lifecycle

Limitations

  • Reliance on third-party software
  • Risk of participants failing to follow test plan or not thinking aloud
  • No real-time observation
  • No control over session timings and test plan
  • Cannot capture body language

To learn more about remote testing, both moderated and unmoderated, please read our unboxing post

Lab-based testing

A ‘qualitative’ research method, in lab-based testing participants are recorded in a dedicated laboratory with each session facilitated and observed by UX consultants.

Benefits

  • Excellent qualitative results
  • Captures facial expressions and body language
  • Perfect for testing when participants with particular specialisms or experience are required
  • Full control over session timings and test plan
  • Sessions can be observed in real-time
  • Most likely to deliver best ROI

Limitations

  • More expensive than some other types of testing
  • Geographic diversity is limited to a single location
  • Poorly designed test labs can be intimidating, e.g. one-way mirrors (although we don’t use them at Box UK)
To find out more about lab-based testing, take a look at our unboxing post on the approach.

Conclusion

The most appropriate type of usability testing is typically dependent on project budget and time constraints. But, having observed hundreds of hours testing I firmly believe the best approach is to mix methods in order to capture both quantitative and qualitative results, and also to use different approaches to test frequently, from early market research through to post-launch testing. For example, although I find laboratory testing is best for delivering excellent user insight, guerrilla testing is useful for capturing feedback very quickly, while dozens of sessions can be run at a modest cost using unmoderated remote testing.

Get in touch

If you’ve realised that you can’t afford not to include usability testing in your projects, and want to learn more about Box UK’s experience of running successful user testing projects across multiple platforms and industry sectors get in touch today.

About the author

Gavin Harris

Gavin Harris

Gavin Harris is an experienced Senior User Experience Consultant specialising in user testing, Information Architecture, HTML prototyping and web accessibility. Over the years he has worked with the likes of the Virgin Group, Land Rover, Kia, Orange and Unilever, and many other enterprise organisations.

Comments

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Jan 1st, 0001

Add Your Comment

Related content

We're hiring. Let's talk. View available roles