About this post

This is one of a special series of unboxing posts exploring the different options available for usability testing, covering the relative benefits of each as well as when to choose one approach over the other. From low-cost rapid remote testing through to moderated lab-based testing, there is no reason why some form of iterative improvement through user testing shouldn’t be a key part of every one of your digital projects, in order to create best-of-breed user experiences that deliver the highest levels of customer satisfaction and return on investment. For more information, take a look at our post comparing the various approaches on offer. 

Mobile usability testing overview

The objective of mobile usability testing is largely the same as other forms of testing: to gather insights about the performance of your digital products and services by observing how users perceive and interact with them. However, the unique contexts and specifications of mobile mean that the way users act can be significantly different from typical desktop behaviour, making it important to test across all commonly-used devices.

Although it can be conducted remotely, mobile testing is typically run face-to-face with a facilitator, as part of a moderated session. The focus is on capturing rich and detailed user insights, using special software on both smartphone and tablet to record the screen, participant’s face, audio (as they speak aloud during tests), and finger gestures and interactions. Due to its portable and light-weight nature testing can be performed anywhere, which means our participants are typically tested in their natural environment rather than a lab.

Participants are set a scenario (for example: “Imagine that you are due to fly from Cardiff airport to Paris next month for a two-week holiday” and then asked to complete a series of related tasks using either their own mobile device or a specific test device (e.g. “check where the closest car park to the airport terminal is” “find out the cost of the cheapest suitable car parking option available” “complete the online booking form” etc.). Sessions are normally between 45 minutes and one hour, with a Q&A session at the end where the client can discuss with the facilitator any issues that may have arisen.

Participants

The number of participants can vary from between 6 to 12 according to project scope and budget. Participants are recruited based on their demographic criteria, skills and experience so that they provide the closest match to target end-users.

Deliverables

Mobile testing deliverables are very much dependent on a project’s specific requirements, but they may include:Before the test:

  • A test plan document outlining the proposed schedule, participant profiles, tasks, questions, session scripts, key research objectives and forms

After the test:

  • A video combining screen recordings, participant finger gestures and body language
  • Recordings taken from device software that captures facial expressions as well as where the user is clicking, or attempting to click, on the screen
  • A summary report and/or stakeholder presentation covering key findings, participant comments, recommendations and next steps
  • A detailed report covering all findings from minor to severe, usability metrics such as time on task and completion rates, and participant comments, recommendations and next steps

When to use it

Mobile testing can be applied to:

  • Any point in the project lifecycle; for example, while one company may want to identify areas that require improvement on an existing site another might want feedback on an early concept or HTML prototype
  • Benchmarking the performance of existing mobile services or new, unreleased products
  • Mobile websites, native apps, and responsive solutions
  • A wide range of devices and operating systems, although we typically tend to test experiences on Apple iOS and Android smartphones and tablets

Example project

For example, a banking client wanted to improve their current customer-facing iPhone app experience. As part of a project to deliver a User Experience (UX) strategy and designs that would support the development of a new smartphone banking app, we initially started by performing usability testing of their current app in order to benchmark the experience alongside identifying and prioritising key opportunities for improvement. Later in the project we also tested key user journeys and task flows (such as primary navigation approach, checking account balances, making transfers and paying a bill) with early interactive prototypes, in order to validate and refine the mobile experience ahead of the development sprints.

Considerations

It’s important to note however that, as mentioned above, people can behave very differently when interacting with mobile devices compared to desktop. One of the key differences is the length of time taken to complete typical tasks; users commonly require rapid information and services via mobile, so make sure your testing plans and scripts take this into account. Related to this is making the user feel comfortable enough to act naturally during the testing, which may require posing real-world scenarios (such as being on the move or having limited time to complete a task). For an even more authentic experience you could employ users’ own devices when testing, although this may impact other areas of the test; for example, they probably won’t have the correct recording software installed. On a technology note, you will also need to think about access to charging points, reception and WiFi, which mean that, although mobile testing can be conducted anywhere, it may be beneficial to retain the control and reliability offered by a lab environment.

At Box UK we have a strong team of UX consultants with hundreds of hours’ testing experience, and a state-of-the-art lab set up to provide the best environment for the participant, facilitator and observer. If you’re interested in finding out more about how we can help you, get in touch with us by calling +44 (0)20 7439 1900 or emailing ux@boxuk.com.