114 usability tests in three days: putting the RCN’s new website in front of nurses at Congress
The RCN’s new website was in beta, and it wanted to know what members thought before full launch. Box UK ran 114 guerrilla usability sessions at RCN Congress and turned the results into prioritised recommendations.
114
usability sessions
3
days at RCN Congress
5k
attendees to recruit from
In short
The Royal College of Nursing’s new digital platform was in beta, and it wanted members’ views before full launch. Its annual Congress brought more than 5,000 attendees to one place for three days.
- Box UK set up guerrilla testing with desktops, tablets and phones in a busy area.
- Moderated and self-guided sessions ran side by side, plus interviews, questionnaires and feedback cards.
- 114 sessions in three days gave enough data to segment findings by audience.
- The RCN received prioritised recommendations and a Net Promoter Score for the redesign.
Customer background
The Royal College of Nursing represents around 440,000 nurses in the UK and worldwide. Box UK had already run discovery for its new digital platform, and the new site was in beta. See also the Royal College of Nursing case study.
The challenge
The RCN was investing in a new platform and wanted to be sure it worked for members before full launch. Congress offered direct access to thousands of them, but only for three days, and attendees had little time to spare.
We needed an approach that could gather a lot of useful feedback, quickly, from very different users: qualified nurses, students, health practitioners and accredited representatives.
Our approach
We chose guerrilla testing: portable equipment, short test plans and sessions of 10 to 15 minutes. We set up desktops, tablets and phones in a high-traffic area and invited attendees to explore the platform and share their thoughts.
To get through as many sessions as possible, we ran self-guided tests using third-party software alongside moderated sessions, and supplemented them with interviews, questionnaires and feedback cards. Because we’d already run discovery for the RCN, the test plans focused on the journeys that mattered most.
“The way the Box UK team conducted things was fantastic and the response and feedback we have captured I am sure will prove invaluable to the further development of the website. It was an absolute pleasure to work with you.”
Paul Newman, Head of IT at Royal College of Nursing
What we delivered
- Heatmaps and journey maps showing how users moved around the site.
- Qualitative feedback from questionnaires, interviews and cards.
- A Net Promoter Score giving the RCN a quick, high-level view of member feeling about the redesign.
- Prioritised recommendations, segmented by audience, to guide the next stage of beta development.
After Congress, our UX consultants combined the heatmaps and journey maps with the questionnaire responses, and presented the findings to the RCN as prioritised recommendations for the next stage of beta development.
The outcome
The RCN went into the next stage of beta development with evidence from 114 sessions with members and other Congress attendees. Because the findings were split by audience, from qualified nurses to students, it could decide which changes to make first based on the value each would deliver.
- More confidence in the finished site, with member needs built into the next stage of development.
- Members involved early, seeing progress and knowing their feedback was being used.
- A positive Net Promoter Score for the redesign: 49.
Frequently asked questions
Quick, informal testing in a public place, with short sessions and portable equipment. At RCN Congress, it let Box UK run 114 sessions in three days.
Keep sessions short, run self-guided tests alongside moderated ones, and add questionnaires and feedback cards. That’s how the RCN tests reached 114 sessions.
Because your users are all in one place. The RCN’s Congress gave direct access to thousands of members.
It gives a quick, comparable measure of how likely users are to recommend the product, which is useful for tracking sentiment across releases.
What this means
for your organisation
If you’re about to launch a new platform and your members gather in one place each year, that’s your best chance to test with them at scale. A few days of short sessions can give you a lot of evidence in very little time.
If you have an event coming up, send us a few lines about it and what you want to test. We’ll tell you how we’d set it up.
Talk to one of our digital experts
Sophie Collins
UX Designer
Do you need help maintaining product quality and business alignment?
Passionate about all things User Experience and Agile, Sophie has worked across the entire software delivery lifecycle.
With a background in Product Ownership and Software Testing, Sophie approaches UX with both pragmatism and quality, focusing on producing results which have value for both businesses and users.
Or call us on 020 7439 1900
Related case studies
-
TBC Bank Usability Testing
Read more: TBC Bank Usability TestingRealistic kiosk testing that found problems before development began
-
Royal College of Nursing
Read more: Royal College of NursingA new website for the RCN’s 440,000 members worldwide
-
Royal Photographic Society
Read more: Royal Photographic SocietyUser research that shaped the brief for RPS’s new website
Have a project you’d like to discuss?
Give us a call on 020 7439 1900 or fill in the form and we will get back to you.