News
User Testing Should Hurt!

11 May 2021
At Varda Care, we don't take criticism personally. We want everyone we interact with to share their honest opinions about the service we provide. It's the only way we can make our service even better. It can be painful, we admit, but it's absolutely necessary. That's why we've made user testing an integral part of our product development process.
A Continuous Process
We managed to conduct three rounds of user testing before the end of last year and have already planned a couple more for this year. We've carried out testing at every stage of the design process, from clickable prototypes to the fully developed service. You might think it's a relief to finally be done with it – but the truth is, user testing is never really finished, nor should it be. This is something we'll carry forward, making user testing a central part of our research and development (R&D) efforts.
Our first round of user testing was conducted by Vest Studio in Ålesund, where we tested the clickable prototype they had designed for our service. The testing took place at their offices in central Ålesund, where we at Varda Care could observe from a separate room without disturbing or influencing the participants. We learned a great deal, both about the service itself and how we should communicate our value proposition.
What We Learned
1. Observation Speaks Louder Than Words
Observing the user testing firsthand was incredibly valuable! We sat in a room next to the testing area, where we could watch the participant's screen as they completed tasks, see a video feed of the participant (to observe facial expressions and reactions), and listen to the audio.
A lot of valuable information can be lost in a summary after a user testing session, information you can pick up on when observing in person. For example, we noticed that one participant spent a lot of time searching and scrolling, even though they said it was very easy to navigate. This is valuable insight you can only gain by observing the user directly.
We recommend having several people in the observation room taking notes throughout the session, recording not only what participants say but, more importantly, what they actually do.
2. Don't Assume You Can Do It Yourself
Don't moderate the user testing yourself! The moderator is the person who sits with the participant and guides them through the tasks, among other things.
You might think you can remain neutral, but you have too much knowledge of and emotional investment in the service to guide a participant through it objectively. Participants are also more likely to feel relaxed if the person conducting the test doesn't work for the company or hasn't been involved in designing the service.
We didn't want participants to feel they had to choose their words carefully or look to the moderator for hints or reactions.
3. Clearly Defined Tasks
Even though you shouldn't help participants during testing, they still need some guidance on what to explore or accomplish. Define the tasks in advance and make sure they aren't too extensive.
If a participant spends a lot of time on a task, don't rush them just to complete everything on the list. Participants can provide valuable insights into different aspects of the service, and your goal is to understand why they experience difficulties with specific tasks.
Don't guide them through the process, even if they ask for help. Instead, respond to questions with questions to gain further insight, such as, "What would you have expected to happen next?"
Make sure to ask open-ended questions as well. Open-ended questions encourage dialogue, discussion, and the sharing of information. They invite participants to describe their experiences, explain their thoughts, evaluate situations, and reflect on their actions.
4. Every Response Is Valuable
There are no wrong answers or incorrect ways to use the service! All feedback is valuable.
It's important to make this clear at the beginning of the session: it's the service being tested, not the person using it.
Never dismiss a response or label a participant as someone who "just finds things difficult." If the service was too difficult for them to use, then the service is too difficult.
5. Just Five Participants
Five participants per round of testing are enough. We initially thought five sounded like a very small number, but experience shows that the number of new usability issues discovered tends to level off after testing with five participants.
We saw this well-known user testing principle confirmed during our own sessions.
Feel free to test with more participants, but make sure there is enough time between each round to make design improvements based on the findings from the previous tests before moving on to another group of five.
This has been an incredibly valuable experience, and we've learned a great deal from it! Feel free to get in touch if you'd like to know more about the process.