I’ve often seen usability tests run with only a very small group of participants. At first, this seems reasonable just a few sessions should be enough to surface the main issues, right? In practice, though, a tiny sample often leads to misleading insights and creates its own risks.
A small sample is a trap: the findings are usually random and not representative of the broader user base.
For example:
- If a study involves only a handful of participants, three of them might struggle with the same task. That could appear to be a “major usability issue,” but with a larger sample it becomes clear it was just an isolated case.
- Conversely, the same small group might all complete the flow without difficulty, leading to the false impression that “everything works perfectly.” With more participants, it often becomes obvious that critical usability problems still exist.
This reflects a basic principle of research: small samples are more likely to produce extreme or skewed results.
When a usability test involves a larger group 15, 20, or even 30 participants the picture changes dramatically. Clear patterns begin to emerge. If the same issue appears across different users, it’s a strong signal worth prioritizing. This helps product teams separate critical usability problems from minor or edge-case issues. Individual struggles no longer distort the overall picture.
Examples
- With a small sample, three users fail to locate the search bar. It may seem like a critical usability issue. But when tested with 20 participants, those three turn out to be the exception — the majority had no trouble finding it.
- With a larger sample, 18 out of 25 participants struggle during the checkout flow. Here, the data speaks for itself — this is a genuine usability problem, not coincidence.
Conclusion
Usability testing is always valuable, but its impact multiplies when the sample size is chosen appropriately.
- Small samples are useful for quick validation or early discovery.
- Larger samples are necessary when you need reliable insights to guide product decisions with confidence.
The true strength of usability testing lies not only in well-designed tasks and scenarios, but also in using the right sample size. The more representative the group of participants, the closer you get to understanding the real user experience.
