Quantitative Usability Sample Calculator

Use it when the goal is to measure usability, not to find out what breaks:

You need a number to compare against a target or against a previous version (success rate, time on task, errors, SUS)

You will report the result to someone who decides with numbers, not with user quotes

You want to estimate whether the difference between two designs is real or noise

When not to use it: if what you need is to understand why something fails, a qualitative study with 5 participants gives you more for less. Jakob Nielsen argued this in 2000 (Nielsen, 2000) and it is still the starting point: 5 users surface around 85% of usability problems. What they cannot do is estimate how many people run into them.

The two approaches do not compete. In our experience the order that works is qualitative first to learn what to measure, quantitative afterwards to size it.

The calculator solves n = Max(40, ((Za + Zb)^2 * s^2) / d^2) and keeps the larger value.

The floor of 40 comes from Raluca Budiu and Kate Moran (Budiu & Moran, 2021): it is the size that holds a 15% margin of error at 95% confidence for binary metrics such as success rate.

It replaced an earlier figure. Jakob Nielsen recommended 20 participants (Nielsen, 2006), based on a different assumption: comparing designs whose differences are large, not estimating a metric precisely. If you find 20 quoted somewhere, it is not made up, it is out of date.

The power calculation answers a different question: how many participants you need to detect the improvement you are after. Za = 1.96 (95% confidence) and Zb = 0.84 (80% power, the usual standard); s is your metric's standard deviation and d the difference you want to be able to detect.

If your metric varies a lot or the expected improvement is small, the calculation exceeds 40 and takes over. If the expected improvement is large, the floor governs.

Budiu and Moran also document cheaper scenarios: 28 to 30 participants if you accept 90% confidence, and 15 to 20 if you also accept a 20% margin. This calculator does not offer them because it takes the conservative scenario, but they are legitimate alternatives when the budget is tight and you take on the risk explicitly.

When you do not declare a standard deviation, the calculator estimates one from the metric. These defaults are a starting point, not a measurement of your product:

Time on task: 30% of the current value. Times spread out because a few participants get stuck and stretch the tail.

Success rate: 20 percentage points.

Errors: 50% of the current value. The most dispersed of the four, and therefore the one that demands the largest sample.

SUS: 15 points on the 0 to 100 scale.

The relationship is quadratic: doubling the standard deviation quadruples the sample. So if you have your own historical data, declaring it changes the result more than any other parameter.

Almost nobody arrives with a target margin of error; they arrive with a budget. So the calculator also solves the inverted question and shows, next to the result, what margin of error you get with 5, 10, 15, 20, 30 or 40 participants.

It is the same calculation read backwards. At a 50% success rate, the worst case, 40 participants give a margin of ±14.8 percentage points: that is exactly the ±15% Budiu and Moran (2021) describe, and it is where the floor of 40 comes from.

Precision improves with the square root of the sample, not the sample. Halving the margin means quadrupling participants: 10 to 40, 20 to 80. That is why an expensive quantitative study does not always deliver a much better number than a cheap one.

For success rates we use the adjusted-Wald interval, not the textbook one. At small samples the classic formula breaks at the extremes: for 10 out of 10 it returns a zero-width interval, as if you were certain nobody ever fails. The adjustment fixes that (Sauro, 2010).

For averages (time, errors, SUS) we use the t distribution over the standard deviation you declare, or over the default estimate if you leave it blank.

An honest caveat on times: Sauro (2010) reports, from 120 real tests, that 20 participants typically land near ±20% on task times. Our default estimate (30% of the current value) produces a smaller margin, so treat it as the optimistic end and declare your own standard deviation as soon as you have data.

40 participants means 40 sessions. Moderated one-on-one that is 20 to 30 hours of session time, before recruiting and analysis. Most quantitative usability studies run unmoderated, with remote testing tools, for exactly this reason.

If you are comparing groups, 40 is per group. Two countries, two segments or two versions means 80 people, not 40 split in half. It is the most common sizing mistake we see in quantitative study briefs.

Budget for no-shows. In unmoderated studies with an incentive, a 10% to 20% drop-off rate is common: recruit above the number the calculator returns.

In LatAm the bottleneck is usually recruiting, not the maths. Gathering 40 people with a specific profile in a single country in the region takes longer than running the study. Settle the panel before committing to dates.