Sample Size in Trading

Definition

Sample size is the count of observations behind a statistic — in trading, usually the number of trades, but it can also be the number of signals, sessions, or historical instances of a pattern. A "55% win rate" or a "1.6 profit factor" is meaningless until you know whether it came from 12 trades or 12,000 representative ones.

The reason sample size matters so much is that results from small samples are dominated by chance. Flip a fair coin four times and it's entirely possible to get four heads — a "100% heads rate" — without the coin being biased at all. Flip it ten thousand times and it will land very close to 50%. Trading statistics behave the same way: over a few trades, luck can produce almost any result; only over a large sample does the underlying reality show through. The purpose of increasing sample size isn't to eliminate randomness — it's to reduce its influence, so the underlying behaviour becomes easier to observe.

This is why sample size is not a footnote to a statistic — it is part of the statistic. A number reported without its sample size is incomplete, and often misleading.

Why It Matters

Every metric in strategy evaluation — win rate, expectancy, profit factor, maximum drawdown — rests on a sample of trades, and each becomes trustworthy only when that sample is large enough. A strong-looking figure from a small sample tells you almost nothing, because a small sample simply hasn't given the underlying reality enough chances to reveal itself.

This cuts in a direction people find uncomfortable: the more impressive a result looks over a small sample, the more suspicious you should be, not less. A strategy showing a 90% win rate over 20 trades hasn't proven it wins 90% of the time — it has shown that, over a short run, it happened to. The same is true of a spectacular expectancy or a flawless-looking backtest with too few trades: small samples are exactly where flukes live, and flukes are exactly what looks most impressive.

To ground it with no complex math: imagine testing a strategy that in reality has no edge at all — a coin flip. Run it over 15 trades and, by chance alone, a meaningful fraction of such tests will show a "winning" record. Someone looking only at that record, without asking how few trades produced it, would conclude they'd found an edge. They'd have found noise. This is the single most common way traders fool themselves, and the defence against it is always the same: ask how large the sample is before believing the result.

What Counts as "Enough"

There's no single magic number, because it depends on what's being measured and how variable the outcomes are — but some principles hold:

• Small samples (roughly under 30) are generally too few to conclude much from. A single unusual outcome can swing the whole result. Figures from samples this small should be treated as preliminary at best, not as findings.
• More variable outcomes need larger samples. A strategy with wildly different trade results needs many more trades to pin down its true behaviour than one with consistent results.
• Bigger is genuinely better — up to the point of relevance. More trades tighten the estimate, but the sample also has to be representative: ten thousand trades all from one calm market tell you little about turbulent ones.
• Consistency across the sample matters. A result driven by one extraordinary trade, or clustered in one short period, is weaker than the same result spread evenly across a large, varied sample — even if the trade count is identical.

The practical takeaway: a number is only as good as the sample behind it, and "enough" means enough trades, varied enough conditions, with no single observation doing most of the work.

MPM Perspective

Sample size is closer to the centre of MPM's thinking than almost any other single idea, because it is what separates a measurement from a coincidence.

Every historical measurement MPM publishes is reported with the number of observations behind it — how many times price interacted with a comparable zone — precisely so a reader can judge how much weight it carries. A figure drawn from thousands of interactions is treated very differently from one drawn from a handful. And where a particular case has too few observations to support a conclusion, MPM's practice is to say so plainly and withhold the claim, rather than present a thin number as if it were solid. A result with an inadequate sample is not quietly rounded up into a finding; it is flagged as inadequate.

This is the discipline behind MPM's wider scepticism of good-looking results. A striking figure from a small sample is not a discovery to celebrate — it is the most likely place for a fluke to hide, and therefore the first thing to distrust. The same standard applies to strategy evaluation: a win rate, expectancy, profit factor, or drawdown means nothing to MPM without an adequate sample behind it. Reporting the sample size, and refusing to over-claim when it's too small, is one of the clearest ways a research platform demonstrates that its numbers are real. (As throughout this cluster, this concerns how MPM evaluates evidence; it is distinct from, and applies equally to, MPM's measurements of historical price behaviour around zones.)

Common Misconceptions

"A great result is a great result, however many trades."

No — this is the central error. A great result over few trades is most likely luck. The fewer the trades, the more a striking figure should be doubted. Impressive-looking small samples are the classic trap.

"More trades always means more reliable."

Larger is better, but only if the sample is also representative. A huge number of trades all drawn from one type of market, or one calm period, can still mislead — size doesn't cure a biased sample.

"If the sample is big enough, the result is guaranteed to hold."

No. A large, representative sample makes a result more trustworthy as a description of the past — but markets change, and past behaviour never guarantees future behaviour. Adequate sample size removes the noise problem; it doesn't remove the fact that conditions can shift.

"One or two exceptional trades don't matter if the sample is large."

They can matter a great deal. If a single outlier trade drives most of a result, the effective sample behind that result is much smaller than the trade count suggests. How the result is distributed across the sample matters as much as the count.

Limitations

Sample size is essential, but it is not the only thing that determines whether a number is trustworthy. A large sample can still mislead if it is unrepresentative (drawn from one market, one regime, or one period), if it is contaminated by look-ahead bias or other methodology errors, or if the wins are concentrated in a few outliers. Adequate sample size is necessary for a result to be believable — but it is not, by itself, sufficient.

And even a large, clean, representative sample only describes what happened; it cannot promise the future will match. Markets evolve, and an edge measured honestly over thousands of past trades can still fade. Sample size protects against mistaking noise for signal — the most common error — but it does not, and cannot, guarantee that a genuine past signal will persist.

How This Fits Into MPM

Sample size is the idea that underpins every other page in the MPM Learning Center's evaluation series — and the discipline behind MPM's historical measurements.

You'll encounter this thinking in:

• Research Papers, where every result is reported with the number of observations behind it, and conclusions are withheld where the sample is too small.
• Reaction Library, where MPM's published measurements of historical price behaviour (how often zones held versus broke) are always accompanied by the number of interactions they're based on — so readers can weigh each figure for themselves.
• Intelligence Circle — a member-only research environment containing advanced market research, historical investigations, and trading strategies developed using the MPM research framework.

MPM's broader stance is that sample size is not a technicality but the foundation of credible evidence: a number without an adequate, representative sample behind it is not a finding, and should not be presented as one.