Consistency Score

In 30 Seconds

• It grades the venue, not your question — is this market × timeframe trustworthy ground for statistics at all?
• "Certified" means measurements here have held out-of-sample — patterns measured on this venue have historically carried forward.
• "Provisional — weight lightly" means passed on thinner evidence — real, but lean on it less.
• Two layers: the badge grades the *venue*; a report's honesty check grades *this specific pattern*. Different questions.
• Refusals are the point — a tool that won't certify some markets is the only kind whose "Certified" means anything.

Definition

The Consistency Score is the certification badge shown at the top of every HIE report. It applies to a venue — a specific market and timeframe together, such as ES on the 1-hour, or NQ on the 1-hour — and it answers one question: *do statistics measured on this venue have a track record of staying true out-of-sample?*

That question is not the same as "does this number match the past." Every statistic HIE reports is counted history by construction — it always matches the past, because it *is* the past. Matching the past is therefore no test at all. The Consistency Score asks something one level deeper and far more useful: when behaviour is measured on this venue in one era, does a *later* era — data the measurement never saw — behave measurably consistently with it? A venue where the answer is yes is ground where historical measurements deserve weight. A venue where the answer is no is ground where a number, however precisely computed, may be decoration.

You'll see the badge in three states:

Consistency score: Certified

Consistency score: Provisional — weight lightly

Consistency score: Suppressed — not shown for this instrument

HIE tells you which of these you're standing on, plainly, rather than displaying a score it can't stand behind.

Why It Matters

Every backtesting tool ever made will happily compute a win rate on any instrument you feed it — including instruments whose behaviour shifts so much over time that the number is meaningless the moment it's printed. The precision looks identical whether the ground is solid or quicksand. That is the trap the Consistency Score exists to close.

By grading the venue first, the badge answers a question that protects you before you read a single statistic: *should statistics exist here at all?* When a market fails the exam, HIE declines to sell you numbers on it — it would rather refuse revenue than hand you meaningless precision. And that refusal is exactly what gives the approvals their meaning: a tool willing to say "we can't certify this market right now" is the only kind whose "Certified" on another market carries information. A badge that certified everything would certify nothing.

For a trader, internalising this changes how you read *every* statistic you ever meet — not just HIE's. You stop trusting numbers by default and start asking "on what ground was this measured?" That single habit is one of the strongest defences there is against precise-looking figures built on shifting sand. Practically: a trader who ignores the badge may spend time interpreting statistics from ground that has already shown itself unstable. Reading the badge first is what decides how much confidence to place in everything that follows.

Why Most Tools Don't Do This

Traditional backtesting tools assume that if they *can* calculate a statistic, they should show it. The Consistency Score starts one step earlier. Before asking whether a number is interesting, it asks whether that number deserves to exist at all on this market and timeframe. That difference sounds small, but it changes what the software is willing to refuse to show — and refusing to show meaningless precision is the whole point.

The Two Layers (the part people miss)

This is the most important distinction on the page, because conflating the two layers is the most common misreading.

There are two separate honesty questions in every HIE report, and they are not the same:

1. The Consistency Score (this badge) grades the venue: is this market × timeframe trustworthy ground for statistics *in general*?
2. A report's own honesty check grades this specific pattern: did the exact condition you asked about hold up out-of-sample?

Both can be true at once. A report can sit on a Certified venue and still tell you that *this particular pattern's* direction flipped on newer data. That is not a contradiction — it's two different facts: the ground is trustworthy, and this one idea didn't hold on it. The badge says "measurements here can be trusted in general." The honesty check says "here's whether *this* measurement held." Read them as the two separate signals they are.

The two layers are the same idea — out-of-sample consistency — applied at two levels: the venue exam asks whether the *market's* statistics hold out-of-sample; the per-report honesty check asks whether *this pattern* held. Same principle, two targets.

How It Works (what you can observe)

The Consistency Score is produced by an exam applied per venue. What you can observe about it, from the outside:

• It's earned per venue, not asserted. Each market × timeframe is graded on its own; certification on one venue says nothing about another.
• It's re-earned, not permanent. Certification is tied to the data behind it; when the underlying data is refreshed, the exam is applied again. A venue's status can change.
• Failing is public. When a venue doesn't pass, HIE shows the absence of certification rather than hiding it — the refusals are visible, which is what makes the approvals credible.

What the exam measures is whether statistics on a venue remain true out-of-sample. *How* it decides — the thresholds, the constants, the internal computation — is part of MPM's proprietary methodology and isn't published, for the same reason no research operation publishes the exact internals a competitor could copy. The page states what the badge means and why it exists; it does not disclose how the score is computed.

There is also a third state, for a venue that hasn't passed the exam:

Consistency score: Suppressed — not shown for this instrument

Note what happens here, because it captures the whole idea: a suppressed market is still *queryable* — HIE will run your question and show you the path statistics — but it refuses to attach a consistency score, and tells you so directly. It shows you the numbers and declines to vouch for the ground beneath them. That honest refusal is the point: a tool willing to stamp "Suppressed — not shown" on a market is the only kind whose "Certified" on another market carries any weight.

Interpreting the Badge

• Certified — read the report's numbers as measured. The venue has a track record of statistics holding out-of-sample. (Still not a guarantee about the future — see Limitations.)
• Provisional — weight lightly — the venue passed on thinner evidence. The numbers are real, but the foundation is lighter; the badge says so in its own text. Lean on it less.
• Suppressed — not shown for this instrument — the venue hasn't passed the exam. HIE will still run your query and show the path statistics, but it declines to certify them and says so. Treat the numbers as uncertified ground.

Whatever the badge says about the venue, still read the report's own honesty check for the specific pattern. Certified ground can carry a pattern that didn't hold; provisional ground can carry one that did. Two layers, always.

Common Mistakes

"Certified means this pattern works."

No. Certified grades the *venue*, not your pattern. A Certified market can carry a report whose specific pattern flipped out-of-sample. The badge and the pattern's honesty check are different questions.

"Certified means the numbers matched the past."

That's true of *every* report on every market — all HIE statistics are counted history. Certification is deeper: it's about whether measurements here have stayed true on data they never saw.

"Provisional means the report is wrong."

No. Provisional means real evidence on a thinner foundation. The badge's own "weight lightly" text is the instruction: use it, but lean less.

"An uncertified market means HIE has no data there."

Not necessarily — it means the venue didn't pass the consistency exam, so HIE won't present weight-bearing statistics on it. Declining to certify is a deliberate act, not a data gap.

Limitations

The Consistency Score is evidence about a venue, not immunity. A market that has passed the exam can still change in the future — out-of-sample consistency measured through a given date does not bind what happens after it. That is precisely why HIE keeps directional readings as context even on Certified venues: the exam filters out ground that has *already* proven unstable; it cannot guarantee stability forever.

The badge is also a venue-level judgement. It tells you the ground is trustworthy in general; it says nothing, on its own, about the specific pattern in front of you — for that, read the report's own out-of-sample honesty check. And like every measure here, certification is only as current as the data behind it; a refresh can change a venue's status.

How This Fits Into MPM

The Consistency Score is one expression of MPM's core discipline: a statistic is only worth showing if the ground it stands on has been tested. It's the venue-level half of a two-part honesty system — the per-report out-of-sample check being the other half.

You'll encounter it in:

• HIE (Historical Intelligence Engine), where the Consistency Score badges every report, grading the venue before you read a single number, and where markets that can't be certified are declined rather than dressed up.
• Research Papers, where MPM applies the same standard — reporting whether findings hold out-of-sample, and being explicit about the ground a claim stands on.
• Intelligence Circle — a member-only research environment containing advanced market research, historical investigations, and trading strategies developed using the MPM research framework.

MPM's broader stance is that the honest question is never "does this fit the history I searched?" but "does it hold on history it never saw?" — and the Consistency Score is that question asked of the venue itself.