Skip to content

Coverage for unfamiliar cases (conformal prediction)

A scorecard says "approve", and the score is under your cut-off, but the applicant looks riskier than every borrower who repaid last quarter. The model has never been tested on cases like this one, so its confidence means little. This page explains the statistics the library uses to notice such cases — split conformal prediction — what its guarantee says, what it assumes, and exactly what the unusualness check in the code does.

It assumes you know what a model score and a p-value are. Symbols are collected on How to read these pages; the plain-language version of this check is The unusual-case check. Unlike the other checks, this one rests on no result from the project's math catalog; it is standard statistics.

The situation

Each decision carries an unusualness score for the case being decided (the field label_score) and a calibration set (the field scores): the same score computed for recent cases where the outcome is known and the answer was right. Higher scores mean "less like a typical case". Statisticians call these nonconformity scores.

The question this check answers is: how unusual is this case, compared with the calibration set? If it is more unusual than all but a small fraction of the calibration cases, the library holds the decision for a person and records the action it would have taken (proposed_action).

Intuition: rank the new case among the old ones

Sort the calibration scores and drop the new case's score into the list. If the new case lands in the middle, it is ordinary. If it lands above every calibration score, it is the most unusual case in the room.

Now suppose the new case really is drawn the same way as the calibration cases. Then it is equally likely to land in any of the \(n_{\mathrm{cal}} + 1\) positions. The chance that it lands at the very top is \(1/(n_{\mathrm{cal}}+1)\); the chance that it lands in the top 10% is about 10%. That is the whole guarantee: if you hold every case that lands in the top \(\alpha\) share, you hold an ordinary case at most an \(\alpha\) share of the time, whatever model produced the scores.

Worked example: three loan applicants

Run:

uv run grounded-decision examples credit_underwriting

The scorecard predicts a probability of default and approves below 12%. The calibration set depends on the proposed action (this is label-conditional conformal prediction):

  • for approve, the score is the predicted default probability, and the calibration set is the predicted probabilities of the 40 past loans that were repaid (0.010 to 0.1075);
  • for deny, the score is 1 minus the predicted probability, and the calibration set comes from the 20 past loans that defaulted.
Applicant Proposed Score Calibration scores at least as large p-value Check Verdict
FICO 760, 2% risk approve 0.02 36 of 40 \((1+36)/41 = 0.9024\) pass act approve
FICO 610, 11% risk approve 0.11 0 of 40 \((1+0)/41 = 0.02439\) fail hold, proposed_action=approve
FICO 480, 45% risk deny 0.55 20 of 20 \((1+20)/21 = 1\) pass act deny

The second applicant is under the 12% approval cut-off, so a plain scorecard approves. But 11% is higher than the predicted risk of every borrower who actually repaid, so the check holds it. The example prints

conformal  label=0.11  bag=40  p=0.02439  vs α=0.10  weirder than calibration
...
reason: conformal p=0.024 vs α=0.100 (reject)

The other four checks passed on that applicant: the hold is entirely this check's doing.

The precise statement

Setting. Calibration scores \(S_1, \dots, S_n\) and a new score \(S_{n+1}\). Define the p-value

\[ p = \frac{1 + \#\{\, i \le n : S_i \ge S_{n+1} \,\}}{1 + n}. \]

Theorem (validity of conformal p-values). If \(S_1, \dots, S_{n+1}\) are exchangeable — their joint distribution does not change when you reorder them, which holds for example when they are independent and identically distributed — then for every \(\alpha \in [0, 1]\),

\[ \Pr(p \le \alpha) \le \alpha . \]

The bound holds for every finite \(n\) and every way of computing the scores, provided the scoring rule was fixed without looking at the calibration cases or the new case ("split" conformal: the model is trained on one set of cases and calibrated on a separate one). Ties only make \(p\) larger, so the bound still holds. If the scores have no ties, also \(\Pr(p \le \alpha) \ge \alpha - 1/(n+1)\), so the test is not much more cautious than it needs to be.

Label-conditional version. If you calibrate only on past cases with outcome \(y\) and score a new case whose true outcome is \(y\), the same bound holds conditionally on the outcome. The credit example uses this (Vovk and co-authors call it Mondrian conformal prediction).

Where this result stands

In the wider literature: a theorem, with the exchangeability hypothesis above (Vovk, Gammerman and Shafer, Algorithmic Learning in a Random World, 2005; Papadopoulos and co-authors, 2002, for the split version).

In this project's math catalog: no result is used. The check carries no catalog reference, and every result is tagged operational.

What this library relies on: only the arithmetic of the p-value. It does not check exchangeability or that the split was kept. See Where each result stands.

What the guarantee means for you. If the calibration cases are representative of today's ordinary traffic, then at \(\alpha = 0.10\) the check holds at most 10% of ordinary cases. It does not say the cases it passes are correct, and it does not bound how many unusual cases it passes.

What the code does

conformal_report(label_score, scores) in grounded_decision/evidence/conformal.py, called by every decision with the snapshot's label_score and scores:

\[ \text{pass} \iff p > \alpha, \qquad \alpha = \texttt{conformal\_alpha} = 0.10 \text{ by default,} \]

with \(p\) exactly as defined above (a linear scan; no sorting).

  • Equality fails. \(p = \alpha\) holds the decision.
  • There is a smallest possible p-value: \(1/(n_{\mathrm{cal}}+1)\). So a hold is possible only when \(1/(n_{\mathrm{cal}}+1) \le \alpha\):
\(\alpha\) Smallest calibration set that can ever hold
0.10 (default) 9 scores (\(p = 1/10\), which is not above 0.10)
0.05 19 scores
0.01 99 scores

With 8 scores and \(\alpha = 0.10\), even the most extreme case gets \(p = 1/9 \approx 0.111\) and passes.

  • Empty calibration set. With no scores, \(p = 1\) and the check passes, so a module with no calibration data has no protection from this check. Set Limits(require_conformal_calibration=True) and an empty calibration set holds every decision instead, with the reason no calibration scores (require_conformal_calibration). Run uv run grounded-decision examples step_require_calibration to see both: the same request acts under default limits and is held under the stricter setting.
  • Trust is always operational, and the check has no catalog reference.
  • The record's metrics carry p_value, alpha, n_cal, label_score and require_calibration; the decision's Health carries conformal_p, so a dashboard can plot \(p\) against \(\alpha\) over time.

Every difference from the theorem:

  1. Exchangeability is assumed, never checked. If today's cases differ from the calibration period (a new product, a recession, a changed model), the 10% bound no longer holds, in either direction.
  2. The split is your responsibility. If the new case's own score is in scores, or the scoring model was fitted on the calibration cases, \(p\) is inflated and the check passes more than it should.
  3. The scores are yours. The code compares numbers; it does not compute them. If you pass "higher is better" numbers (a model's confidence), the check holds the most typical cases and passes the most unusual ones.
  4. Several examples use a fixed score. In fraud_review, content_moderation and clinical_triage the candidate score is a constant (0.1 or 0.12) below a fixed calibration ramp, so this check never fails there; step_conformal_outlier sets the score to 1.5 by hand to show a failure. The examples that compute real scores from data are credit_underwriting, incident_response, manufacturing_spc and cloud_autoscaling; each has one case held by this check alone (\(p = 1/41\) or \(1/31\)).

Limits, and what to do about them

Unusual is not the same as wrong, and usual is not the same as right

The check measures distance from the calibration set. It does not look at the related cases or the density, and it does not know which action is correct. An ordinary-looking case passes even if the model is wrong about it. The direction of the decision comes from the event you send; the other checks ask whether the evidence is strong enough; this one asks whether the case is like ones you have seen.

  • Building a score from your model. Common choices: for a classifier, \(1 - \hat p(\text{proposed outcome})\) (one minus the model's probability for the outcome the action assumes); for a regression, the absolute residual \(\lvert y - \hat y\rvert\) on calibration cases and a predicted-error proxy for the new case; with no model, a share of related cases that contradict the case, as manufacturing_spc does. Calibrate on cases whose outcome is known, scored by a model that never saw them.
  • Label-conditional sets. When the action implies an outcome (approve implies "will repay"), calibrate on past cases with that outcome, as credit_underwriting does. A pooled calibration set mixes good and bad outcomes and makes the check much weaker.
  • Choosing \(\alpha\) and the size of the set. \(\alpha\) is the share of ordinary cases you are willing to send to a person. Make sure the calibration set is large enough for that \(\alpha\) (table above); 40 or more is a comfortable size at the default. Change \(\alpha\) per module with Limits(conformal_alpha=...). See Choose parameters.
  • Refreshing the set. Replace calibration scores as new outcomes mature (the credit example uses last quarter's matured loans). A set that is too old breaks exchangeability; one that is too small cannot hold anything.
  • Production. Turn on require_conformal_calibration so a missing calibration set holds decisions instead of passing silently.