Skip to content

Example library

Every idea in this library comes with a runnable example: a fraud desk, an emergency department, a moderation queue, an A/B test, a factory line. Each one runs in a second, prints a report you can read top to bottom, and compares what today's rule would have done with what the library does. This page tells you how to run an example, how to read its report, which example is closest to your own problem, and what data you would need to adapt it.

It assumes you have installed the package. Nothing else; terms are defined as they appear, and the glossary has them all.

Run an example

make examples                                        # list every example id by track
uv run grounded-decision examples fraud_review       # run one example
uv run grounded-decision examples fraud_review --json   # the same, as data
uv run grounded-decision examples --track step       # every example in one track
make examples-all                                    # run everything
make demo                                            # a short clinical-triage walkthrough

From Python:

from grounded_decision.examples.catalog import list_examples, run_example

result = run_example("fraud_review", as_json=True)
result["summary"]  # {"naive_acts": 3, "grounded_acts": 1, ...}
result["records"]  # one decision record per row

Every example returns {id, title, records, summary}. records holds one decision record per decision. It is an empty list for the examples that call a helper instead of making a decision (cache admission, facility location, and the others marked "helper" below); that is their normal behaviour, not an error.

Reading a report

The text report has the same shape in every decision example. Here is the middle row of uv run grounded-decision examples fraud_review, trimmed, with the parts labelled underneath.

tick 1/3  event=suspicious
  why    same $840, but linked only through one shared laptop chain - a naive rule would freeze the card
  INPUT  no saved cursor — this case starts fresh
         Event suspicious  {amount_usd=840, merchant=electronics}
         evidence  FLICKER, 4 observations
                   neighborhood d=1.5  λ=0.3333  strength dλ²=0.1667  (need > 1.05)  TOO THIN — below KS
                   density p̂=0.5  flicker band p*=0.08  FLICKERING (inside window — do not trust)
                   conformal  label=0.1  bag=40  p=1  vs α=0.10  in-distribution
                   support    share=0.5  vs min=0.50  supports the proposed action
  NAIVE  fire block without checking the graph
  DECISION
         hold — failed: reconstruction, threshold
         certificate      PASS  can we compile this policy (star-height ≤ 3)?
         game             PASS  can the policy keep making progress (parity win, nonnegative mean)?
         reconstruction   FAIL  do noisy neighbors recover the seed (dλ² > 1+margin)?
         threshold        FAIL  is density outside the operational flicker window?
         conformal        PASS  is this candidate weirder than the calibration bag?
         support          PASS  do the related cases support the proposed action?
  OUTPUT hold_for_human
         proposed_action=block  (blocked, kept on the record)
         extras['state']=review  (cursor to persist)
  NEXT   this case is done; the next row is an independent decide (fresh cursor)
Part What it tells you
The business problem (top of the report) The decision, who makes it today, and what each mistake costs.
index One line per row ("tick"): the event, a one-word evidence label, what the library did, and why the row exists. Read this first.
why The scenario this row represents, in domain words.
INPUT What arrived: the saved state for this entity (if any), the event (what happened, named the way the policy names it, plus its details), and the evidence: a summary of the related cases.
neighborhood d … λ … dλ² Neighbour agreement. \(d\) is the average number of links per related case; λ is how much more often linked cases share an outcome than chance (0 = chance). Their product \(d\lambda^2\) is the evidence strength; it must be above 1.05. "KS" is the line at 1. See Do the related cases agree?
density p̂ … flicker band p* Knife-edge check. p̂ is the fraction of possible links that are present; p* is your policy cut-off. "Inside window" means the density is so close to the cut-off, for this many cases, that a few links either way would flip the reading. See Are we on a knife-edge?
conformal label … bag … p … α Unusual-cases check. label is this case's unusualness score; bag is how many recent, correctly decided cases it is compared with (the calibration set); p is the share of those at least as unusual. It must be above α (0.10). See Have we seen cases like this?
support share … vs min … Support check. The share of related cases carrying the outcome the proposed action implies; it must be at least the minimum (0.50). Printed only when the example measures support (fraud review and clinical triage); elsewhere the check passes with support not measured. See Do the related cases support this action?
NAIVE What today's rule (always carry out the proposed action) would do.
DECISION The six checks, each PASS or FAIL with the question it asks. The first two check the policy itself and pass in nearly every example; the last four read the evidence. Any FAIL means the action is not carried out.
OUTPUT The verdict and what to do with it (below). proposed_action is what the policy wanted; it is always recorded. extras['state'] is the entity's state to save and send back with its next event ("cursor" in the printout).
NEXT What is stored for the next request, and whether naive and grounded agreed.
close Totals: what was carried out, what was held, how often today's rule would have acted where the library did not.

Labels used in the index and OUTPUT lines:

Label Meaning
fire <action> / ACT Act: every check passed; carry out the action.
HOLD <action> / ABSTAIN / hold_for_human Abstain with a proposal: a check failed; do not carry it out; send the case and its reason to a person. The record's verdict field says abstain.
DEGRADE <action> Degrade: carry out the reduced, safer action the policy names for this situation.
wait (no action yet) / await_next_event Abstain with nothing proposed: the workflow is waiting for its next step (for example, a request was submitted and awaits approval). Not a hold.
STRONG Evidence strength is above the line and nothing else failed.
THIN Evidence strength is at or below the line: too few links or too little agreement.
FLICKER The knife-edge check failed: density sits near the cut-off for this many cases.
WEIRD The unusual-cases check failed: this case looks unlike the calibration set.

--json gives the same information as data, plus a summary with naive_acts, grounded_acts, grounded_abstains and blocked_overconfident_acts (how many times today's rule would have acted where the library did not; degrades count here too).

Helper examples (no decision record) print the same headings with a different meaning: INPUT is the question asked, DECISION is the rule the helper applied, OUTPUT is what it picked, refused or measured, and NEXT is what your service should do with that answer.

Pick an example: real-world cases

Each of these has its own page written for the person who owns that decision. "Decision" cases make an act / hold / degrade call through a policy and evidence. "Helper" cases call a separate function that answers a hard or statistical sub-question honestly, or refuses.

Domain Example id Decision or question Kind What the demo shows Page
Health clinical_triage Discharge or escalate an emergency patient decision A low-risk score backed by only a few loosely similar past visits is held for the charge nurse. Clinical triage
Fraud and risk fraud_review Block a card or let the authorisation through decision A block resting on a thin chain of shared laptops is held for review; a block on a dense device ring goes ahead; a pass on a card inside that ring is held, because the ring says fraud. Fraud review
Fraud and risk parameter_learning Which check settings to use, learned from past outcomes decision Settings learned for "at most 5% wrong" act more often than hand-set ones (29.8% against 25.2%) and are wrong on 2.3% of acts instead of 6.9%; a monitor catches a drift. Parameter learning
Fraud and risk credit_underwriting Approve or deny a loan decision An applicant under the risk cut-off but riskier than any recent borrower who repaid is held. Credit underwriting
Trust and safety content_moderation Take a post down or leave it up decision A match to a loose cluster of template posts is held instead of taken down. Content moderation
Operations incident_response Auto-mitigate an alert or page a person decision A broad outage mitigates; a lone breaching monitor and a flapping new service are held. Incident response
Operations workflow_liveness Provision access after approval decision Approval provisions when the requester's peers hold the same access; otherwise it goes to security review. Workflow liveness
Operations cloud_autoscaling Scale a fleet out or in decision A one-sample spike, oscillating load and a too-short window are each held. Cloud autoscaling
Operations manufacturing_spc Stop the line or keep running decision A real drift stops the line; a thinly witnessed probe and a lone probe on a quiet line are held. Manufacturing SPC
Operations sensor_fusion Publish an impedance-imaging map decision The only example with a degrade: an unconstrained model publishes a qualitative image instead of a map. Sensor fusion
Operations cache_admission Give one premium cache slot to an object, with no take-backs helper Skip objects below yesterday's median, take the first one above it. Cache admission
Data science ab_test_tables Ship an A/B variant decision A lift that comes from two segments, and a run with a sample-ratio mismatch, are both held. A/B test tables
Data science community_detection Publish tonight's community labels decision Two of three nights are held: one too mixed, one too sparse. Community detection
Data science feature_selection Keep the single most useful yes/no signal helper An illustration of the API on data built from the chosen signal. Feature selection
Data science outcome_shift Is a lower mean latency an honest summary of a canary? helper One canary is a uniform speed-up; the other lowers the mean but worsens the tail. Outcome shift
Marketplaces and logistics matching_marketplace Which guarantees may a matching service quote? helper The router's answers for cut, cover and graph-fingerprint requests, including one refusal. Matching marketplace
Marketplaces and logistics facility_location What worst-case guarantee may a warehouse plan promise? helper Measure the plan's own gap instead of quoting a worst-case factor. Facility location
Refusals hilbert_refuse "Find the rational zeros of this polynomial" helper A request with no complete algorithm is refused up front, not left to hang. Hilbert refuse

If you are new, start with fraud_review: it is the most complete end-to-end template (policy, evidence built from raw rows, the stateless request handler). Then read Naive vs grounded for the pattern the cases share.

Learning tracks

The other five tracks teach one idea at a time, show behaviour over time, or run a whole desk. Each track has its own page.

Step by step (--track step)

A tutorial built on one access-request service; run them in order. Page: Step by step.

Example id What it teaches
step_hello The smallest policy: parse, decide, act.
step_reconstruction_gate A hold caused only by weak neighbour agreement.
step_threshold_window A hold caused only by the knife-edge check.
step_conformal_outlier A hold caused only by an unusual score.
step_require_calibration Refusing to act when there is no calibration set at all.
step_grammar_sequence A multi-step workflow: submit, then approve, with state kept between steps.
step_unknown_event_stutter An event the policy does not know: nothing moves, nothing is proposed.
step_parse_rejects_garbage A malformed policy is rejected with the tokens it expected.
step_star_height_reject A policy nested too deeply is rejected when it is built.
step_pipeline_and Two decisions combined: act only if both act.
step_json_roundtrip The same decision through the JSON request handler a service would call.

Evidence over time (--track adaptive)

The same policy run over a stream of changing evidence, to see when verdicts flip. Page: Adaptive loops.

Example id What it shows
adaptive_evidence_ramp Evidence strength rising steadily; the first tick that acts.
adaptive_threshold_walk Link density wandering through the knife-edge window.
adaptive_volatility_regimes The same starting point under calm, mild and wild noise.
adaptive_prior_grid A grid of links-per-case and agreement values: which cells act.
adaptive_workflow_over_time A multi-step workflow while the evidence improves.
adaptive_incident_loop The incident policy under mixed evidence.
adaptive_conformal_drift Scores drifting away from the calibration set.
adaptive_autoscaling_loop Load rising and falling through a regime.

Combinations (--track combo)

A decision plus a helper, or several decisions together. Page: Combinations.

Example id What it combines
combo_underwriting A credit decision plus an outcome-distribution check.
combo_marketplace_session Graph fingerprinting and a cover request beside a decision.
combo_moderation_tables A table-based significance test plus a decision.
combo_cache_and_features The one-slot cache rule and single-signal selection beside a decision.
combo_parent_and_session Two services over JSON combined: act only if both act.
combo_full_conjunction An evidence ramp, a refusal and the cache rule in one run.

Live service desks (--track live)

A worker processing a stream of events, with saved state per entity and a ledger of side effects. Page: Live service desks.

Example id Desk Today's rule
live_card_auth Card authorisation Capture every authorisation; freeze every flag.
live_ed_shift Emergency department Discharge below a heart rate, otherwise escalate.
live_trust_queue Take-down queue Take down when reports exceed yesterday's cut-off.
live_loan_desk Loan origination Approve every eligible scorecard result.
live_rideshare_match Ride dispatch Dispatch the nearest driver.
live_oncall_night SRE on-call Mitigate every alert; page on every "still bad".
live_line_spc Factory line Stop the line on any fault bit.
live_experiment_ship Experiment ship gate Ship when the observed lift beats a fixed margin.
live_cdn_peak CDN edge at peak Cache every object above a value.

Market data desks (--track markets)

The same worker on a simulated market (seeded, not vendor data). Page: Market data desks.

Example id Desk Today's rule
mkt_equity_halt Equity halt surveillance Halt on any large down print.
mkt_index_open Index opening auction Always send the opening order.
mkt_fut_roll Futures calendar roll Roll on the calendar date.
mkt_fut_margin Futures variation margin Liquidate on any adverse tick.
mkt_idx_opt_quote Index options auto-quote Quote every strike.
mkt_fut_opt_exer Futures-option exercise Exercise whenever in the money.
mkt_vol_target Volatility-targeting overlay Rebalance daily.
mkt_basis_arb Cash–futures basis Trade every small basis gap.
mkt_dispersion Index vs single-name volatility Always sell index volatility.
mkt_gamma_hedge Options gamma hedge via futures Hedge every fill.
mkt_vol_term Volatility term structure Always fade the gap.

Games

Ten games you can play or watch in a terminal, from fully known boards to hidden cells (Minesweeper) and randomness a world controls (snake's apples, 2048's tiles, Tetris's pieces). In each, a fast but fallible player proposes every move and a gate of exact checks lets it act or degrades to a safe move. Overview, the shared engine, tic-tac-toe and snake: Games; each other game has its own page.

Example id Game What it shows
games_tictactoe Tic-tac-toe Every square rated exactly; a rule of thumb that can lose becomes a player that cannot.
games_snake Snake A greedy player gated by a proven invariant never dies and fills the board faster than the safe cycle.
games_checkers Checkers (English draughts) A fast greedy player gated by a verified tactical search: proposals that lose material within the searched horizon are replaced. Page: Checkers.
games_chinese_checkers Chinese checkers A greedy jumper gated by progress and no-stranding checks finishes races without the classic stuck-straggler failure. Page: Chinese checkers.
games_go Go (Tromp-Taylor rules) A move proposer behind a gate. Page: Go.
games_hexxagon Hexxagon (Ataxx on a hexagon) A greedy converter checked against every reply: never loses an exchange it can avoid. Page: Hexxagon.
games_minesweeper Minesweeper (hidden information) Acts only on proved-safe cells; prices every guess exactly. Page: Minesweeper.
games_2048 2048 (worlds that place the tiles) Expectimax against worst-case search; a gate that never gives up guaranteed survival, proved in every world. Page: 2048.
games_tetris Tetris (worlds that pick the pieces) A look-ahead gated by exact checks over the preview; every loss announced in advance; the S Z loss theorems stated with their widths. Page: Tetris.
games_connect4 Connect Four Greedy gated by exact threat checks; an exact solver proves small boards and 7 x 6 endgames, cross-checked by brute force. Page: Connect Four.
games_breakout Breakout (worlds that pick capsules and spin) Exact integer physics; an aiming player gated by an exact intercept certificate and spin-proof bounces; every lost ball announced a flight ahead. Page: Breakout.

Reading a case

Every decision-case page has the same sections, in the same order, so you can skim to what you need:

  1. At a glance: the decision, its owner, today's rule, what this adds, and what the demo did.
  2. The decision today and where today's rule goes wrong: the problem in your domain's words, with no symbols.
  3. What the system checks, in your terms: each check as a question about your data, and a box saying what the checks do not verify. Read that box. The most important limit: three of the evidence checks ask whether the related cases are informative, not whether they point towards the proposed action. Only the support check asks that, and only where the case measures it. The direction comes from the event you send.
  4. Data you provide: each input, where it comes from, how often it is refreshed, and the minimum quality bar.
  5. Worked example: one row per scenario, with the numbers in a collapsed box.
  6. What happens next: for act, hold and degrade, who is told, the time limit, the fallback if nobody acts, and how the outcome feeds back.
  7. Tuning and expected workload, limits, risks and compliance, wiring it in, run it yourself, further reading.

Helper-case pages (cache admission, outcome shift, feature selection, facility location, matching marketplace, Hilbert refuse) have no act/hold verdict, so they use the same order with different names: "The question today" and "Where today's habit goes wrong"; "What the library will and will not claim" in place of section 3; "What your service does with the result" in place of section 6; then "Tuning and monitoring" and "Limits and risks".

Questions to ask of any case before relying on it:

  • Which check caused each hold, and would it fire on my data? (Several demos send a fixed unusualness score, so that check cannot fail in the demo even though it can in production; the case page says so.)
  • How many related cases do I typically have? Below about ten, the knife-edge check holds almost everything.
  • Who handles a hold, within what time, and what happens if they don't?
  • What does a wrong act cost here, and does the case's limits section cover my regulatory obligations?

Adapting a case: data checklist

Before you copy a case into your own system, check you can supply each of these. From your data to evidence and Build evidence from your data show the code.

  • [ ] Events. A fixed list of event names your upstream system (model, monitor, workflow) emits, each meaning one direction (suspicious / clean). The library does not decide the direction.
  • [ ] What each action implies. For each action, the outcome the related cases should carry if the action is right (block: confirmed fraud), so the support check can run.
  • [ ] Related cases. For each decision, a way to find the cases related to this one (shared device, similar visit, same content hash, same peer group) and which pairs of them are linked.
  • [ ] Outcomes for related cases. A confirmed outcome for each related case (fraud confirmed, patient deteriorated, post removed on review). Know the delay before an outcome is final, and never include the current case's own outcome.
  • [ ] Enough related cases. Typically ten or more per decision; the knife-edge window is very wide below that (half-width 0.5 up to about 6 cases, about 0.30 at 10, 0.21 at 16, 0.12 at 40).
  • [ ] A density cut-off with a meaning in your domain ("this dense is a ring, not a household"), chosen from history: Choose thresholds and calibration.
  • [ ] An unusualness score for each case (often your model's score or its error), and a calibration set of scores from recent cases decided correctly: at least 9 at the default α = 0.10, refreshed on a schedule.
  • [ ] State storage per entity (card, patient, incident) and a policy version stored next to it, so a saved state is never restored into a changed policy.
  • [ ] A hold path: a queue, an owner, a time limit, a fallback, and a way to write the human's ruling back as an outcome.
  • [ ] An estimate of the hold rate on last month's data before launch: Estimate impact before launch.

Where to go next