Skip to content

Live service desks

A single decision is easy to follow. A real service is harder: events for many customers, patients or machines arrive on a stream, each one must pick up where that entity left off, the evidence has to be rebuilt from fresh data, and every verdict has to turn into something that happens (a card frozen, a page sent) or a case put in front of a person. This track runs nine such services ("desks"), each through one shift, so you can watch that whole loop in domains you recognise.

It is for engineers wiring the library into a stream or request handler, and for desk owners (fraud operations, an emergency department, a moderation team, an on-call rota) who want to see what their shift would look like. After reading it you should be able to predict what each desk does with each event, say what a person on the desk would see when the system holds, and know which pieces you replace to run a desk on your own data. It assumes you have seen one decision end to end (Your first decision); words such as verdict, proposed action and state are defined in the glossary.

uv run grounded-decision examples --track live
uv run grounded-decision examples live_card_auth
uv run grounded-decision examples live_card_auth --json

The evidence on this page is simulated

The examples cannot query your warehouse, so every desk's evidence comes from a seeded simulator that returns a graph of a chosen shape. The numbers below (agreement strengths, densities, p-values) are illustrations of how the checks behave on those shapes. They are not measurements of any real card network or hospital, and they are not targets to copy into your configuration.

How every desk works

Every desk is the same loop, run once per incoming event:

  1. load the entity's saved state (where it is in its policy; empty for a new entity);
  2. load the evidence: the related cases around this event, as a small graph;
  3. call handle_request with the state, the event and the evidence;
  4. route the verdict: act runs the action, a hold (the record says abstain with a proposed action, routed as hold_for_human) goes to a person, and an abstain with nothing proposed (await_next_event) means the policy is waiting for its next event;
  5. save the new state and the full decision record together.

In the examples that loop is LiveDesk in src/grounded_decision/examples/livefeed.py, with an in-memory dictionary for state and a list for the ledger. The production version of a live desk is the reference service, src/grounded_decision/examples/reference_service.py, which you can start with make service. It adds what LiveDesk leaves out: a database for state with a policy fingerprint on every row (so a saved state is never restored into a different policy), a lock around the module (it is not thread-safe), idempotency keys for retried requests, an outbox so side effects happen once, a review queue, metrics and a shadow mode for rollout. Build a decision service walks through it; Integrate into a service shows how to swap each piece for the one your platform already has.

In LiveDesk In the reference service and in production
module the compiled policy, built once per worker process
saved_state_by_entity a state table keyed by entity, with the policy fingerprint
ledger the outbox of side effects and the human review queue
records the append-only audit log of DecisionRecord.to_dict()
tick / process the body of your request handler or queue consumer

Three habits every desk shows and every real service needs:

  • Save the state even when the action is held. The policy moves on every event it recognises; only the side effect is withheld. live_oncall_night below shows what goes wrong otherwise.
  • Fail closed on missing calibration. The desks compile with Limits(require_conformal_calibration=True), so an empty calibration set makes the "have we seen cases like this?" check fail instead of pass. (live_oncall_night reuses the incident-response policy, which uses the default limits but always supplies calibration scores.)
  • Log the whole record. It names the checks that failed, which is what the person handling a hold needs.

What the evidence is, desk by desk

Each desk grounds its decision in a corroboration graph: the related cases that bear on this decision (members), which pairs of them genuinely corroborate each other (links), and the last known outcome of each member (labels). In production this is two warehouse queries; evidence_from_corroboration_graph turns their result into the EvidenceSnapshot the module reads. How to build one from your own tables is in Build evidence from your data.

Desk Members Linked when Label
live_card_auth accounts in the card's device neighbourhood logged in from the same device in 30 days last fraud review: confirmed or cleared
live_ed_shift patients from the last six hours same chief complaint and similar vitals final disposition: admitted or discharged
live_trust_queue accounts that reported the post this hour reports corroborate (same category, independent) reporter's past reports upheld or not
live_loan_desk applications and customers share a phone, address, employer or device repaid or charged off / fraud
live_rideshare_match drivers and riders in the zone, 15 minutes matched with each other last trip completed or cancelled
live_oncall_night probes and hosts across regions error rates moved together over 10 minutes burning or healthy
live_line_spc sensors on the line readings moved together fault or ok
live_experiment_ship traffic segments (country by platform) conversion moved together variant beat control or not
live_cdn_peak client networks requesting the object requested it in the same minutes human or automated traffic

The simulator produces four shapes of graph, and every tick on this page uses one of them. Knowing the four shapes is enough to predict every verdict on the page:

Shape (kind in the output) What it looks like What the checks say
dense 24 members in two tight groups; most pairs inside a group are linked; labels agree all pass: the related cases are informative and clearly away from the cut-off
rumor (a "rumour graph") 12 members, only three linked pairs; those pairs agree agreement fails (three agreeing pairs are three anecdotes) and knife-edge fails (too few cases)
windowed 50 members, link density 0.21 against a cut-off of 0.20 knife-edge fails: a few links more or less would flip which side of the cut-off it is on
outlier the dense graph, but this case's unusualness score is above every calibration score unusual-case check fails: nothing confirmed recently looked like this

Three of the four data checks, in one line each (each links to its full explanation); the fourth, support, is not measured by these desks and passes with support not measured:

  • Do the related cases agree? The links per case times the square of how much linked cases agree beyond chance; we call this evidence strength (dλ²). Above 1.05 the related cases are informative enough that this case's outcome is recoverable from them; at or below it, they are not.
  • Are we on a knife-edge? Fails when the graph's link density sits so close to the desk's cut-off that a little noise would flip it. Small graphs have wide margins.
  • Have we seen cases like this? Fails when this case's unusualness score is higher than almost every score in the calibration set of recently confirmed cases.
The numbers behind the four shapes

All from uv run grounded-decision examples <id> --json, field summary.ledger.

  • Agreement. Passes when \(d\lambda^2 > 1.05\), where \(d\) is the average number of links per member and \(\lambda\) is how much more often linked members share a label than chance (1 = always, 0 = chance). Dense graphs on this page score between 2.8 and 9.8. The rumour graph has \(\lambda = 1\) but \(d = 0.5\) (three links among twelve members), so \(d\lambda^2 = 0.5\).
  • Knife-edge. Fails when the measured density \(\hat p\) is within a half-width of the cut-off \(p^\ast\). The half-width is \(\min\big(0.5,\ \ln 5 / (\ln n)^2\big)\) for \(n\) members: about 0.26 at 12 members, 0.16 at 24 and 0.105 at 50. The rumour graph's density, \(3/66 \approx 0.045\), is within 0.26 of both cut-offs used here (0.08 and 0.20). The windowed graph's 0.21 is within 0.105 of 0.20. Dense graphs (0.40 to 0.46) clear both.
  • Unusual cases. Passes when \(p = (1 + \#\{\text{calibration scores} \ge \text{candidate}\}) / (1 + 40) > 0.10\). The calibration set is 40 scores from 0.35 to 0.935 (RECENT_RESOLVED_CASE_SCORES). A routine candidate scores 0.10, so \(p = 1\); the outlier's 1.6 gets \(p = 1/41 \approx 0.024\).

Each desk names its density cut-off as a constant (CARD_DEVICE_RING_DENSITY_THRESHOLD = 0.08, REPORT_GRAPH_DENSITY_THRESHOLD = 0.20, ...). In production that number is the density at which the desk's decision has historically flipped; Choose thresholds and calibration shows how to read it from your logs.

What these checks do not verify

The checks ask whether the related cases are strong, stable and familiar enough to act on. They do not ask whether those cases point towards the proposed action. Direction comes from the event: the upstream risk model, triage rule or monitor decides flag versus auth, high_risk versus low_risk. On the card desk, an auth on a dense device graph is captured whether that graph's members were confirmed fraud or cleared. Keep the decision about which event to send in the system that has the score.

When a hold meets a deadline

Some desks cannot leave a decision open: a card authorisation must be answered in a second or two, a dispatch request in a minute, a cache write immediately. For those, a hold must come with a default answer that is safe without the automatic action, chosen in advance by the desk owner:

Desk Deadline Safe default while the person decides
live_card_auth seconds your existing fallback (for example step-up authentication), not a silent decline
live_rideshare_match about a minute the request goes to the human dispatch console
live_cdn_peak immediate serve from origin; do not write the cache
live_line_spc one machine cycle keep running and raise a maintenance ticket (safety interlocks stay in the PLC)
live_oncall_night minutes post to the incident channel; escalate if nobody acknowledges

Then put a time limit on every hold and an escalation when it expires. Operate and monitor covers hold queues, time limits and alerts on the hold rate.

The desks

live_card_auth: card authorisations

The situation. An issuer's authorisation worker. The entity is a card. The risk model upstream labels each authorisation auth or flag. Policy: (test(auth) ; act(capture) | test(flag) ; act(freeze))*. Today's rule: capture every auth, freeze every flag, whatever the amount or the evidence.

uv run grounded-decision examples live_card_auth

What to notice in the output. Six authorisations, four captured or frozen, two held.

  • card_4411 buys $12 of groceries on a dense device graph (dλ² = 9.83): captured. The same card's $840 electronics flag rests on a graph that is still informative (dλ² = 6.27): frozen.
  • card_8802 gets the same $840 flag but its device graph is a rumour graph (dλ² = 0.5, density 0.045 inside the window around 0.08): the freeze is held for fraud operations. So is card_3300's $1,200 travel flag.
  • card_4411 comes back for coffee. Its saved state is restored from the freeze and the purchase is captured.

The takeaway. The same flag on two cards gets two answers, and the difference is the evidence around each card, not the amount. In production, authorisations arrive on a topic keyed by card id, the device-graph snapshot is cached per card (refreshed every 15 minutes) because of the response deadline, and a held freeze goes to the fraud-ops queue with proposed_action=freeze while the authorisation gets your fallback answer. The density cut-off would come from the device-graph density at which past fraud reviews switched from mostly cleared to mostly confirmed. The full single-decision version is Fraud review.

live_ed_shift: emergency department

The situation. Decision support for an emergency department. The entity is a patient; each is new, so every decision starts from the policy's first state. Policy: (test(low_risk) ; act(discharge) | test(high_risk) ; act(escalate))*. Today's rule: discharge if heart rate is under 100, else escalate. The recommendation becomes the bed-board default and a clinician confirms it.

uv run grounded-decision examples live_ed_shift

What to notice in the output. Six patients, four recommendations posted, two held.

  • The ankle sprain (heart rate 72) and the chest pain (heart rate 128, SpO₂ 90%) each have a dense cluster of similar presentations tonight (dλ² = 8.19 and 5.90): discharge and escalate are posted.
  • The first rash of the shift (patient_103, heart rate 88) and a thinly corroborated shortness of breath (patient_104, heart rate 118) have no cluster to compare with (dλ² = 0.5): both are held for the charge clinician. Note that the hold applies to escalation as well as discharge.
  • The second ankle sprain and an upper respiratory infection have clusters and post.

The takeaway. A presentation the department has not seen tonight goes to a person, whichever way the triage rule leans. In production, events come from the EHR registration feed (HL7 ADT through the interface engine, or a FHIR subscription) once triage vitals are filed; the syndrome graph is rebuilt every 15 minutes from six hours of encounters; a hold puts the patient on the clinician review list with the proposal and the failed checks. The cut-off would come from the cluster density at which past dispositions for a complaint stopped being consistent. See also Clinical triage.

live_trust_queue: take-down queue

The situation. One reported post, followed through a night. Policy: (test(report) ; act(take_down))*. Today's rule: take the post down whenever reports exceed yesterday's cut-off. Density cut-off: REPORT_GRAPH_DENSITY_THRESHOLD = 0.20.

uv run grounded-decision examples live_trust_queue

What to notice in the output. Five report batches, two take-downs, three holds.

  • Tick 0: a clear pile-on, density 0.44, far from 0.20: take down.
  • Ticks 1 and 2: the reporters agree strongly (dλ² = 9.34 and 8.31) but the density, 0.21, sits on the cut-off: held. Strong agreement does not rescue a knife-edge.
  • Tick 3: density back at 0.42: take down.
  • Tick 4: a two-account brigade on a rumour graph: held.

The takeaway. Today's rule fires on every tick, including the two on the cut-off; that is the take-down-then-restore flicker moderators already know. A hold leaves visibility unchanged and sends the post to the moderator queue. In production, reports go to a queue partitioned by post id and a streaming job keeps the last hour of reports per post. The cut-off would be the reporter-graph density above which past moderator rulings were consistently "uphold". See also Content moderation.

live_loan_desk: loan origination

The situation. Each application is two events: apply, then the scorecard's eligible or ineligible. Policy: test(apply) ; (test(eligible) ; act(approve) | test(ineligible) ; act(deny)). Today's rule: approve every eligible scorecard result.

uv run grounded-decision examples live_loan_desk

What to notice in the output. Three applications: one approved, two held, and three await_next_event rows.

  • apply never proposes an action, so it is recorded as await_next_event. That is not a hold; nobody needs to look at it. The saved state is what lets the eligible event continue the same application.
  • Applicant A (FICO 760, dense identity graph, typical file): approve.
  • Applicant B (FICO 610, dense graph) has a document score of 1.6, above every calibration case (p = 0.024 against 0.10): held for manual underwriting.
  • Applicant C (FICO 740) has an identity resting on three isolated links (dλ² = 0.5): held.

Beside the module, a release gate compares last week's repayment behaviour (days to first missed payment) with the book the scorecard was validated on. It allows a Wasserstein-2 distance (roughly, how many days the whole distribution has to move to match) of at most \(50 \times 0.5^3 = 6.25\) days. The unchanged week moved 0.25 days and passes, so A is in released_to_funding; the regime-shift week moved 58.1 days and fails, so released_if_regime_shifted is empty. The service ANDs this gate with the verdict; it is not a sixth check. The gate measures any movement, including a uniform shift in a direction you would welcome; Compare outcomes and features explains what it does and does not tell you.

The takeaway. Multi-step policies need their state saved between events, and "nothing to do yet" must be kept separate from "a person must decide". The cut-off would come from the identity-graph density at which past applications switched between mostly repaid and mostly charged off. See also Credit underwriting.

live_rideshare_match: dispatch

The situation. Each ride request is a new entity. Policy: (test(request) ; act(dispatch))*. Today's rule: dispatch the nearest driver.

uv run grounded-decision examples live_rideshare_match

What to notice in the output. Four requests, two dispatched, two held.

  • Downtown requests sit in a dense match graph where trips have been completing all evening (dλ² = 6.24, then 2.80): dispatch.
  • Airport and suburb requests rest on three isolated driver-rider pairs: held for the human dispatch console.

Beside the module, three planning calls go through the hardness router and are logged next to the decision, never folded into it. The six zones form a ring.

  • A cheap one-round graph fingerprint of the zone map is computed (zone_map_fingerprint_ok=True).
  • A five-round fingerprint is refused (high_order_fingerprint_refused=True). This is a cost budget: the router caps the depth at 2 because the work grows steeply with depth. It is not a claim that the problem is unsolvable.
  • Standby zones are chosen with the router's general vertex-cover approximation, which returns all 6 zones (standby_zone_count=6). A ring of six zones is bipartite, so the true minimum, 3 alternate zones, can be found exactly in polynomial time; the general method only promises at most twice the minimum. See Use the hardness router and Matching marketplace.

The takeaway. Automatic dispatch where recent matches corroborate each other; a person where supply is thin. The cut-off would come from the match-graph density below which past automatic matches in a zone were often cancelled.

live_oncall_night: SRE on-call

The situation. The checkout service through one night, with one saved state carried across every event. The policy is the Incident response state machine. Today's rule: page on every still_bad, mitigate on every alert.

uv run grounded-decision examples live_oncall_night

stateDiagram-v2
    [*] --> mitigating: 02:00 alert (mitigate)
    mitigating --> paging: 02:04 still_bad (page HELD)
    paging --> mitigating: 02:05 ack
    mitigating --> paging: 02:11 still_bad (page)
    paging --> mitigating: 02:13 ack
    mitigating --> resolved: 02:40 clear (close)

What to notice in the output. Six events, five actions, one hold.

  • 02:00 an error-budget burn corroborated across regions (dλ² = 9.61): mitigate.
  • 02:04 one flapping probe on a rumour graph: the page is held, but the state still moves to paging, because the event happened.
  • 02:05 ack returns it to mitigating. 02:11 still_bad on a corroborated graph (dλ² = 2.78): page. 02:13 ack. 02:40 clear: close.

The takeaway. Save the state on a hold. If the 02:04 state were thrown away, the next event would be read against the wrong step of the incident. The flip side is that the saved state can say paging when nobody was paged; the person handling the hold needs to see that, and Incident response explains what it means operationally. In production, alerting and paging-tool webhooks feed the worker, state lives on the incident row, and a hold posts the proposal and failed checks to the incident channel. The cut-off would come from the probe-correlation density at which past alerts switched from mostly false to mostly real.

live_line_spc: factory line

The situation. Quality stops on line_B. Policy: (test(fault) ; act(stop_line) | test(ok) ; act(keep_running))*. Today's rule: stop the line on any fault bit.

uv run grounded-decision examples live_line_spc

What to notice in the output. Five readings, three actions, two holds.

  • Top of the hour, all stations agree: keep running.
  • A station-3 fault bit with nothing corroborating it (dλ² = 0.5): the stop is held, a maintenance ticket is raised, the line stays up.
  • Stations 3 to 8 agree on a fault (dλ² = 6.48): stop.
  • After the restart, ok. Another lone probe: held.

The takeaway. A lone sensor does not stop the line on its own, but it does create a ticket. This decides quality stops only: hard safety interlocks stay in the PLC and never go through this module. In production the worker runs at the edge, fed over OPC UA or MQTT, and recomputes the sensor co-movement graph every cycle. The cut-off would come from the co-movement density at which past alarms were confirmed as real faults. See also Manufacturing SPC.

live_experiment_ship: experiment ship gate

The situation. A checkout copy experiment. Policy: (test(observe) ; act(log_batch) | test(ship) ; act(ship_variant))*. Today's rule: ship when the observed lift exceeds 2 percentage points. Density cut-off: SEGMENT_AGREEMENT_DENSITY_THRESHOLD = 0.20.

uv run grounded-decision examples live_experiment_ship

What to notice in the output. Four hourly batches and a ship request.

  • The 14:00 batch's segment graph has density 0.21, on the 0.20 cut-off: committing it is held for an analyst.
  • At the end of the day: 50 of 200 control users converted against 70 of 200 in the variant, a 10-point lift. The service draws 80 random tables with the same totals (the hypergeometric null behind Fisher's exact test) and counts how many show a lift at least as large; the Monte Carlo p-value is \(3/81 \approx 0.037\). With only 80 draws the smallest possible p-value is \(1/81\), so use thousands in practice.
  • The module acts on the ship request, so variant_shipped=True.

The takeaway. The ship needs both halves: the lift must be significant and the segments must agree. A significant table on a rumour segment graph would not ship, and a corroborated graph with a non-significant table would not either. See also A/B test tables and Combinations.

live_cdn_peak: edge cache during a spike

The situation. During a traffic spike, each object reaching the edge has an admission value (expected cache hits per unit of space). Each object is a new entity. Policy: (test(admit) ; act(cache) | test(skip) ; act(pass_through))*. Today's rule: cache every object whose value is at least yesterday's sample value, 4.0.

uv run grounded-decision examples live_cdn_peak

What to notice in the output. Six objects with values [1, 1.5, 9, 2, 12, 3].

  • A selection rule beside the module (the one-sample prophet rule) takes the first value above the sample: object 2 (value 9) is cached. Objects below the sample pass through.
  • Object 4 (value 12) is also above the sample, so today's rule would cache it. Its demand rests on three isolated pairs of client networks, a likely crawler: the cache write is held and the object is served from origin.

The takeaway. A high value is not enough when the demand behind it is not corroborated. About the selection rule: it uses one number (yesterday's value) as the bar, single_item_one_sample, so no "at least half the best value" factor holds and the result reports factor_claimed=None. That guarantee needs one past sample per arrival and the bar at their largest (single_item_max_of_samples; see Cache admission). The cut-off would come from the co-request density that separated human from automated demand in past spikes.

Adding a desk

  1. Name the entity key, the events, the side effect of each action in your production language, and the always-fire rule you are replacing.
  2. Write the policy so that each event that should do something marks an action, and each action follows its own event (Write a policy).
  3. Define members, links and labels for your evidence, and a named density cut-off with a comment saying where its value came from.
  4. Build a list of IncomingEvent with evidence from the livefeed.py builders (or your own CorroborationGraph), and run it through LiveDesk.process_stream.
  5. Give the runner a docstring with the same sections as the others: business problem, why always-fire fails, evidence graph, in production.
  6. Register it in RUNNERS in examples/live.py and name the id in backticks on this page and on the Example library.

Reading the JSON output

--json prints one object per desk. summary.ledger has one row per event: the side effect, the proposed action, the failed checks, the state before and after, and the full evidence. summary.side_effects is the shift in one line, and summary.blocked_overconfident_acts counts the actions today's rule would have taken that the desk did not. records is the raw DecisionRecord.to_dict() list: the audit log. The text report prints, for each event, the input, the six checks, the output and the state to save. It uses some older labels: "fire" means act, "HOLD" means a hold, "cursor" means the saved state, "flicker band" means the knife-edge window, and "KS" is the 1.05 agreement line.