Skip to content

Limits and performance

Before you put a decision library on a request path you need three answers: how long a decision takes, how large a policy can grow, and what happens when something is too large. This page gives measured costs, explains what they grow with, lists every size cap and setting in Limits, and shows how to plan capacity and tune the caps.

It is for the engineer sizing a service and the reviewer approving it. It assumes you know the split between building a module once and deciding per request (Architecture). Default values are also listed on Defaults and limits.

Measured costs

Measured on an Apple M5 Max laptop, Python 3.12.13, best of five runs with timeit. Your numbers will differ; the ratios are what matter.

Operation Time Notes
Build the fraud_review module 8.9 µs Hand-built chart, 4 states; includes the policy checks
One decide on fraud_review 7.0 µs Includes reset(); 40 calibration scores
Build the evidence from 40 login rows 9.6 µs device_graph_evidence, the example's evidence builder
Parse and build a two-alternative text policy 69 µs The loan policy from State and policy versions

Reproduce the first two rows with:

import timeit
from grounded_decision import Event
from grounded_decision.examples import fraud_review as fr

logins, outcomes = fr.dense_fraud_ring_logins()
evidence = fr.device_graph_evidence(logins, outcomes, candidate_model_score=0.10)
module = fr.build()


def one_decision():
    module.reset()
    module.decide(Event("suspicious"), evidence)


for label, fn, n in [("build", fr.build, 2_000), ("decide", one_decision, 20_000)]:
    best = min(timeit.repeat(fn, number=n, repeat=5)) / n
    print(f"{label}: {best * 1e6:.1f} µs")

In a real service, decide is the cheapest step. Reading the related cases from your database and writing the record dominate.

How costs grow with policy size

A policy written as text with \(k\) alternatives, test(e0) ; act(a0) | test(e1) ; act(a1) | ..., measured the same way:

Alternatives \(k\) States Arrows Build One decide
5 6 5 0.14 ms 7.0 µs
20 21 20 0.77 ms 7.3 µs
50 51 50 3.13 ms 7.8 µs
100 101 100 10.4 ms 8.8 µs
128 refused CompileError: NFA has 514 states > 512

A compiled text policy keeps only the arrows the policy allows: an event the policy does not expect in the current state has no arrow, and the entity stays where it is. decide scans the arrow list to find the next state, so decide time grows slowly with policy size even though the checks themselves do not. Hand-built charts contain only the arrows you draw.

What each step costs

Step When Grows with Cap
Check how deeply the policy nests repetition build size of the policy text max_star_height
Compile text to an automaton (Thompson construction) build size of the policy text max_nfa_states
Make it deterministic (subset construction) build up to \(2^{m}\) for \(m\) automaton states in the worst case max_dfa_states
Build the progress game build states + arrows max_game_vertices
Solve it: parity (Zielonka's algorithm) build recursion over priorities max_game_vertices, max_parity_depth
Solve it: average reward (Karp's algorithm) build \(n \cdot (n + m)\) for \(n\) states and \(m\) arrows max_game_vertices, max_karp_vertices
Move to the next state per decision number of arrows none beyond the build caps
Neighbour agreement and knife-edge checks per decision constant none
Unusual-cases check per decision number of calibration scores none: you choose how many to pass

So one decision costs time proportional to the number of arrows plus the number of calibration scores. Nothing on the per-decision path solves a game or parses text. A calibration set of ten million scores is a bug in your evidence builder, not in decide; cap it where you build the evidence.

The Limits object

Limits is a frozen dataclass. Every module holds one (by default DEFAULT_LIMITS = Limits()); you change settings by building a new instance and passing it in. Every size cap is enforced; the two timing budgets are targets checked by a test, not timers.

Field Default Read by When exceeded or set
max_star_height 3 build StarHeightError: star-height 4 exceeds max 3 (catalog 134); no module is built. See Is the policy well-formed?
max_nfa_states 512 build CompileError: NFA has 514 states > 512
max_dfa_states 256 build CompileError: DFA exceeded 256 states
max_game_vertices 200 build The module is built, but the progress check is recorded as failed, so every decision abstains, with the reason … vertices exceed limit …
max_parity_depth 64 build Same as above, with parity recursion exceeded depth limit
reconstruction_margin 0.05 per decision Neighbour agreement passes when \(d\lambda^2 > 1 + \text{margin}\)
conformal_alpha 0.10 per decision Unusual-cases check passes when the p-value is above this
require_conformal_calibration False per decision When True, an empty calibration set fails the unusual-cases check (no calibration scores (require_conformal_calibration)) instead of passing
operational_fk_prefactor 2.0 per decision Scales the knife-edge window; larger means more holds. See Are we on a knife-edge?
friedgut_kalai_prefactor \(2^{19}\) = 524,288 per decision Only changes the reported theorem width (threshold_width_theorem), never the verdict
max_wl_k 2 the hardness router Refuses graph-fingerprint requests above this cost; not used by decide. See Questions the library refuses
max_karp_vertices 200 build The average-reward solver raises GameTooLarge; the progress check fails closed with … vertices exceed the mean-payoff limit 200, so every decision abstains. With the defaults, max_game_vertices (also 200) is reached first
max_table_cells_exact 64 sample_two_row(..., limits=) ValueError: N cells exceed the exact-sampling limit 64; use sample_switch
max_switch_steps 10,000 sample_switch(..., limits=) ValueError: N switch steps exceed the limit 10000
min_support 0.5 per decision Support check passes when the share of related cases carrying the action's implied outcome is at least this. See Do the related cases support this action?
require_support False per decision When True, evidence without a support share fails the support check (support not measured (require_support)) instead of passing
decide_ms_budget 50 tests/test_performance_budget.py A target, not a timer: the test builds fraud_review and requires the median of 200 decide calls to be under it
compile_ms_budget 200 tests/test_performance_budget.py A target, not a timer: the same test requires the median of 20 module builds to be under it

With the default caps, a text policy made of alternatives runs into max_nfa_states at 128 alternatives (127 alternatives build, with 128 states), before it reaches max_game_vertices. A hand-built chart has no automaton step, so for it max_game_vertices is the cap that applies.

No clock anywhere

Neither build nor decide measures time. A slow decide returns late rather than raising. The budgets are checked on the library itself in CI (tests/test_performance_budget.py), not on your service. If you have a latency budget, enforce it in your service (a request timeout, or abstaining and routing to a person when evidence collection runs long).

Planning capacity

Work from the measured numbers above and your own evidence path:

  • CPU. At about 7 µs per decision, one core can in principle decide more than 100,000 times a second. If you guard a shared module with a lock (see State and policy versions), that lock is held only for those microseconds; your database and evidence queries are the real limit.
  • Build time. Build once per policy version at start-up, never per request. Even a 100-alternative policy builds in under 50 ms.
  • Storage. The fraud_review record is about 3.5 KB as single-line JSON (1.3 KB of it is the catalog pointers repeated on every record), and its evidence snapshot with 40 calibration scores is about 0.6 KB. One million decisions a day is roughly 4 GB a day before compression. See The decision record for what to keep.
  • Several modules. Pipeline.decide runs its modules one after another, so its cost is the sum. See Composing modules.

Tuning

Limits is frozen, so DEFAULT_LIMITS.conformal_alpha = 0.05 raises an error. Build a new instance per module or per tenant:

from grounded_decision import Limits, Module, parse

strict = Limits(
    max_dfa_states=32, max_game_vertices=40, require_conformal_calibration=True
)
module = Module.from_expr("tenant_a", parse("test(a) ; act(b)"), limits=strict)

The module passes its Limits to every step it runs: the build, the four evidence checks, and the compile of any not(...) inside the policy. Pass the same instance to dispatch(subgoal, limits=...) if you use the hardness router for that tenant.

You want to Change Effect
Reject oversized or hostile policy text Lower max_star_height, max_nfa_states, max_dfa_states Bad policies fail at build with an error you can show the author
Bound the build-time analysis for large charts Lower max_game_vertices Large charts still build, but every decision abstains
Never act without calibration require_conformal_calibration=True Empty calibration sets fail the unusual-cases check
Hold more often on unfamiliar cases Raise conformal_alpha (for example 0.2) More cases count as unusual
Demand stronger agreement among related cases Raise reconstruction_margin Fewer cases pass neighbour agreement
Widen or narrow the knife-edge window Raise or lower operational_fk_prefactor More or fewer holds near the cut-off
Require the related cases to point the same way more clearly Raise min_support Fewer actions pass the support check
Never act without a support share require_support=True Evidence without support fails the support check

The evidence settings (margin, window prefactor, alpha, support minimum) are best learned from labelled history with a certified error rate (Learn parameters from your history); what each one does is in Choose thresholds and calibration. In the reference service, changing any field changes the settings fingerprint (parameters_fingerprint) stamped on every decision, but not the policy fingerprint, so saved states stay valid (State and policy versions).

Where to go next