Skip to content

Are we on a knife-edge? (knife-edge window)

Many policies use a cut-off on how densely the related cases are linked: "a team counts as cohesive if at least 20% of possible working relationships exist", "a cluster of accounts counts as a ring above 8%". When the measured density sits right on that cut-off, one extra or one missing link flips which side you are on, and the decision with it. This check holds the decision when the density is too close to your cut-off to trust, given how many cases the density was measured on.

This page is for domain owners who set the cut-off and data scientists who supply the graph. It assumes you know that each decision comes with evidence describing the related cases as a graph (see The six checks). The name of this check in decision records is threshold.

The question

Is the link density of the related cases far enough from our cut-off that a little noise could not flip the answer?

The link density (\(\hat p\), "p-hat") is the fraction of all possible links between the related cases that actually exist. Your cut-off (p_threshold, written \(p^\star\)) is the density your product treats as meaningful. Around the cut-off the library draws a knife-edge window: a band in which it does not trust which side the density falls on. The band is wide when there are few cases (a density measured on five accounts is very noisy) and narrows slowly as the number of cases grows.

The intuition: a dead band that shrinks with size

density 0.0       0.1       0.2       0.3       0.4       0.5
         |---------|---------|---------|---------|---------|
                   [=========|=========]   50 cases: 0.20 ± 0.105
                             ^ cut-off 0.20
                              ^ Erin 0.21: inside, hold
                                                      ^ Dan 0.45: outside, act
         <===================|=============================]   10 cases: 0.20 ± 0.30

The = marks the window: inside it, the check holds. With 10 cases the window runs from below 0 up to 0.50.

The window's half-width depends only on the number of related cases \(n\). It shrinks like \(1/(\ln n)^2\), which is slow: going from 50 to 1,000 cases only cuts it from 0.105 to 0.034.

Worked example

uv run grounded-decision examples step_threshold_window

Two access requests from 50-person teams with strong neighbour agreement. The cut-off for "cohesive team" is 0.20. Dan's team has link density 0.45; Erin's has 0.21:

                   density p̂=0.45  flicker band p*=0.2  stable (outside window)
         → ACT  because all six channels passed and action 'grant_access' is marked on this grammar state

                   density p̂=0.21  flicker band p*=0.2  FLICKERING (inside window — do not trust)
         threshold        FAIL  is density outside the operational flicker window?
                            evidence density inside operational window (theorem width=5.51e+04, vacuous=True)

With 50 cases the half-width is 0.105, so the window is 0.095 to 0.305. Dan's 0.45 is outside it (act); Erin's 0.21 is 0.01 from the cut-off, inside it (hold). The summary also prints a "theorem width" of about 55,000 (5.51e+04), which is explained under "What it does not tell you".

How wide is the window for my graph size?

Half-widths with the defaults, computed with operational_width(n, 0.1) from grounded_decision.evidence.thresholds:

Related cases \(n\) Window width Half-width (distance from cut-off that still holds)
3 to 6 1.00 or more 0.50 (the cap)
7 0.850 0.425
8 0.744 0.372
10 0.607 0.304
16 0.419 0.209
20 0.359 0.179
40 0.237 0.118
50 0.210 0.105
80 0.168 0.084
200 0.115 0.057
1,000 0.068 0.034
10,000 0.038 0.019

What this means in practice:

  • Graphs of 6 or fewer cases (and any graph under 3) pass only if the density is more than 0.5 from the cut-off. With a cut-off of 0.08, that means a density above 0.58. Tiny graphs effectively always fail. The held discharge in clinical_triage is one: 5 related patients, density 0.40 against a cut-off of 0.15, 0.25 apart, inside the 0.5 half-width.
  • Graphs of about 10 cases need the density about 0.3 away from the cut-off.
  • Graphs of a few hundred cases still hold anything within about 0.05 of the cut-off.

The exact rule

With \(n\) = n, \(\hat p\) = p_hat and \(p^\star\) = p_threshold from the evidence, \(c\) = Limits.operational_fk_prefactor (default 2.0) and \(\varepsilon = 0.1\) (fixed in the decision path):

\[ w = \frac{c\,\ln\!\bigl(1/(2\varepsilon)\bigr)}{(\ln n)^2} = \frac{2 \ln 5}{(\ln n)^2}, \qquad \text{hold} \iff \bigl|\hat p - p^\star\bigr| \le \tfrac12 \min(w,\,1). \]

\(w\) is the window width; for \(n < 3\) it is set to 1. A density exactly on the edge of the window holds. When you build evidence with snapshot_from_graph, \(\hat p = E / \bigl(n(n-1)/2\bigr)\) with \(E\) the number of links.

The record's threshold entry (also summarised in health):

Field Meaning
metrics.n, p_hat, p_threshold the inputs
metrics.epsilon \(\varepsilon\), 0.1
metrics.operational_width \(w\), the width the decision uses
metrics.in_operational_window true means held by this check
metrics.theorem_width the same formula with prefactor \(2^{19}\) (see below)
metrics.theorem_vacuous true when that width is 1 or more, which is always in practice
metrics.in_theorem_window always false when the theorem width is vacuous
trust operational in practice (see below)

What it does not tell you

  • It does not say which side of the cut-off you are on. Density 0.10 and density 0.30 against a cut-off of 0.20 are treated the same: both are 0.10 away. Whether being above or below the cut-off should lead to this action is your policy's job, expressed in the event you send. Only the support check looks at the direction of the evidence; see The six checks.
  • The window is a practical dead band, not the theorem. The shape \(\ln(1/(2\varepsilon))/(\ln n)^2\) comes from a mathematical result (Friedgut–Kalai sharp thresholds) about how quickly a graph property switches from rare to common as density rises, around that property's own switching point. The library centres the window on your cut-off instead, and replaces the result's proven constant \(2^{19}\) with 2. With \(2^{19}\) the width exceeds 1 at every graph size you will meet (it drops below 1 only for \(n > e^{918}\)), so it would hold everything; that is why the record reports it as theorem_vacuous and the decision uses the prefactor-2 width, tagged operational.
  • The check does not choose the cut-off. It cannot tell you whether 0.20 is the right number; it only refuses to decide too close to it. The cut-off comes from your domain, or from your labelled history with grounded-decision learn --learn-decision-line.
  • It reads the same graph as neighbour agreement. A thin graph usually fails both. They are separate questions, not independent evidence.

What to do when it fails

The reason reads evidence density inside operational window (...). Look at metrics.n, p_hat and p_threshold:

  • Small n: the graph is too small to measure density reliably. Gather more related cases; going from 10 to 40 cases cuts the half-width from 0.30 to 0.12. For very small graphs this check will hold almost everything, which is usually right.
  • p_hat close to p_threshold on a large graph: the case really is borderline. Route it to a person, or use a different signal.
  • Many holds and you believe the window is too cautious: the window width (operational_fk_prefactor) and, optionally, the cut-off itself can be learned from labelled history, with a certified error rate (Learn parameters from your history; --learn-decision-line for the cut-off). The learner may also find that the window does not help on your data and switch it off (prefactor 0). By hand, lowering the prefactor narrows the window proportionally (1.0 halves it), but then nothing certifies the result; see Choose thresholds and calibration and Troubleshoot: why did it abstain?.

The adaptive_threshold_walk example lets a density drift through the window over time, if you want to watch the verdict switch:

uv run grounded-decision examples adaptive_threshold_walk

Where next