Manufacturing SPC: stop the line only when the fault is corroborated

A production line with dozens of sensors raises statistical process control (SPC) alarms several times a shift, and most of them are a single sensor glitching rather than the process drifting. Stopping on every alarm costs output and restart scrap; ignoring alarms lets a real drift run scrap for an hour. This page shows a line controller that stops the line when neighbouring stations confirm the alarm, and keeps running and opens an engineering ticket when they do not. It is written for process and quality engineers; the runnable example id is manufacturing_spc.

At a glance

Decision	On an out-of-control alarm: stop the line, or keep running and send the alarm to an engineer.
Owner	The line controller; the shift supervisor and process engineering.
Today's rule	Stop on any reading beyond three-sigma control limits.
What this adds	A check that nearby sensors confirm the alarm, and that the pattern looks like past confirmed faults.
Demo outcome	4 sample periods: 2 acted (one stop, one continue), 2 held.
Example id	manufacturing_spc (uv run grounded-decision examples manufacturing_spc)

Safety interlocks are out of scope

This module decides quality stops, not safety stops. Emergency stops, guard interlocks and any trip required for people or equipment safety must never be routed through it. A held alarm here means "a person decides", which is too slow for safety functions.

The decision today

Every sensor has a centre line and control limits; a reading more than three standard deviations (3σ) from the centre line raises an "out of control" alarm. The controller stops the line on any alarm and the shift supervisor restarts it.

Mistake	What it costs
Stop on a sensor glitch	An hour of output, a restart, start-up scrap, and operators who learn to override the alarm.
Keep running through a real drift	A run of scrap until someone notices, possibly a customer return.

Where today's rule goes wrong

With many sensors, three-sigma limits alone guarantee false alarms. In the demo, three alarms arrive and the naive rule stops on all three. Two are glitches:

• Thinly instrumented conveyor. On packaging line B the infeed vibration probe reads 4.8σ; the midspan and outfeed probes read 0.6σ and 0.3σ. Three sensors in a chain are too few witnesses, and they disagree.
• Lone probe on a dense line. On extruder A, the station-2 vibration probe reads 5.1σ while all 14 other sensors are within limits. There are plenty of witnesses, and they overwhelmingly contradict the alarm.

The real fault is a barrel-heater drift on extruder A: all six sensors on stations 4 and 5 read 4.2σ together.

What the system checks, in your terms

The evidence is the line's sensor adjacency graph for one sample period: every sensor that reported, linked to sensors on the same or an adjacent station (a physical fault at one station should show on both), and each marked in or out of limits.

Check	Question in plant terms	Data it reads	Fails when	What a hold means
Neighbour agreement	Do neighbouring sensors tell a consistent story?	In/out of limits per sensor; station adjacency	Too few sensors, or neighbours split	Not enough consistent witnesses to confirm a fault.
Knife-edge	Are there enough sensors for the pattern to be read reliably?	Number of sensors and links	Small sensor counts (here: 3 sensors)	The line is too thinly instrumented for this alarm to be confirmed.
Unusual cases	Does the share of sensors contradicting the alarm look like past confirmed faults?	Share of in-limit sensors on an alarm; 30 confirmed stoppages	Far more contradicting sensors than any confirmed fault had	No real fault has ever looked like this.
Policy shape and progress	Is the stop/continue policy well formed?	The policy, once at build	Never here	— (build-time checks; the progress check is weak by construction, see progress).

What these checks do not verify

• Neighbour agreement is direction-blind. On the lone-probe alarm, the 14 in-limit sensors agree with each other strongly, so that check passes. It asks whether neighbours predict each other, not whether they support stopping.
• Direction comes from the unusual-cases check, because the candidate score is the share of sensors contradicting the event (in-limit sensors on an alarm, out-of-limit sensors on an all-clear). That is what holds the lone probe, and what would hold an in_control event sent while stations 4–5 are out of limits. If the calibration set is empty, that check passes by default and the direction protection disappears; set Limits(require_conformal_calibration=True) in production.
• The knife-edge check reacts only to sensor count. On extruder A the link density is fixed by the sensor layout (0.486 on every row), so this check cannot respond to the readings; it holds only on small, thinly instrumented lines.

Data you provide

Input	Source	Refresh	Minimum / quality bar
(sensor_id, station, deviation_in_sigmas) rows	Historian or MES query for the sample period	Every sample period	Every sensor that reported; station numbers that reflect physical adjacency.
Event name (out_of_control / in_control)	Your existing SPC rule set	Every sample period	—
Confirmed-fault contradicting shares	Maintenance log: stoppages later confirmed as real faults	Monthly	At least 9 at α = 0.10; the demo uses 30 (0.30 to 0.88). Confirmation may take days; never include the current alarm.
Confirmed-good-run contradicting shares	Quality inspection of good runs	Monthly	At least 9; the demo uses 30 (0.000 to 0.145).

Worked example

Sample period	What happened	What the checks found	Verdict	What the line does
Heater drift, extruder A	6 of 15 sensors (stations 4–5) out of limits	Neighbours consistent; contradicting share (0.60) typical of real faults	Act stop_line	Stops.
Conveyor probe, packaging B	1 of 3 sensors out	Too few, disagreeing witnesses; too few sensors to read the pattern	Hold, proposed stop_line	Keeps running; ticket to process engineering.
Lone probe, extruder A	1 of 15 sensors out	Neighbours consistent, but 14 of 15 contradict the alarm — more than any confirmed fault	Hold, proposed stop_line	Keeps running; ticket names the probe.
All in limits, extruder A	0 of 15 out	Everything consistent	Act continue	Keeps running.

"Hold" is the record verdict abstain: the proposal stays on the record and is not carried out.

The numbers (from uv run grounded-decision examples manufacturing_spc)

Extruder A has 15 sensors, three per station over five stations, 51 links, average links per sensor \(d = 6.8\), link density \(\hat p = 0.486\) on every row. Cut-off \(p^{*} = 0.20\).

Row	\(n\)	\(\lambda\)	Strength \(d\lambda^2\) (> 1.05)	\(\hat p\)	Knife-edge	Contradicting share	Unusualness \(p\) (> 0.10)
Heater drift	15	0.647	2.85 pass	0.486	outside, pass	0.60	0.516 pass
Conveyor	3	0.0	0.0 fail	0.667	inside, fail	0.667	0.387 pass
Lone probe	15	0.686	3.20 pass	0.486	outside, pass	0.933	0.032 fail
All in limits	15	1.0	6.8 pass	0.486	outside, pass	0.0	1.0 pass

\(\lambda\) is how much more often linked sensors share a label than chance. At \(n = 15\) the knife-edge half-width is 0.219 and \(|0.486 - 0.20| = 0.286\) is outside it. At \(n = 3\) the half-width is capped at 0.5 and \(|0.667 - 0.20| = 0.467\) is inside. The lone probe's 0.933 exceeds all 30 confirmed-fault shares, so \(p = 1/31\).

--json summary: naive_acts: 4, grounded_acts: 2, grounded_abstains: 2, blocked_overconfident_acts: 2.

What happens next

• Act stop_line. Send the stop to the PLC; the supervisor restarts after the fault is fixed, as today.
• Hold. Do not stop. Open a ticket for process engineering with the record attached: the proposed action, the alarming sensor (from the readings and event payload), and which check held. Give the ticket a response time short enough to bound scrap if the hold was wrong — for example within the next inspection interval. A lone probe is usually a loose mount or failing sensor, but it can also be a real local fault (a bearing, a loose fixture) that only one sensor sees. If the same probe keeps alarming, or a second sensor at that station follows, treat it as real.
• Feedback. When the engineer confirms a held alarm was a real fault, add its contradicting share to the confirmed-fault calibration set at the next refresh; this teaches the module that such patterns happen.

Tuning and expected workload

• CONTROL_LIMIT_SIGMAS = 3.0 stays your SPC rule.
• FAULT_PATTERN_DENSITY_THRESHOLD = 0.20 was chosen from the historian as the density where real faults and coincidental multi-sensor alarms were hardest to tell apart. On a fixed layout it mostly decides how small a line must be before alarms there always go to a person.
• α = 0.10 (unusualness) decides how far outside past confirmed faults an alarm may be. Lower it to hold fewer lone-sensor alarms.
• Expected tickets: roughly the alarms today's operators already override, plus all alarms on thinly instrumented lines. Measure it in shadow mode first; see estimate impact and choose parameters.

Limits, risks and compliance

• Safety is out of scope (see the box at the top).
• Compare with run rules. Western Electric and Nelson rules reduce false alarms by looking at a single sensor's recent history (two of three beyond 2σ, eight on one side of the centre line). This module looks across sensors at one moment instead. They are complementary: you can use run rules to name the event, then this module to decide whether to stop.
• Synthetic data. Readings and calibration shares are invented.

Wiring it in

1. At controller start, build the module once: spc_module = build().
2. Monthly, refresh both calibration tuples from the maintenance log (confirmed stoppages) and quality inspection (confirmed good runs).
3. Each sample period, pull (sensor_id, station, deviation) rows for the line, name the event from your SPC rule set, build the evidence, and decide:

from grounded_decision.examples.manufacturing_spc import build, line_sensor_evidence

event = Event("out_of_control", {"line": line_id})
evidence = line_sensor_evidence(readings, event_name=event.name)
response = handle_request(
    spc_module,
    {
        "state": None,  # each sample period is independent
        "event": event.to_dict(),
        "evidence": evidence.to_dict(),
    },
)
append_to_audit_log(response)
if response["verdict"] == "act":
    send_to_plc(line_id, response["action"])  # stop_line / continue
else:
    open_engineering_ticket(line_id, response["proposed_action"], response["reason"])

append_to_audit_log, send_to_plc and open_engineering_ticket are your own functions. If your line has a latched stop state, persist response["extras"]["state"] per line instead of passing None.

The policy is the library's two-path helper: out_of_control proposes stop_line, in_control proposes continue.

Run it yourself

uv run grounded-decision examples manufacturing_spc

CLI label	Meaning here
fire, ACT	act: the proposed action is carried out
HOLD, hold_for_human, ABSTAIN	hold: nothing is carried out; record verdict abstain
strength dλ², "KS"	evidence strength and its 1.05 line
"flicker band", p*, FLICKER	the knife-edge window and its cut-off
bag	the calibration set
cursor, extras['state']	the saved state (not used here: periods are independent)

The naive-versus-grounded counts are in the summary field of --json output.

Further reading

• Act, abstain, degrade
• The six checks
• Troubleshoot: why did it abstain?, Operate and monitor
• Theory: recovering a label from related cases, knife-edge densities, coverage for unfamiliar cases
• Glossary