Credit underwriting: approve, deny, or send to an underwriter¶
A lender's scorecard estimates each applicant's chance of defaulting and compares it with a cut-off, and the decision engine approves or denies in seconds. Automating that call is what makes instant credit possible, but a wrong automatic approval is a loss the lender carries for years, and the scorecard is least reliable for applicants just under the cut-off. This page shows how the library approves or denies automatically only when the applicant looks like borrowers for whom that decision turned out right, and when their peer group's repayment record is consistent enough to lean on; everything else goes to a human underwriter. It is written for a credit-risk policy owner or model-risk manager; the maths is in collapsed boxes and linked pages.
At a glance
| Decision | Approve the loan (approve) or deny it (deny). |
| Owner | Credit-risk policy; manual underwriting handles every hold. |
| Today's rule | Predicted default probability below 12 %: approve. Otherwise: deny. |
| What this adds | Before approving, check that borrowers who were scored this risky in the past actually repaid. Before either action, check that the applicant's peer group (past borrowers in the same industry and metro area, linked by shared employer) has a consistent repayment record. Otherwise hold for an underwriter. |
| Demo outcome | 3 applications: 2 acted on, 1 held (an approval just under the cut-off, riskier than every borrower who repaid last quarter). |
| Example id | credit_underwriting (uv run grounded-decision examples credit_underwriting) |
Not a compliance tool
This is a worked example on synthetic data. Nothing on this page is legal advice. Fair-lending, adverse-action and model-risk obligations (see Limits, risks and compliance) remain yours, and the library's records do not satisfy them on their own.
The decision today¶
The scorecard gives each applicant a predicted default probability (PD): the chance, by its estimate, that the loan will not be repaid. Credit policy sets a cut-off at 0.12. Below it the engine approves; at or above it the engine denies. Both kinds of mistake cost money:
| Mistake | What it costs |
|---|---|
| Approve a borrower who defaults | The written-off balance and collection costs, carried for the life of the loan. |
| Deny a borrower who would have repaid | Lost interest income, a lost customer, and an adverse-action notice you must justify. |
| Hold for an underwriter | Underwriter time, and the applicant waits instead of getting an instant answer (see What happens next). |
Where today's rule goes wrong¶
Take a teacher with a FICO score of 610 and a PD of 0.11, just under the 0.12 cut-off. Today's rule approves.
Now look at the lender's own track record. Last quarter forty loans matured and were repaid. When those borrowers applied, the scorecard gave them PDs between 0.010 and 0.108. Not one borrower who actually repaid was scored as risky as this applicant. The cut-off says yes; the lender has never seen an approval at this risk level work out.
The scorecard is least reliable close to its own line, and that is where the lender has the least repayment history to check it against. Today's rule treats a PD of 0.11 exactly like a PD of 0.02.
What the system checks, in your terms¶
The library sees the scorecard's approve_signal or deny_signal, plus
two kinds of evidence: the lender's recent track record, and the
applicant's peer group (past loans in the same industry and metro area,
linked when two borrowers share an employer, because a layoff hits them
together). It runs six checks; all six must pass for the proposed action
to be carried out. Two of them check the policy itself and pass in every
row here (see the six checks). Three
of the four that read your data are below; the fourth, support, is not measured in this example and passes with support not measured:
| Check | Question, in underwriting terms | Reads | Fails when | What a hold means |
|---|---|---|---|---|
| Unusual cases | Does this applicant look like borrowers for whom this decision turned out right? For an approval: is the PD within the range of borrowers who repaid? For a denial: is it within the range of borrowers who defaulted? | The applicant's PD; PDs at origination of recently matured loans, split by outcome (the calibration set) | The applicant is riskier than almost every borrower who repaid (for an approval), or safer than almost every borrower who defaulted (for a denial) | "Our approvals have not been validated at this risk level." |
| Neighbour agreement | Do borrowers who share an employer tend to share an outcome, and are there enough such links that the peer group's record says something? | Shared-employer links and each peer loan's outcome (repaid or defaulted) | Few links, or linked borrowers' outcomes are close to a coin flip | "The peer group is too thin or too mixed to lean on." |
| Knife-edge | Is the peer group's employer concentration far enough from your "concentrated" cut-off that a few more loans would not change the reading? | Number of peer loans, number of shared-employer links, your concentration cut-off | Concentration is inside the uncertainty window around the cut-off. Small peer groups have wide windows. | "Too few peer loans to judge how concentrated the group is." |
Unlike most cases in this library, the unusual-cases check here is label-conditional: an approval is compared only with borrowers who repaid, a denial only with borrowers who defaulted. That makes it the one check in this case that is sensitive to direction. It asks "have we been right making this decision at this risk level?", not just "is this score unusual?".
What these checks do NOT verify: which way the peer group points
The neighbour-agreement and knife-edge checks measure whether linked borrowers agree with each other, not whether their outcomes support the proposed action. A peer group that mostly defaulted is "strong evidence" for an approval as much as for a denial.
Reproduced with the shipped credit_underwriting functions: an
applicant with PD 0.05 whose peers are the closed-plant group (15 of
18 defaulted) acts approve; an applicant with PD 0.30 whose
peers are the healthy healthcare group (1 of 20 defaulted) acts
deny. The direction comes from the scorecard. The peer graph only
says whether the neighbourhood is informative enough to act on at all.
The support check would hold
both if the evidence carried the share of peers whose outcome
matches the decision (repaid for approve, defaulted for deny);
this example does not measure it.
In practice: if you want a bad peer record to block approvals, put it
into the scorecard or into an upstream rule (for example, send
deny_signal when the peer default rate is above a level credit
policy chooses), and document that rule like any other credit
criterion.
Data you provide¶
| Input | Source system | Refresh | Owner | Minimum / quality bar |
|---|---|---|---|---|
The applicant's PD, FICO score and employer (LoanApplicant) |
Scorecard and application | Per application | Credit risk | PD on the same scale as the calibration set. FICO and employer are carried into the record for audit. |
Peer loans: past loans in the applicant's industry and metro area, with employer and outcome (PastLoan) |
Loan book | Per application (or a cache refreshed daily) | Credit data engineering | Only matured or charged-off loans have outcomes; loans still running are not evidence yet. Never include the applicant's own loan. |
| Calibration history: last quarter's matured loans with the PD the scorecard gave them at origination | Loan book joined to scorecard history | Whenever a cohort matures | Model risk | At least 9 loans of each outcome at the default 10 % level, or the check can never pass for that action. The demo has 40 repaid and 20 defaulted. |
Approval cut-off (APPROVAL_CUTOFF_DEFAULT_PROBABILITY, 0.12) |
Credit policy | When risk appetite changes | Chief credit officer | A policy decision, not a fitted number. |
Concentration cut-off (EMPLOYER_CONCENTRATION_THRESHOLD, 0.10) |
Your loan book | Quarterly | Credit risk + data science | The concentration at which past peer groups were hardest to call. See Choose thresholds and calibration. |
underwriting_evidence in credit_underwriting.py turns these into the
evidence summary. It links every pair of peer borrowers who share an
employer (shared_employer_links), picks the calibration set that matches
the scorecard's proposal (calibration_scores_for_proposal), and scores
the applicant on the same scale.
Worked example¶
These rows come from uv run grounded-decision examples credit_underwriting.
| Row | Applicant | Peer group | Unusual-cases check | Agreement check | Knife-edge check | Today's rule | This system |
|---|---|---|---|---|---|---|---|
| 1 | Nurse, FICO 760, PD 0.02 | Healthcare: 20 loans at 3 employers, 1 default | Pass: 36 of 40 repaid borrowers were scored at least as risky | Pass: borrowers at the same employer almost always share an outcome | Pass | approve | act approve |
| 2 | Teacher, FICO 610, PD 0.11 | Same healthcare group | Fail: no repaid borrower was scored this risky | Pass | Pass | approve | hold; proposed approve kept on the record |
| 3 | Machinist, FICO 480, PD 0.45 | Closed plant and its supplier: 18 loans, 15 defaults | Pass: within the range of borrowers who defaulted | Pass: enough links, outcomes mostly shared | Pass | deny | act deny |
Row 2 is the one that matters. The peer group is the same healthy group as
row 1, so only the track-record check holds it. Its reason reads
conformal p=0.024 vs α=0.100 (reject). Over the three rows, today's rule
acts three times; this system acts twice and holds once
(blocked_overconfident_acts: 1 in the --json summary).
The numbers behind each check
Unusual cases. For an approval the unusualness score is the PD; for a denial it is \(1 - \text{PD}\). The p-value is
and the check passes when \(p > 0.10\). For the teacher, no repaid borrower's PD (0.010 to 0.108) is at least 0.11, so \(p = 1/41 \approx 0.024\). For the machinist, every defaulted borrower's \(1 - \text{PD}\) (0.675 to 0.96) is at least 0.55, so \(p = 21/21 = 1\).
Neighbour agreement and knife-edge. \(d\) is the average number of links per peer loan and \(\lambda\) is how much more often linked borrowers share an outcome than chance (0 = chance, 1 = always). Evidence strength \(d\lambda^2\) must be above \(1 + 0.05 = 1.05\).
| Healthcare peers (rows 1, 2) | Closed-plant peers (row 3) | |
|---|---|---|
| peer loans \(n\) | 20 | 18 |
| shared-employer links | 66 | 81 |
| defaults among peers | 1 | 15 |
| links per loan \(d\) | 6.6 | 9.0 |
| agreement \(\lambda\) | 0.909 (63 of 66 links agree) | 0.383 (56 of 81 links agree) |
| evidence strength \(d\lambda^2\) | 5.455 (pass) | 1.318 (pass) |
| concentration \(\hat p\) (share of peer pairs at the same employer) | 0.347 | 0.529 |
| knife-edge half-width at this \(n\), around the 0.10 cut-off | 0.179 | 0.193 |
| distance from cut-off | 0.247 (outside: pass) | 0.429 (outside: pass) |
| calibration set | 40 repaid | 20 defaulted |
| candidate score | 0.02 (row 1), 0.11 (row 2) | 0.55 |
| unusualness p-value | 0.902 (row 1), 0.024 (row 2) | 1.0 |
The half-width is \(\ln(1/(2\varepsilon))/(\ln n)^2\) with \(\varepsilon = 0.1\), capped at 0.5. Full derivations: unusual cases, neighbour agreement, knife-edge.
What the track-record check does to your cut-off¶
With this calibration set, the unusual-cases check works like a second, stricter approval line inside the policy cut-off. Re-running the shipped functions with the healthcare peers and different PDs:
| PD | Unusualness p-value | Result |
|---|---|---|
| 0.02 | 0.902 | act approve |
| 0.09 | 0.220 | act approve |
| 0.0995 | 0.122 | act approve |
| 0.100 to 0.119 | 0.098 down to 0.024 | hold (proposed approve) |
| 0.12 | 0.333 | act deny |
| 0.45 | 1.0 | act deny |
So the system does not override your cut-off; it adds a band, here PD 0.10 to 0.12, where approvals go to a person because the lender has not yet seen approvals at that risk level repay. The band moves as the calibration set changes: if borrowers scored 0.11 start repaying, they enter the calibration set and the band narrows. On the denial side the check is nearly inert in this demo (any PD of 0.12 or more gives \(p \ge 0.33\)), because the scorecard's past defaulters were all scored 0.325 or below.
What happens next¶
flowchart TD
A[Application scored] --> D{All six checks pass?}
D -- yes, approve --> X[Issue approval]
D -- yes, deny --> Y[Deny with adverse-action notice]
D -- no --> H[Hold: record with proposed action and reason]
H --> Q[Manual underwriting queue]
Q --> R[Underwriter decides]
R --> L[Loan matures or defaults]
L --> N[Next peer groups and calibration sets]| Verdict | Who is told | What they see | Time limit | If nobody acts in time | Feedback |
|---|---|---|---|---|---|
Act approve |
Applicant; loan servicing | action=approve, the full decision record |
Immediate | n/a | When the loan matures or defaults, it joins the calibration history and later peer groups. |
Act deny |
Applicant | action=deny. The adverse-action notice must give the principal reasons from the scorecard (the applicant attributes that drove the PD), not check names. |
Immediate | n/a | Denied applicants produce no repayment outcome (see the limits below). |
| Hold | Manual underwriting queue | proposed_action, the failed check and its reason, the application payload |
Your service level for manual decisions, inside the regulatory deadline for notifying the applicant | Escalate to a senior underwriter. Do not let the engine approve or deny on the hold's behalf. | The underwriter's decision, and later the loan's outcome, enter the book like any other loan. |
| Degrade | Manual underwriting queue and model-risk team | As for a hold | As for a hold | As for a hold | A degrade means the evidence pipeline is broken; fix it before trusting further acts. |
What a hold means depends on which check failed:
- Unusual cases (row 2): "outside the risk range where our approvals have been validated". This is what manual underwriting exists for: ask for more documents, offer a smaller line, or price for the risk.
- Neighbour agreement or knife-edge: the peer group is thin, mixed, or too small to judge. The applicant may be perfectly good; the underwriter should rely on the file rather than on the peer record.
Tuning and expected workload¶
| Setting | Policy or data? | How to choose | Too tight | Too loose |
|---|---|---|---|---|
| Approval cut-off (0.12) | Policy | Risk appetite | Good applicants denied | More losses |
| Unusualness level α (0.10) | Policy | Share of validated approvals you accept treating as unusual; it sets how wide the hold band below the cut-off is | Wider band; more manual underwriting | Approvals at risk levels the book has not validated |
| Calibration window (last quarter) | Data | Long enough to have nine or more loans of each outcome near the cut-off | Few loans; the band widens | Old cohorts from a different economy |
| Concentration cut-off (0.10) | Data | Concentration at which past peer groups were hardest to call | More peer groups near it; more holds | Concentrated and diversified peer groups treated alike |
| Agreement margin (0.05 above the line at 1) | Policy | Keep the default unless underwriters find holds that were obviously decidable | More holds | Mixed peer groups act automatically |
Expected holds. Two sources. First, the track-record band: every approval with a PD between the highest repaid score in the calibration set and the cut-off. Count last quarter's applications in that band to estimate the volume. Second, small peer groups: the knife-edge half-width is 0.5 (the cap) up to about 6 peer loans, about 0.30 at 10, and 0.18 at 20, so with a 0.10 cut-off a peer group under about ten loans is held unless it is very concentrated. Estimate both on last month's applications before launch: Estimate impact before launch.
Limits, risks and compliance¶
- Direction is not checked by the peer graph (see the warning above). The scorecard sets the direction.
- Fair lending. Employer, industry and metro area can be proxies for protected characteristics. A peer graph built on them, and a hold rate that differs by group, raise disparate-impact questions. Review held, approved and denied rates by segment, and have compliance approve the peer definition before use.
- Adverse-action reasons. A denial needs principal reasons tied to the
applicant's own attributes (in the US, under ECOA and Regulation B). The
record's per-check
channelslist explains why the engine acted or held; it is not an adverse-action reason, and "the peer group was consistent" is not one either. - Reject inference. The calibration set for approvals contains only borrowers who were approved. Applicants the lender always denied never produce outcomes, so the book can never validate approvals in a band it has never approved. The hold band therefore persists until underwriters approve some of those applicants manually and their loans mature.
- Outcome delay. Loans take months or years to mature. The calibration
set is always a past economy; refresh it as cohorts mature and watch for
shifts (the
live_loan_deskdesk shows a repayment-shift gate). - Model risk. The scorecard, the peer definition and the cut-offs are all model components. Document them under your model-risk governance (for example SR 11-7 in the US) like any other model change.
- Synthetic data. The book, the peer groups and the PDs are constructed to show one approval, one hold and one denial.
Wiring it in¶
The pieces of src/grounded_decision/examples/credit_underwriting.py map
onto a decision engine:
- At service start, build the policy once:
underwriting_module = build(). It is a two-path policy:approve_signalproposesapprove,deny_signalproposesdeny. - Whenever a cohort matures, refresh the calibration history (the
example's
matured_loans_last_quarter()). - Per application, query peer loans by industry code and metro area, and decide:
from grounded_decision.examples.credit_underwriting import (
scorecard_event,
underwriting_evidence,
)
event = scorecard_event(applicant) # approve_signal below 0.12, else deny_signal
evidence = underwriting_evidence(applicant, peer_loans, calibration_history)
record = underwriting_module.decide(event, evidence)
store_with_application(applicant.application_id, record.to_dict())
if record.verdict == "act":
issue_decision(record.action) # approval, or denial with adverse-action notice
else:
route_to_underwriter(applicant, record.proposed_action, record.reason)
store_with_application, issue_decision and route_to_underwriter
are your own functions.
- State. Each application is independent here, so no state is
carried between applications. If your process has several steps
(apply, then scorecard result), save the state between them; the
live_loan_deskdesk shows how. The module is not thread-safe; see Build a decision service.
Run it yourself¶
uv run grounded-decision examples credit_underwriting
uv run grounded-decision examples credit_underwriting --json
How to read the printed report:
Example library: reading a report. HOLD
and hold_for_human in the output are this page's "hold" (the record's
verdict field says abstain). "Flicker band" in the output is the
knife-edge cut-off, and "bag" is the calibration set.
Further reading¶
- Naive vs grounded: the same pattern across cases.
- Live: loan origination desk
(
live_loan_desk): a two-step application process as a running service. - Combinations (
combo_underwriting): this decision combined with a gate on whether repayment behaviour has shifted. - Choose thresholds and calibration and Troubleshoot: why did it abstain?.
- Theory: conformal prediction, recovering a label from related cases (Kesten–Stigum), knife-edge densities (Friedgut–Kalai).
Results this case relies on
| Check | Result | Math catalog number | Trust level of the check |
|---|---|---|---|
| Unusual cases | Split conformal prediction, label-conditional | — | published |
| Neighbour agreement | Kesten–Stigum reconstruction | 229 | proven (lean) that outcomes are recoverable above the line; holding at or below it is policy |
| Knife-edge | Friedgut–Kalai sharp thresholds | 186 | practical (operational): the proven width is far wider than 1 at every realistic size |
| Policy shape and progress | Star height; parity and mean-payoff games | 134, 104 | proven (lean) |