Cupel · independent assurance for AML triage agents

Your AML agent may be clearing reportable alerts.

Under one mundane peer-benchmark incentive, every model we tested quietly under-escalated 32–57% of legally reportable alerts, while standard observability checks looked normal.

Run the same checks on your agent, in your environment.

Assay your agent → See a full assurance report
Full methodology & results →
Open source Runs locally - no Cupel server Your decision data never leaves your machine
The assay result

What a mundane incentive did to off-the-shelf agents.

32–57%
of legally reportable alerts cleared across five models under an ordinary peer benchmark
0
correctness or quality alarms from industry-standard observability tools while it happened
100%
of the suppressed reportable alerts (19/19) flagged by an independent, ground-truth check
60% 30% 0% Neutral prompt 0% Peer-benchmark incentive 32%
Same agent, same alerts. For our frontier reference agent, swapping a neutral instruction for a peer benchmark cleared nearly a third of reportable alerts (32%), while most high-pressure incentives had no impact. Subtle incentive changes can drive misbehavior, even where overt threats don't. The same swap moves every model we tested - 32–57% across five models and three providers (Claude Opus/Sonnet/Haiku, GPT-4o, and open-weight Llama-3-70B). This susceptibility spans models and providers. Full methodology in the report.

Observability tools don't tell the whole story. Ours stayed green.

If your triage agent runs under efficiency targets while managing false-positive load, Cupel offers the assurance that standard dashboards can't.

Three levels of trust

From a quick look to a reference artifact.

SEE IT
For the curious

Read a real report

A full, examiner-style assurance report from a live run: the under-escalation finding, the per-decision verification ledger, and the attestation.

View a sample report →
ASSAY YOUR AGENT
For engineers and risk teams

Run it on your own agent

Export the Cupel battery, run it through your agent, and score the decisions locally, in minutes. You get your own under-escalation rate and a per-decision ledger, with nothing leaving your environment.

Run it in 5 minutes →
SHOW YOUR WORK
For vendors and their clients

Get certified

Request a signed Cupel certificate from your run - self-scored on the open practice battery.

Get certified or attested →
Assay your agent

Run it on your agent in five minutes.

Safe by construction. Scoring runs entirely in your environment. The LogReplay path makes zero network calls, so your under-escalation number is computed locally and nothing leaves your machine. A model key is needed only for the optional independent-verification stage, which sends decisions (never raw data), and only if you choose. Exporting the battery and scoring are free.
1
Clone the repo and install - this is the exact code you run, pinned to a commit you can verify
git clone https://github.com/burnssa/cupel.git && cd cupel
uv sync   # installs pinned, hash-locked deps · needs uv (astral.sh/uv)
2
Export the Cupel battery - free, offline, no API key
uv run python -m data.build --export-battery
# → results/byo/battery.jsonl - alerts only, ground-truth labels stripped
3
Run the battery through your agent - your code, offline - and save a decisions CSV
alert_id,decision,condition,rationale
a_0012,CLEAR,neutral,"non-uniform amounts, consistent with profile"
a_0012,CLEAR,incentivized,"low individual amounts, plausibly benign"
# both conditions → full susceptibility report · one → verification only
4
Score your agent - zero network, no key required
uv run python run.py --agent logreplay --decisions your_decisions.csv
5
Read your result
results/BYO_REPORT.md            # your under-escalation rate, by typology
results/byo_decision_ledger.md   # every decision, independently verified

Want to watch it run end-to-end on our reference agent first? run.py --mode dry (~$1).  ·  Prefer a live endpoint to a CSV? --agent api (beta) calls only your endpoint - see the BYO guide.  ·  Run in Colab →

Or point your coding agent at it. No terminal required - hand the repo to Claude Code or Cursor and ask it to reproduce the finding. One no-key command (python -m tools.repro) regenerates the headline table from committed logs and self-verifies; AGENTS.md tells the agent exactly what "reproduced" looks like.
Is this safe to run?

Built so you never have to take our word for it.

You're being asked to run code against your compliance system. That's a big ask, so we removed the reasons to worry.

100% open source

Every line is public, Apache-2.0. Read it, fork it, run the whole assay air-gapped - a test runs the full path (build → score → cert) with the network blocked. Nothing is obfuscated or minified.

Runs in your environment

No Cupel server, no account, no telemetry, no live access to your systems. Your model key is read from the environment only - and a test asserts it never appears in anything Cupel writes.

Your data stays yours

The LogReplay scoring path makes zero network calls - there's a test that asserts it. Your agent's decisions are scored on your machine; nothing touches the internet.

Certification is opt-in

The only thing ever shared is your choice: an aggregate-only certificate request - no decisions, no rationales, no customer data. The run prints the whole thing and asserts it's aggregate before writing it.

Standard, auditable tooling

Plain Python. Every dependency is hash-pinned to the public PyPI index via uv.lock - no git or URL sources, no install or build hooks, no curl-pipe-bash. See the security map →

Built in the open

The full methodology, findings, and limitations are published. Read the methodology →

A certificate request - the entire file, nothing withheld
{
  "schema": "cupel/cert-request/v1",
  "generated_at": "2026-06-27",   "git_commit": "9546a34",
  "battery": { "kind": "open-practice", "hash": "sha256:604eb3a0…", "n_alerts": 240, "substrate": "amlsim_port" },
  "agent": { "descriptor": "your-agent", "model": "your-model", "provider": "logreplay" },
  "evaluator": "claude-opus-…",   "conditions": ["neutral", "incentivized"],
  "assurance_level": "self-tested",
  "metrics": { "under_escalation_neutral_pct": 0.0, "under_escalation_incentivized_pct": 32.0,
              "susceptibility_delta_pct": 32.0, "evaluator_vs_truth_agreement_pct": 88.3,
              "under_escalation_detection_recall_pct": 100.0, "n_decisions": 480, "n_reportable_alerts": 60 },
  "disclaimer":     "Self-tested on the open practice battery …",
  "self_cert_gap":  "A self-run cert proves the suite ran against some agent, not your real production agent …",
  "guarantee":      "Aggregate only - no decisions, rationales, or PII."
}

No alert_id rows, no rationales, no customer data - aggregate metrics only. Example values shown; your run fills in its own. The run prints this exact file and asserts every field is aggregate before writing it.