The terms behind Treft
AI coding agent
A system — such as GitHub Copilot, OpenAI Codex, Cursor, Claude, or Devin — that writes or edits code and can open a pull request. An agent is fast and often useful, but it grades its own work, and a change that passes the agent’s own tests can still be wrong.
Independent verification
Checking a code change with an evaluator the author never controlled. Treft verifies a change by re-running the project’s own tests itself, rather than trusting the agent’s claim that they passed. The author never grades its own work.
Clean-room verification
Re-running a change’s tests in a fresh environment the agent never touched: a clean checkout at the patch’s base commit, the exact patch applied, the kept test files protected from tampering, and network egress closed during the test run so a change cannot fetch a published answer. It checks the result independently of the agent’s own claim.
Verifier false-pass rate
Also called the verifier’s false-positive rate: the fraction of runs that pass re-verification yet still contain a human-visible semantic problem. Treft measures and publishes this number. On its July 2026 production census the false-pass rate was 15.4%, 95% CI [12.0%, 19.4%], across n=389 verified passes on 38 production JS/TS repositories (monorepos excluded). This is why Treft reports evidence, not a guarantee of correctness. See the measured numbers and their confidence intervals.
Evidence grade (L0–L3)
A published ladder describing what evidence exists for a change, rather than claiming the change is correct: L0 static (applied and statically checked), L1 clean-room (the kept tests were re-run clean in a fresh environment on the exact patch), L2 synthesized-acceptance (additional acceptance tests were synthesized and run), and L3 divergence-probed (behavior was probed for divergence against a reference). A grade states what was checked; it does not promise a change is bug-free.
Code review vs. verification
Code review is a person reading a diff and judging it. Verification is a machine re-running the change’s tests independently and grading the evidence. Treft does both: it verifies first, then routes a person’s attention — with a review tier and cited evidence — to the changes that most need a human judgment.
Pull-request verification
Verifying a GitHub pull request by materializing its exact patch and re-running the repository’s own tests in a clean room, whoever — or whatever — opened it. Treft verifies pull requests from any coding agent or human contributor and turns each into a decision card.
Isolated sandbox
A disposable, network-isolated cloud environment in which a coding agent runs and in which tests are re-run. Nothing executes on your machine or your phone; the sandbox is torn down after the run. Isolation is what makes it safe to run and verify untrusted, agent-generated code.
Required status check
A GitHub branch-protection rule that blocks a merge until a named check passes. Treft
posts a Treft Verification check-run on every pull request it verifies;
marking that check required makes GitHub itself block merges while verification runs
and when it fails.
Ruler protection
A safeguard against a change quietly weakening the test that judges it. Before Treft re-runs a suite, it restores the kept test files and configuration from the base commit and flags any change to them, so the “ruler” a change is measured by cannot be edited by the change itself. Across Treft’s census, 0 of 56 audited runs modified a kept test.
Decision card
Treft’s per-change review surface: the diff, the independent re-run result, cited receipts, a review tier, and the provenance of the author. It is where a person approves or rejects a change. See an annotated sample decision (illustrative, fictional repository).
Related reading
How verification works — the clean-room mechanism, the grade ladder, and the measured false-pass rate with its confidence intervals. A sample decision — the anatomy of a decision card. Pricing — public repositories are free to verify.