PBGF Conformance Specification

1. Purpose and scope

Software is increasingly produced by agents whose output volume exceeds human review capacity, and whose tests are frequently authored by the same agent that authored the code. In that setting, a green check is a claim, not a control: the question is no longer “did checks run” but “what did the checks prove, and who checked the checker.”

This specification defines conformance requirements for promotion verdicts — the pass/fail/blocked decisions rendered at a promotion boundary (merge, release, deploy) that determine whether an artifact advances. A verdict conforms when the gate that produced it can demonstrate, with evidence, the four requirements in §4.

This specification is not:

• an implementation. Any gate, pipeline, or product may conform.
• a catalogue of checks. It governs how a check earns and exercises authority, not which checks an organization should run.
• a guarantee of correctness. A conforming verdict proves that a calibrated detector, bound to a specific artifact and environment, produced this result under stated conditions — nothing more. §7 states what conformance does not claim.

Relationship to PBGF. The framework (PBGF) is the empirical argument: why trust attaches to promotion rather than behaviour, why governance partitions into three tiers, and what fails when those distinctions are ignored. This specification is the framework’s normative core extracted into testable requirements. The framework persuades; this document conforms or does not.

2. Terms

Normative keywords MUST, MUST NOT, SHOULD, and MAY are used per RFC 2119.

• Artifact — the concrete unit proposed for promotion (a commit tree, a built image, a release bundle), identified by cryptographic digest.
• Promotion boundary — a point at which an artifact advances in authority or exposure (merge, release, deploy, ticket-close). Boundaries are plural; the tier partition (§4.1) is evaluated per boundary.
• Detector — the mechanism that evaluates a property of the artifact and contributes to a verdict. May be static, behavioural (execution-based), or model-based.
• Fixture — a known-good or known-bad exemplar used to calibrate a detector. Protected fixture: one the producer of the artifact under test cannot read or author.
• Producer — whatever created the artifact under evaluation: human, agent, or pipeline. Where the producer also authored the tests, those tests are producer-authored evidence and cannot alone satisfy §4.2.
• Verdict — the gate’s decision for one artifact at one boundary: at minimum PASS, FAIL/BLOCK, NOT-APPLICABLE, and UNATTESTABLE (§4.4).
• Evidence chain — the signed record binding a verdict to the coordinates that produced it (§4.3).

3. The claim structure

A conforming gate makes exactly one kind of claim:

Artifact A, at boundary B, was evaluated by detector D — whose authority was earned under §4.2 — in environment E against policy P, all pinned by digest; the observed outcome, compared against a pre-committed expectation, yields verdict V; and if any of the foregoing could not be established, V is UNATTESTABLE and the artifact does not advance.

Every requirement in §4 exists to make one clause of that sentence provable. Anything a gate asserts beyond that sentence is outside this specification and MUST NOT be labelled a conforming claim.

4. Conformance requirements

4.1 Tier assignment MUST be mechanical

Every enforced property MUST carry a tier — ENFORCEABLE, VERIFIABLE-AT-PROMOTION, or ADVISABLE — produced by a recorded procedure, not by human assertion:

1. Attempt a deterministic check for the property.
2. Red-team the check: attempt to satisfy it syntactically while violating the property (transformation attack), and to violate it while passing (evasion attack).
3. Classify: survives red-team as deterministic and transformation-invariant → ENFORCEABLE; expressible as a hermetic, reproducible execution with an observable outcome → VERIFIABLE-AT-PROMOTION; neither → ADVISABLE.

Evidence required: the tier-assignment record per property — the candidate check, the red-team attempts and their outcomes, the resulting classification, and a revalidation date.

Rationale (informative). Inter-rater agreement between informed humans on tier assignment is poor; a tier asserted by judgment is an opinion wearing a label. The pipeline makes the label reproducible — and makes its honesty auditable, since the red-team record shows why a property failed to reach a higher tier. The ENFORCEABLE class is empirically thin; a conforming gate reports the thinness rather than inflating the tier.

Non-conformance examples. A property labelled ENFORCEABLE on the strength of an AST pattern-match defeated by dynamic dispatch or encoding (transformation attack not run, or run and ignored). A “policy engine” whose every rule is silently ADVISABLE because nothing binds it to the boundary.

4.2 Blocking authority MUST be earned by two-sided calibration

A detector MUST NOT hold merge-blocking (or release/deploy-blocking) authority at a boundary until it has passed two-sided adversarial calibration at that boundary:

1. Catch side: the detector correctly flags every fixture in a protected known-bad corpus for the property it enforces.
2. Pass side: the detector correctly passes every fixture in a protected known-good corpus. A detector that blocks indiscriminately has not earned authority; refusal is not robustness.
3. Independence: protected fixtures MUST NOT be authored by, or readable by, the producer of artifacts the detector will judge. Producer-authored tests MAY be run, but they are producer evidence and satisfy neither side.
4. Revalidation: authority is held against a pinned corpus version. Material change to the detector, its environment, or the corpus REQUIRES recalibration before authority resumes. The corpus SHOULD grow adversarially; calibration is not one-and-done.

Evidence required: the calibration record — detector identity (digest), corpus identity and version (digest), environment identity, per-fixture outcomes for both sides, and the ratification decision (who or what granted authority, when, against which record).

Rationale (informative). This is the requirement that distinguishes a gate from a ritual. In agentic workflows the producer routinely authors both the code and the tests, collapsing the producer/judge separation that review-based quality control assumes. Restoring the separation requires not merely an independent judge but a judge whose competence was itself established independently: a gate never shown to catch the known-bad is a green light of unknown meaning. Two-sidedness is essential — the calibration proves the complete property (catches bad AND passes good), not the easy corner.

Non-conformance examples. A linter granted blocking status by configuration, never shown to catch its target failure. An LLM reviewer with merge authority and no protected corpus. A detector calibrated once, then modified, still blocking on stale authority.

4.3 Verdicts MUST be bound and preregistered

A verdict is meaningful only as a point in a coordinate space. A conforming gate MUST:

1. Bind every verdict to: artifact digest; detector identity and version; calibration-corpus version under which authority was granted; policy identity and version (where policy is externally supplied — e.g. a signed policy bundle — its digest); and execution-environment identity (for behavioural detectors: image digest, isolation configuration, observer configuration).
2. Preregister the expectation where the evaluation is scenario-based: the expected outcome committed and signed before execution, with admissibility judged as signed expectation versus signed observation. An expectation recorded after observing the outcome is a postdiction and does not conform.
3. Represent refutation. The evidence schema MUST be able to record any observable outcome under any scenario — including the outcome that falsifies the expectation. A schema, wiring path, or serializer that can only record the predicted result is a confirmation filter and does not conform.
4. Sign at claim granularity. Every field an admissibility decision reads MUST be signed. Fields MUST be typed by provenance — configured (harness-supplied), captured (observed passing through), measured (produced by the evaluated run) — and a record MUST NOT sign a claim at a higher provenance class than its source supports. Evidence that was not produced MUST be recorded as an explicit signed null, never fabricated and never silently omitted.

Evidence required: the signed evidence chain per verdict — preregistration record, execution record, the binding coordinates, and the admissibility comparison.

Rationale (informative). Unbound verdicts are rebindable: a pass produced under one detector, corpus, or environment can be presented as evidence for another. Preregistration is what makes the gate falsifiable rather than merely repeatable — and a gate that cannot write down its own refutation cannot be trusted to report one.

Non-conformance examples. A CI status with no record of which detector version produced it. A receipt whose “expected outcome” was written after the run. An evidence schema that throws on serializing an unexpected admit. A receipt field labelled “measured” whose value came from harness configuration.

4.4 Absence of proof MUST fail closed

Whenever any element of the claim in §3 cannot be established, the verdict MUST be UNATTESTABLE, and an UNATTESTABLE artifact MUST NOT advance. Triggering conditions include, at minimum:

• detector authority absent, expired, or held against a superseded corpus (§4.2);
• unverifiable signature on any consumed input (policy bundle, fixture corpus, prior attestation);
• staleness beyond a declared freshness bound on any pinned input;
• evidence chain incomplete: a required field neither present nor explicitly signed-null;
• environment identity mismatch between calibration and evaluation where the tier requires identity.

UNATTESTABLE MUST be distinguished from FAIL (the detector judged the artifact bad) and from infrastructure failure (the evaluation could not run). Infrastructure failure MUST NOT be recorded as a governance refusal, and MUST NOT be admissible as evidence that enforcement occurred.

Evidence required: the UNATTESTABLE record naming the unestablished element; the freshness bounds declared per input.

Rationale (informative). Fail-open is how controls die silently: the gate that passes when it cannot attest is indistinguishable, from the outside, from a gate that attested. Routing every could-not-establish condition to one explicit blocked state means degradation is always a decision someone sees, never an absorption. The freshness bound belongs to policy, not to operations: tightening it from evidence is safe; letting inputs age because refresh is friction is the standing danger.

Non-conformance examples. A gate that skips a check when its policy server is unreachable and reports green. A signature-verification failure downgraded to a warning. A “temporary” bypass flag with no expiry and no UNATTESTABLE accounting.

5. Conformance levels

Levels are cumulative. Each names who vouches for the evidence.

Level 1 — Self-attested. All four requirements of §4 are met and the evidence chains exist, retained and queryable, signed by the gate operator. The operator is trusted for both the evaluation and the record.

Level 2 — Calibrated and separated. Level 1, plus: protected fixtures are demonstrably outside the producer’s reach (enforced separation, not policy); calibration ratification is performed by a party or mechanism other than the detector’s author; signing keys are outside the evaluated workload’s trust domain; and preregistration records are crash-durable (persisted before execution begins).

Level 3 — Independently attested. Level 2, plus: evidence chains are emitted in an interoperable attestation envelope (e.g. an in-toto Statement with a behavioural-verification predicate) verifiable without access to the gate operator’s infrastructure; execution identity is attested by a mechanism rooted outside the operator (e.g. hardware attestation of the verification environment); and the calibration record itself is independently reproducible from the published corpus digest.

(Informative: Level 1 is achievable by a disciplined team with existing CI primitives plus the evidence requirements of this document. Level 2 is where the producer/judge separation becomes structural. Level 3 is where a third party — an auditor, a counterparty, a regulator — can verify a verdict without trusting the operator’s word. The ladder is deliberately shaped like SLSA’s so that organizations already fluent in supply-chain levels can locate themselves without new vocabulary.)

6. Composition with adjacent standards and frameworks (informative)

This specification governs a claim no adjacent standard makes, and deliberately makes none of theirs:

• Provenance (SLSA, in-toto, TEE-attested builds) proves where an artifact came from — source, dependencies, toolchain. It is explicitly silent on whether the code is correct. A provenance attestation and a conforming promotion verdict compose on the same artifact: where it came from and how it behaved when judged, two predicates, one envelope.
• Policy-as-code (OPA/Rego, admission control) governs decisions over declared facts. A conforming gate MAY consume signed policy bundles as pinned inputs (§4.3, §4.4) and MAY delegate its declarative-tier checks to such an engine. The engine’s evaluation does not itself confer conformance: authority (§4.2) and binding (§4.3) remain the gate’s obligations.
• Runtime containment (sandbox policies, capability confinement) bounds what a process may do while running. Containment reduces blast radius; it does not judge the artifact. The two are complementary controls at different points.
• Input-side assurance (adversarial training, reasoning monitoring, constrained generation) improves the producer. This specification is producer-agnostic by design: nothing in §4 becomes unnecessary when the producer improves, because §4.2’s premise is that producer quality is not a control.
• Deployer governance frameworks (e.g. IMDA’s Model AI Governance Framework for Agentic AI) govern how organizations deploy and oversee agents — accountability allocation, human oversight design, staged rollout, monitoring. Such frameworks and this specification meet at the promotion boundary: they define who is accountable for the gate and how it is operated; this document defines what the gate must prove. A deployer framework’s preference for deterministic, bound-by-design controls and deny-by-default failure handling is precisely the posture §4.4 makes testable at the artifact level.

7. What conformance does not claim

A conforming verdict does not claim:

• that the artifact is free of defects. It claims the artifact passed calibrated detectors for the specific properties enforced, at their assigned tiers.
• that novel harms outside the calibrated corpus are caught. The corpus is finite and grows adversarially; conformance includes honesty about its extent (§4.2 evidence), not omniscience.
• that behaviour observable only after promotion (deferred execution, runtime-resolved dependencies) was judged. The promotion boundary binds artifact content and boundary-observable behaviour; later boundaries with richer context close part of this gap, and a conforming gate at deploy MAY re-partition tiers accordingly (§2, boundary plurality).
• that the gate’s own platform is beyond compromise. The trust ceiling of the gate’s compute is an explicit coordinate of Level 2/3 evidence, not a solved problem to be assumed.

A gate that states these limits conforms; a gate that claims their absence does not, regardless of its other properties.

8. Versioning and governance of this specification

This document is versioned semantically. Requirements MAY be tightened in minor versions when motivated by published evidence (a demonstrated evasion, a falsified assumption); requirements MUST NOT be loosened except in a major version with a documented rationale. The asymmetry is deliberate and mirrors §4.4’s freshness doctrine: tightening from evidence is safe; loosening from friction is the standing danger.

Conformance claims MUST cite the specification version they were evaluated against.

Citing this draft: PBGF Conformance Specification, Draft v0.1, moriapp.dev/pbgf-cs. Draft status means requirement numbering may change before v1.0; cite the section and version together.

Licence: this specification text is released permissively so that conformance can be claimed, implemented, and audited without licence friction. A reference implementation is released separately under Apache-2.0.