Conformance Levels#

HushSpec defines six conformance levels. Each level subsumes all requirements of the levels below it, so a Level 4 implementation is also a Level 3 one. The normative definitions are in spec/hushspec-core.md section 8; this page is the reader's version, plus the vector coverage behind it.

For the current per-SDK status on main, see the SDK Conformance Matrix. To publish a claim of your own, fill in the Conformance Statement template.

To test an executable outside the built-in reference runner, use the experimental external engine controller. It binds L0-L3 results to captured engine bytes and retains a verifiable execution packet.

Claiming a level#

Quick Reference#

A higher level includes lower-level obligations. SDK feature availability and external engine claims are different kinds of evidence.

Claim Evidence#

A claim is made against a specific corpus, not against "the fixtures". Every file under fixtures/ is inventoried in fixtures/MANIFEST.json with its SHA-256, its category, and the level at which it becomes required, and a claim names the corpus by the SHA-256 of that manifest.

The machine-readable form of a claim is a conformance report: hushspec-conformance-report.v1.schema.json. The reference runner writes one, and validates it against its own schema before writing:

hushspec-testkit --fixtures fixtures --report report.json

highest_level in the report is the largest N for which levels 0 through N all pass. A level with any unattempted vector reports not_attempted, which is never a synonym for a pass. The runner diffs the manifest against what it actually ran, so a vector the run missed is recorded as not_attempted rather than left out of a report that still cites the manifest by digest.

Everything needed to do this without cloning the repository ships as hushspec-conformance-<version>.tar.gz, attached to every release: the prose, the schemas, the vectors, and the manifest.

Level 0: Parser#

A Level 0 implementation can:

• Parse valid HushSpec YAML documents into a structured representation
• Reject syntactically invalid YAML, and input violating the YAML profile of core spec 2.4 (aliases, merge keys, duplicate keys, multi-document streams, yes/no booleans)
• Reject documents missing the required hushspec field
• Meets every raw YAML parse-acceptance assertion in fixtures/core/raw-yaml/scalars.json; a missing or malformed raw corpus cannot establish a passing parser result

This is the minimum bar for any tool that reads HushSpec documents.

Level 1: Validator#

A Level 1 implementation additionally:

• Validates all field types and constraints (booleans are booleans, integers are integers)
• Rejects documents with unknown fields at any nesting level
• Validates enum values (severity, mode, default, merge_strategy)
• Enforces uniqueness constraints (e.g., secret pattern name fields)
• Validates numeric constraints (non-negative integers, positive ratios)
• Validates regex syntax in pattern fields against the profile of core spec 3.14
• Validates when conditions
• Decodes each accepted fixtures/core/raw-yaml/scalars.json value without rewriting the YAML source before it reaches the parser
• Rejects every invalid/ vector, with the error code its .expect.yaml sidecar names -- and any message_contains substring it names -- if the implementation reports codes at all

This level is required for linters, schema validators, and policy authoring tools.

Error codes#

Every fixtures/<module>/invalid/<name>.yaml has a <name>.expect.yaml sidecar naming the code its refusal must carry, drawn from spec/registries/error-codes.yaml:

reject: true
code: "E001"
message_contains: "anchors are not allowed"

Without this, "the document was rejected" is a weak assertion: a vector that tests the YAML profile passes just as well when the engine refuses it for an unrelated reason.

All four reference SDK fixture runners assert the code and the message_contains substring, not merely that the vector was rejected. The spec's rule is the weaker one, and stays that way for third-party engines: an implementation that reports no codes at all conforms at this level; one that reports codes from the registry MUST report the registered one. Where each SDK's codes come from is tabulated in the SDK API Contract.

Level 2: Merger#

A Level 2 implementation additionally:

• Resolves extends references (via at least one resolution strategy: filesystem, URL, registry, or built-in)
• Correctly implements all three merge strategies (deep_merge, merge, replace)
• Detects and rejects circular inheritance

This level is required for any tool that supports policy composition.

Level 3: Evaluator#

A Level 3 implementation additionally:

• Accepts an action (type + inputs) and a resolved HushSpec document
• Produces a correct structured evaluation result containing at least a final allow, warn, or deny decision, under the semantics of core spec sections 3, 5 and 6 -- including the normalization and matching algorithms of section 3.14
• Implements aggregation and precedence per core spec 6.1 (deny > warn > allow) and denies unknown action types per section 5
• Passes every vector under fixtures/<module>/evaluation/: for each case the decision, plus each of matched_rule, reason, origin_profile and posture the case states. The vector format is hushspec-evaluator-test.v1.schema.json
• Reproduces the fixed example.com/requests: 9 decision in each raw-YAML vector that declares one

This is the full engine level. All four HushSpec SDKs pass it, and go on to Levels 4 and 5. Clawdstrike's conformance level is not qualified here: it needs a pinned report from a harness that actually executes that engine. The bundled reference runner cannot establish external-engine conformance by relabeling its report.

Level 4: Auditor#

Level 3 says an engine reaches the right decision. Level 4 says it can prove which document it reached that decision under, and why, to someone who was not there.

A Level 4 implementation additionally:

• Emits decision receipts at format 0.2 that validate against the published schema
• Computes policy.content_hash as the canonical hash of the resolved document, and reproduces every fixtures/core/hash/ vector byte for byte
• Records rule_trace during evaluation rather than reconstructing it afterwards, and reproduces every committed receipt under fixtures/receipts/expected/ after canonicalization
• Accepts every fixtures/receipts/valid/ vector and rejects every fixtures/receipts/invalid/ one
• Resolves extends with chain provenance and digest pinning, passing fixtures/core/resolve/
• Reproduces canonical JSON and content hashes for raw-YAML cases that declare a canonical output

Given a Level 4 receipt and the policy it names, a third party can recompute the hash, replay the trace, and get the same answer.

Level 5: Attested#

Level 4 evidence is only as trustworthy as the document it was produced under. Level 5 adds provenance: which policy was in force, who signed it, and whether the record has been altered since.

A Level 5 implementation additionally:

• Is a conforming signature verifier: every case in fixtures/signing/vectors.yaml returns valid or the exact reason code
• Verifies on load, so every hop of an extends chain is checked against the keyring or its digest pin, fails closed, and records the outcome in receipt.policy.signature
• Verifies a hash-linked log, identifying which line first breaks the chain for every fixtures/log/invalid/ vector
• Refuses every schema-derived invalid entry in fixtures/log/schema-vectors.json
• Signs and verifies receipts (fixtures/receipts/signed/)
• Verifies policy bundles (fixtures/bundle/vectors.yaml)

An implementation may claim Level 5 for verification only: producing signatures, logs and bundles is described by the same specifications, but verification is what a relying party depends on.

Fixture coverage#

Every requirement below has at least one published vector; there are no gaps to fill in later. Paths are relative to fixtures/, and fixtures/MANIFEST.json pins each file's digest, category and level.

Rule blocks (core section 3)#

Cross-cutting evaluation behaviour#

Merge and resolution#

Extension modules#

Evidence chain#