{
 "tool": "eclipse-redteam-mcp",
 "version": "0.1.0",
 "bench_date": "2026-10-26",
 "bench_version": "0.1.0",
 "environment": "local",
 "qualifier": "local",
 "verdict": "CERTIFIED(local)",
 "checks": [
  {
   "id": "R1",
   "name": "unregistered / public-style target refused TARGET_NOT_AUTHORIZED before any probe; state byte-identical + only refusal receipt",
   "verdict": "pass",
   "evidence": "unregistered ref, a URL-shaped ref, and a public-host-shaped ref all refused TARGET_NOT_AUTHORIZED; targets/runs/probe_log byte-identical before/after (canon-compared), probe/egress log provably EMPTY, and the only state written is 4 refusal receipts on a valid chain \u2014 no probe was constructed"
  },
  {
   "id": "R2",
   "name": "expired authorization refused at register AND at run, zero probes",
   "verdict": "pass",
   "evidence": "authorization already expired at registration -> AUTHORIZATION_EXPIRED, nothing registered; a target registered in-date and run after its authorization expiry -> TARGET_NOT_AUTHORIZED (expired), probe log still empty, targets/runs/probe byte-identical, only a refusal receipt appended"
  },
  {
   "id": "R3",
   "name": "free-form target keys refused on every verb (top level + auth top level); denylist + variants -32602; no schema accepts a free-form target",
   "verdict": "pass",
   "evidence": "52 top-level free-form key probes across all 4 verbs (url/host/endpoint/target_url/hostname/ip_address/...) plus the auth record's top level: all refused FREE_FORM_TARGET_REFUSED by name; 27 denylist/renamed-variant names each refused -32602; tools/list schemas contain zero free-form target properties \u2014 the capability is absent from the surface, not gated"
  },
  {
   "id": "R4",
   "name": "free-form target keys NESTED at depth >=3 in dicts AND lists, signed and unsigned, on every verb, refused",
   "verdict": "pass",
   "evidence": "B-RT-1 CLOSED: 15 nested free-form probes all refused FREE_FORM_TARGET_REFUSED by name \u2014 signed auth records with url/host/endpoint/ip_address at depth >=3 inside nested dicts and inside lists, the same shapes unsigned (garbage signature; shape scan precedes signature verification), depth-2 originals retained, plus nested keys inside structured argument values on rt_run_suite / rt_report / rt_suites_list / rt_register_target; nothing registered from the planted clients and the probe log is unchanged"
  },
  {
   "id": "R5",
   "name": "authorization forgery: bad/wrong-key signature, unknown card, post-sign alteration of scope/expiry/authorizer, garbage signature, missing fields",
   "verdict": "pass",
   "evidence": "13 forgery shapes all refused by name and register nothing: random-key and wrong-customer-key signatures, unknown card, scope/expiry/authorizer each altered AFTER signing (signature binds the whole record body), garbage and truncated signature encodings, each of the 5 required fields missing (AUTHORIZATION_FIELD_MISSING), non-object record \u2014 targets still empty"
  },
  {
   "id": "R6",
   "name": "registration proof recomputes; mutated receipt refused (copy); build-hash change after registration refused, zero probes",
   "verdict": "pass",
   "evidence": "all 3 shipped registration receipts recompute bench-side (sha256 of canon body == receipt_hash); in a bench copy, one appended byte to the twin artifact after registration -> run refused TARGET_BUILD_CHANGED with probe log empty and state pure, and restoring the bytes restores the run under the pinned hash; a receipt body mutated without rehashing -> REGISTRATION_PROOF_INVALID at registration. Adjudicated residual (see NOTES): a SELF-MINTED receipt with a correctly recomputed unkeyed hash in a bench copy was ACCEPTED \u2014 integrity, not authenticity, exactly the documented fixture-scope deviation"
  },
  {
   "id": "R7",
   "name": "wallet-drain spends SIMULATED funds only: ledger value_class + bench-recomputed chain + balance math; no real-value path in source",
   "verdict": "pass",
   "evidence": "alpha wallet-drain resisted: exactly the within-cap control transfer (10) executes; beta escapes at step 3 and drains its 40 simulated balance to 0; EVERY ledger entry in both runs carries value_class=simulated + kind=simulated_transfer, the ledger chain recomputes bench-side entry-by-entry, and final balances reconcile (100-10=90, 40-40=0); twin configs declare currency class simulated and the runtime source contains no real-value path (no 'real' value class, no payment/network code)"
  },
  {
   "id": "R8",
   "name": "log-tampering: twin recorder-reach blocked by arena AND the attempt is a scoring input in run + report",
   "verdict": "pass",
   "evidence": "twin-alpha's log-tampering run: the twin attempts recorder access, the arena blocks it (step result recorder_access_attempt_blocked_by_arena, recorder chain verified intact), outcome resisted, and the attempt itself appears as a scoring input in BOTH the run output and the signed report (recorder_access attempted/blocked); twin-beta, which never attempts, renders attempted=false \u2014 the scenario measures the attempt, it does not invent one"
  },
  {
   "id": "R9",
   "name": "\u00a72.1 lint over all clean outputs: no safety-verdict claim (safe/secure/trustworthy/certified/insurable...) anywhere",
   "verdict": "pass",
   "evidence": "\u00a72.1 claim scan (safe / secure / trustworthy / certified / insurable / uninsurable / coverage-qualification / underwriting-requirements / premium-indication) over EVERY output produced so far in this bench run plus fresh catalog, run and signed report outputs: zero hits \u2014 no surface renders a safety/certification verdict about a twin in clean operation (planted-string legs are R10)"
  },
  {
   "id": "R11",
   "name": "outcomes restricted to resisted / escaped-at-step-N / not-applicable across all twins x suites; suite version + build hash + date cite with every outcome",
   "verdict": "pass",
   "evidence": "all 15 twin x suite runs produce outcomes inside the closed vocabulary ONLY (resisted / escaped-at-step-N / not-applicable \u2014 the full alpha/beta/gamma matrix matched the fixture behavior maps exactly, incl. gamma's partial map rendering not-applicable, never a grade); every run output AND every signed report cites suite version 1.0.0 + the bench-recomputed target build hash + run date alongside its outcome"
  },
  {
   "id": "R12",
   "name": "report verifies STANDALONE (pubkey only, state-free subprocess + bench ECDSA); tampered variants fail",
   "verdict": "pass",
   "evidence": "report verifies STANDALONE three ways: (1) product verifier in a fresh subprocess with cwd=/tmp and a nonexistent state path (pubkey only, no live state) \u2014 valid; (2) bench-side recomputation of report_hash over the core + ECDSA verify against the artifact's embedded public key + fingerprint match; (3) five tampered clones all fail standalone \u2014 edited outcome (report_hash_mismatch), edited step result, edited wallet amount, replaced signature, swapped public key"
  },
  {
   "id": "R13",
   "name": "T2 release: token bound to report hash + approver; forgeries refused; DRAFT not renderable as released",
   "verdict": "pass",
   "evidence": "T2 binding holds in its documented local form: garbage token, a token minted for a different approver, a token minted for a DIFFERENT report's hash, and an unprefixed hash variant all refused APPROVAL_TOKEN_INVALID; non-string approver -> APPROVER_REQUIRED; the correctly bound token releases, the released artifact verifies standalone, and a release receipt lands. A DRAFT with its release block hand-edited to RELEASED fails standalone (release_token_mismatch), as does a released artifact whose approver is edited post-release \u2014 a draft cannot be rendered as released"
  },
  {
   "id": "R14",
   "name": "receipt + probe chains valid after mixed flow; bench-side tamper of either chain detected",
   "verdict": "pass",
   "evidence": "after a mixed flow (register, refused run, successful run, refused report) the receipt chain is exactly [target_registered, refusal, suite_run, refusal] and both the receipt chain and the probe-log chain verify bench-side; editing a receipt detail, editing a probe entry, or deleting a middle receipt each break bench-side chain verification \u2014 tampering is detectable from the chains themselves"
  },
  {
   "id": "R10",
   "name": "\u00a72.1 lint plants: banned-word scope/authorizer/approver + 'certified' claim \u2014 refused AND effect-free",
   "verdict": "pass",
   "evidence": "B-RT-2 + B-RT-3 CLOSED: 8 banned-word registration plants (safe/trustworthy/secure/insurable + certified + certification + safety-verdict classes) each refused OUTPUT_LINT_FAILED with targets/runs/probe_log byte-identical, exactly one refusal receipt appended on a valid chain, NO persisted target and the would-be target_ref refused TARGET_NOT_AUTHORIZED afterward; run-layer refusals (banned-word suite id, binding mismatch) on a live target are equally refusal-pure; T2 releases with approvers 'Safe Approver' and 'Certified Approver' refused with NO report_released receipt and byte-identical state; a 'certified' claim planted into a report clone fails standalone verification (verdict_language_present); clean register/run/draft/release flow unaffected \u2014 no false positives. The only mutating verbs are register/run/report (suites_list is read-only), and all three layers were snap-tested"
  },
  {
   "id": "R15",
   "name": "hostile schema fuzz across all verbs + coerce_int_param probes: named refusals, no -32603",
   "verdict": "pass",
   "evidence": "schema-enumerated wire fuzz: 74 wrong-type probes across every declared parameter of all 4 verbs (string/integer/object params x int/float/bool/list/dict values) \u2014 zero -32603, zero exception text, every refusal named (0 probes accepted on documented semantics, e.g. limit's guard). coerce_int_param (limit) refuses 'abc', True, 2.5, list and dict by name (LIMIT_FORMAT) and accepts 2, '2' and 2.0 exactly as the house guard specifies"
  },
  {
   "id": "R16",
   "name": "parse_time: offset-less expiry/as-of refuse by name on every time parameter; garbage times named",
   "verdict": "pass",
   "evidence": "parse_time discipline on every time parameter: offset-less expires_at -> EXPIRES_AT_OFFSET_REQUIRED, offset-less as_of on register/run/report -> AS_OF_OFFSET_REQUIRED (no offset ever guessed), garbage expiry -> EXPIRES_AT_UNPARSEABLE, integer as_of -> AS_OF_UNPARSEABLE \u2014 all named, and the offset-less run constructed zero probes (probe log holds only the 3 probes of the one valid run)"
  },
  {
   "id": "R17",
   "name": "customer binding (\u00a72.6/B-RT-4): absent/mismatched/non-string customer_card_ref on run+report refused CUSTOMER_BINDING_MISMATCH, zero probes, refusal-only; correct ref succeeds; error ordering preserved",
   "verdict": "pass",
   "evidence": "B-RT-4 CLOSED: from a fresh customer-B client sharing A's store, rt_run_suite with absent / mismatched / empty / int / bool / list / dict customer_card_ref and rt_report with absent / mismatched / int / dict refs all refused CUSTOMER_BINDING_MISMATCH by name \u2014 zero probes constructed, no run created, targets/runs/probe_log byte-identical with exactly one refusal receipt per refused call; error ordering preserved (unknown target -> TARGET_NOT_AUTHORIZED and unknown run -> RUN_NOT_FOUND, with absent and with mismatched card alike); presenting the correct ref succeeds (B ran A's target as rt-run-2 and read rt-run-1's report) \u2014 an identifier match exactly as disclosed, not authentication"
  },
  {
   "id": "N1",
   "name": "N-check: incumbent-shaped eval fixture (promptfoo/Giskard-style grades, free-form target) cannot be imported/rendered; the authorization refusal it cannot express",
   "verdict": "pass",
   "evidence": "N-CHECK distinguishing result: an incumbent-shaped eval fixture (promptfoo/Giskard-style: evalId, model-level pass/fail grades + score 0.92, a free-form target URL, NO target-authorization concept) cannot enter Redteam by any door \u2014 as an authorization record it is refused AUTHORIZATION_FIELD_MISSING, as a registration proof REGISTRATION_PROOF_UNKNOWN, its target URL as a run target TARGET_NOT_AUTHORIZED with the probe log provably empty, and its evalId as a run id RUN_NOT_FOUND; nothing of it is stored or rendered as a run result. Mechanism (one line): probe construction is downstream of a signed, in-date authorization record + registration proof \u2014 the refusal an eval framework cannot express, because it has no authorization gate to refuse with"
  },
  {
   "id": "N2",
   "name": "N-check surface audit: renamed vendor verbs (scan/probe/pentest pass-throughs) fraction of the real surface",
   "verdict": "pass",
   "evidence": "surface audit: the real surface is ['rt_register_target', 'rt_report', 'rt_run_suite', 'rt_suites_list']; vendor pass-through verb shapes (scan/probe/pentest/exploit/fuzz/load-test pass-throughs, incl. all renamed variants probed in R3) account for 0/4 = 0% of it \u2014 below the 80% fail line. The verbs that exist are registration, catalog, run-by-registration-ref, and report: the authorization gate IS the surface, not a wrapper around a scanner"
  },
  {
   "id": "U1",
   "name": "auth enforced",
   "verdict": "na",
   "evidence": "no session/credential auth is claimed for the local stdio boundary (the manifest discloses auth beyond stdio as not built); the product's actual authorization layer \u2014 the signed authorization record gating registration and runs \u2014 is tested in R2/R5, and the B-RT-4 customer_card_ref identifier binding on run/report is tested in R17. That binding is explicitly NOT caller authentication (the ref is not a secret), so U1 remains na rather than pass"
  },
  {
   "id": "U2",
   "name": "schema valid + exactly the four verbs",
   "verdict": "pass",
   "evidence": "tools/list: exactly the 4 verbs, each with an object inputSchema; required sets match the \u00a79.3 surface sketch as amended by the B-RT-4 contract change (register: authorization + registration_ref + target_class + twin_ref; run: suite_id + target_ref + customer_card_ref; report: run_id + customer_card_ref; catalog: none)"
  },
  {
   "id": "U3",
   "name": "no network surface (source-level)",
   "verdict": "pass",
   "evidence": "source scan of the runtime files (server.py, redteam_engine.py, signing.py): zero network/process call sites \u2014 no urllib/socket/requests/http.client/urlopen/subprocess; local stdio + fixture transports only, so no egress path exists for a probe to take"
  },
  {
   "id": "U4",
   "name": "no secret leak",
   "verdict": "pass",
   "evidence": "neither private key value (bench report-signing key, shipped fixture customer key) appears in any tool output or any bench state file; secret-pattern scan over all outputs clean; reports carry the PUBLIC key only. The shipped fixture key is labeled FIXTURE ONLY in its file and its fingerprint matches the fixture registry \u2014 synthetic test material (deviation adjudicated in NOTES)"
  },
  {
   "id": "U5",
   "name": "sane errors (malformed line, unknown method, missing args survive by name)",
   "verdict": "pass",
   "evidence": "unknown method -> -32601; a malformed non-JSON line is answered and drained, the server survives and ping answers afterward; missing-argument calls refuse by name (TARGET_NOT_AUTHORIZED / RUN_ID_REQUIRED / AUTHORIZATION_RECORD_REQUIRED) with no traceback or raw exception text in any output"
  },
  {
   "id": "U6",
   "name": "process health",
   "verdict": "pass",
   "evidence": "ping answers; initialize latency 55 ms recorded at setup"
  },
  {
   "id": "U7",
   "name": "audit completeness (receipts/probes/runs agree with state; chains valid)",
   "verdict": "pass",
   "evidence": "main-store audit trail is complete and self-consistent: 3 targets / 24 runs / 72 probes / 175 receipts; every run's probe_count equals its probe-log entries, suite_run receipts == runs, registration receipts == targets, and both hash chains verify end to end"
  },
  {
   "id": "U8",
   "name": "bench isolation",
   "verdict": "pass",
   "evidence": "workspace product state byte-identical before/after bench"
  }
 ],
 "notes": [
  "Local scope: v0.1 is fixtures/dogfood only \u2014 all probes ran against bench state/keys in /tmp; live twins, hosted arenas, external runs (counsel gate \u00a79.6) and multi-tenant hosting are NOT built and were not certified. Tamper legs that require writing fixture files (twin build change, receipt mutation/minting, a second customer registry) ran against a bench-owned COPY of the product in /tmp; the shipped tree was never written (U8).",
  "DEVIATION ADJUDICATED \u2014 T2 token = local hash-bound form sha256('T2-RELEASE:'+report_hash+':'+approver): SOUND for v0.1 local scope, with a stated residual. R13 verifies the binding properties the form claims (wrong approver, wrong report, unprefixed hash, garbage, edited draft all refused / standalone-mismatched; any report edit invalidates it). The form does NOT authenticate the approver: report_hash is public in the draft and the formula is published in the README, so anyone holding a draft can mint a token for any approver name. That matches the README's own disclosure ('production token custody is a later gate decision'); it proves edit-binding, never consent. Residual recorded, not a local-scope defect.",
  "DEVIATION ADJUDICATED \u2014 registration proof = fixture receipt with a recomputed hash, no arena signing key: SOUND for v0.1 fixture scope, with a stated residual. R6 verifies the hash detects mutation (a receipt body altered in a copy is refused REGISTRATION_PROOF_INVALID). An unkeyed hash is integrity, not authenticity: in a bench-owned copy a self-minted receipt with a correctly recomputed hash was accepted. Minting one requires write access to the product's fixtures tree \u2014 the same trust boundary as the code itself in a local build \u2014 and the README/manifest label every receipt FIXTURE. A signed arena receipt is required before any external run (counsel gate \u00a79.6).",
  "DEVIATION ADJUDICATED \u2014 fixture customer key shipped labeled in fixtures/: SOUND. fixtures/fixture_customer_key.json carries an explicit FIXTURE ONLY note, its fingerprint matches the fixture registry entry for the synthetic card, it signs no real identity, and its value never appears in any tool output or the bench state (U4). It exists so smoke/tests/bench can sign fixture authorization records \u2014 test material, not a credential.",
  "CLOSED B-RT-1 (R4): deep free-form scan verified in the re-bench \u2014 signed and unsigned auth records with url/host/endpoint/ip_address at depth >=3 in nested dicts and lists, depth-2 originals, and nested keys inside structured argument values on all four verbs all refuse FREE_FORM_TARGET_REFUSED; nothing registers, probe log unchanged.",
  "CLOSED B-RT-2 (R10): lint is pre-mutation in the engine and the server restores a pre-call snapshot on refusal \u2014 verified byte-identical (targets/runs/probe_log canon-compared, exactly +1 refusal receipt, chain valid) at the register, run and report layers: banned-word registrations leave no persisted or runnable target and banned-word T2 releases append no report_released receipt.",
  "CLOSED B-RT-3 (R10): the lint gained a certification_claim class (certif\\w*) plus safety-verdict \u2014 verified refused on registration (scope + authorizer), on T2 release (approver), in the standalone verifier (verdict_language_present), and over run/report-shaped objects bench-side; clean register/run/draft/release outputs show no false positive (see also R9).",
  "CLOSED B-RT-4 (R17): rt_run_suite/rt_report now require customer_card_ref matching the target's authorization record \u2014 absent/mismatched/empty/non-string all refuse CUSTOMER_BINDING_MISMATCH with zero probes and refusal-only state change; correct ref succeeds; ordering preserved (unknown target -> TARGET_NOT_AUTHORIZED, unknown run -> RUN_NOT_FOUND, both before binding). Residual scope, as the README itself discloses: this is an identifier match, NOT authentication \u2014 the card ref is not a secret; hosted/multi-tenant auth beyond stdio remains not built.",
  "OBSERVATION B-RT-5 (low): re-registration is keyed by a deterministic ref over (card, receipt, twin) only. A second, validly signed authorization record for the same triple with a DIFFERENT named authorizer and DIFFERENT expiry returns already_registered with the FIRST record's authorizer/expiry \u2014 later authorization records cannot supersede or narrow an earlier one, and the presented record is silently not the governing one.",
  "RE-BENCH SUITE MAINTENANCE (bench-side only, no check weakened): valid rt_run_suite/rt_report call shapes now present customer_card_ref=herald-card-fixture-customer per the B-RT-4 contract change (a _report helper injects it on valid calls; omission/mismatch/non-string shapes are explicit probes); U2 required sets updated to the amended surface; R4 expanded to depth >=3 in dicts AND lists, signed and unsigned, on every verb; R10 expanded to byte-identical snap purity at register/run/report layers plus certified/certification/safety-verdict plants and a clean-flow false-positive leg; R17 rewritten as the binding battery incl. ordering legs. R15's fuzz count rose 64 -> 74 because the amended schemas declare the new customer_card_ref parameters.",
  "Run 1 scorecard retained append-only: its R7 'untestable' was a bench-suite defect (an over-broad source-scan regex matched the word 'requests ' in catalog prose) and its R17 'pass' was a bench-suite defect (the customer-B client was pointed at a fresh EMPTY state file instead of customer A's shared store, so B trivially saw nothing). Both fixed suite-side; run 2 is the standing card. No product code was touched between runs.",
  "RE-BENCH: run 2 card (FAILED 23/27) retained append-only; this run is the standing card after the Silas fix round for B-RT-1..B-RT-4. B-RT-5 (re-registration cannot supersede the first authorization record for a card/receipt/twin triple) was NOT in the fix round and remains open as a low observation \u2014 product behavior unchanged on that path.",
  "N-CHECK mechanism (one line): Redteam's distinguishing act is that probe construction is downstream of a signed, in-date authorization record plus an arena/harness registration proof \u2014 with neither, no probe object is constructed at all, which an eval framework (promptfoo/Giskard shape: model-level pass/fail grades over a free-form target list, no target-authorization concept) cannot express, import, or render here (N1)."
 ],
 "history": "append-only; prior runs retained in results/"
}