{"bench_date":"2026-10-25","bench_version":"0.1.0-rebench-4","checks":[{"evidence":"8/8 KEY_MATERIAL_REFUSED (openai_style_key): (1) wrap.sealed_envelope.ciphertext, (2) wrap[0].sealed_envelope.ciphertext, (3) wrap.sealed_envelope.nonce, (4) deep.deeper.sealed_envelope.tag, (5) top-level sealed_envelope.ciphertext on lease_status (tool takes no envelope), (6) sealed_envelope.ciphertext as dict {inner: canon}, (7) owner_approval.signature as dict {inner: canon}, (8) owner_approval.signature as list [canon]. Plus card-note variant: b64(canon) under wrap.sealed_envelope.ciphertext -> KEY_MATERIAL_REFUSED (base64_encoded_key_material). D7 CLOSED — stays closed. Fix is structural as queued: identity-anchored exemption in server._collect_strings + scan_for_key_material dispatch.","id":"K3-D7","name":"D7 BYTE-FOR-BYTE repros from card -3 (canon 'sk-'+29x'V', tools/call lease_status) — all must refuse KEY_MATERIAL_REFUSED","verdict":"pass"},{"evidence":"3/3 KEY_MATERIAL_REFUSED via tools/call lease_status, args JSON round-tripped (parser form): (1) single key \"owner_approval.signature\": canon -> refused (openai_style_key); (2) single key \"owner_approval.params_hash\": 'c2stVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlY=' -> refused (base64_encoded_key_material); (3) single key \"owner_approval.target_ref\": same blob -> refused (base64_encoded_key_material). Segment-tuple paths hold: one dotted segment never equals a two-segment exempt/identifier path. D8 CLOSED — stays closed.","id":"K3-D8","name":"D8 BYTE-FOR-BYTE repros from card -3 (dotted-key path forgery) — all must refuse KEY_MATERIAL_REFUSED","verdict":"pass"},{"evidence":"After all 12 D7/D8 probes: canon and b64(canon) absent from every response blob, absent from temp state.json and vault.json; domain state secrets==[] leases==[] (scan refusals wrote refusal receipts only, carrying tool+code+arg_path+pattern — no values).","id":"K3-D78-INERT","name":"D7/D8 inertness: evaded-class payloads never echo, never store","verdict":"pass"},{"evidence":"WIRE (binding surface, all args JSON round-tripped = parser-admitted, distinct objects): mirror copy of the real envelope with ciphertext replaced by canon as an extra arg on a valid secret_enroll -> KEY_MATERIAL_REFUSED; finite self-nesting {\"sealed_envelope\":{\"sealed_envelope\":{\"sealed_envelope\":{\"ciphertext\":canon}}}} -> KEY_MATERIAL_REFUSED; list self-nesting wrap[[[{sealed_envelope.ciphertext: canon}]]] -> KEY_MATERIAL_REFUSED. DIRECT-API (informational, wire-unreachable): passing the SAME Python dict object as both sealed_envelope and a mirror arg makes scan_for_key_material return None for that object's ciphertext at both occurrences (identity check is position-blind once the node is retrieved from the top-level arg); the identical content JSON round-tripped (distinct objects) refuses (scan hit at $.mirror...). A self-referential dict/list passed directly raises RecursionError in _collect_strings (no cycle/depth guard). RULED accepted observations, not defects: MCP transport is stdio JSON — json.loads builds a tree (no aliasing, no cycles, duplicate keys impossible), so neither shape is reachable by any MCP caller; the exempt node is retrieved FROM the top-level argument position at dispatch, and the engine stores only canonical_envelope(top-level arg), so no aliased occurrence can reach storage. See notes for the exact bound + optional hardening.","id":"K3-IDENTITY-A","name":"NEW ANGLE (a) aliasing / self-nesting of the real envelope node","verdict":"pass"},{"evidence":"sealed_envelope=[{...envelope, ciphertext: canon}] -> KEY_MATERIAL_REFUSED; direct scan hit at $.sealed_envelope[0].ciphertext (openai_style_key) — dispatch requires isinstance dict, list wrapper gets no exempt node. sealed_envelope=[real valid envelope] -> KEY_MATERIAL_REFUSED (encoded_key_material at $.sealed_envelope[0].ephemeral_pubkey_pem, full-force scan), NOT enrolled: list contents are ordinary content, fail-closed, never exemption-treated.","id":"K3-IDENTITY-B","name":"NEW ANGLE (b) sealed_envelope as a LIST containing the envelope on secret_enroll — identity must NOT exempt list contents","verdict":"pass"},{"evidence":"3/3 KEY_MATERIAL_REFUSED via lease_status: owner_approval=[canon] (path (owner_approval,0)); owner_approval=[{signature: canon}] (path (owner_approval,0,signature)); owner_approval=[real ECDSA signature string from a valid approval] -> refused (encoded_key_material) at the list position — the real signature is exempt ONLY as a string at the exact (owner_approval, signature) tuple; in a list it scans at full force (fail-closed).","id":"K3-IDENTITY-C","name":"NEW ANGLE (c) owner_approval as a LIST with signature at [0] — tuple path differs, must not exempt","verdict":"pass"},{"evidence":"4/4 KEY_MATERIAL_REFUSED: {owner_approval:{signature: b64-string, note: canon}}; {owner_approval:{signature: b64-string, extra:{deep: canon}}}; valid secret_enroll whose owner_approval carries extra memo: canon (not re-signed) -> KEY_MATERIAL_REFUSED (transport scan precedes approval verification); {owner_approval:{signature: b64-string, params_hash: canon}} -> refused via surface pattern at the identifier path. The signature-value exemption exempts exactly one string and nothing else in the artifact.","id":"K3-IDENTITY-D","name":"NEW ANGLE (d) signature string at the real path + extra key material elsewhere in the approval object — extras scan at full force","verdict":"pass"},{"evidence":"RULING APPLIED: a top-level arg named sealed_envelope on secret_enroll IS the envelope position (one segment) — exemption reaches only its string-typed allowlisted ciphertext fields. Probes: valid enroll + top-level dotted key \"sealed_envelope.ciphertext\": canon -> KEY_MATERIAL_REFUSED (single segment (\"sealed_envelope.ciphertext\",) is not the envelope arg); valid enroll + dotted key \"owner_approval.signature\": canon -> KEY_MATERIAL_REFUSED. Inside the real envelope: extra field note='hello' (approval re-signed over mutated envelope) -> ENVELOPE_INVALID, nothing enrolled (D1 closed field set); extra field note=canon (re-signed) -> KEY_MATERIAL_REFUSED; ciphertext as LIST [canon] -> KEY_MATERIAL_REFUSED (container under exempt name recursed at full force); ciphertext as LIST [real b64 ciphertext] -> ENVELOPE_INVALID (not enrolled, no exemption for non-string field values). No S4 probe enrolled anything (secrets==[] after the whole identity section).","id":"K3-IDENTITY-E","name":"NEW ANGLE (e) dotted keys on secret_enroll + D1 ruling inside the REAL envelope (dispatch identity is by argument NAME at top level)","verdict":"pass"},{"evidence":"Fresh temp stores/keys, own secret 'verity4-static-key-<random hex>', ref verity-rb4-secret-1, agent agent_verity_rb4: enroll OK (IN CLAIM note, secret absent from output) — the real envelope's ciphertext fields and the real approval signature pass the scan at their exact positions; issue OK (khl_ token shown once); status active; list count 1; reference executor AND stub both decision=allow with capturing sink header X-API-Key == secret exactly; revoke OK (valid params-bound approval); post-revoke redemption refuses LEASE_REVOKED in BOTH executors; redemption_audit count 4, chain_valid=true; secret and lease token absent from all MCP outputs and from state.json+vault.json (hash-only at rest).","id":"K-LEGIT","name":"LEGITIMATE-EXEMPTION CONTROLS: valid enroll (real envelope + real signature) + full lifecycle, both executors","verdict":"pass"},{"evidence":"tools/list = exactly [secret_enroll, lease_issue, lease_status, lease_list, lease_revoke, redemption_audit]; every description carries 'credential leases, never raw keys' AND the CONDITIONAL PASS / G-TM note.","id":"K-LEGIT-TOOLSLIST","name":"tools/list surface","verdict":"pass"},{"evidence":"bazaar-manifest.json bench.not_tested states '78-test suite'; actual: grep 'def test_' tests/test_keyhold.py = 78; author suite independently re-executed by this bench this run: Ran 78 tests, OK. Claim TRUE (card -3's stale-count note is resolved). Fail-closed observation (non-scoring): a verbatim copy of a real envelope in a NON-envelope argument position refuses KEY_MATERIAL_REFUSED (encoded heuristic on ephemeral_pubkey_pem at full force) — no legitimate protocol input places an envelope outside $.sealed_envelope, and the legit lifecycle is unaffected; recorded so the behavior is known, not a defect. Cosmetic non-scoring note: README run-line still says '(73 tests)' — stale vs the 78-test suite; manifest is correct.","id":"K-LEGIT-MANIFEST","name":"MANIFEST COUNT: '78-test suite' claim vs actual suite size","verdict":"pass"},{"evidence":"4/4 KEY_MATERIAL_REFUSED via tools/call lease_status with fresh canon 'sk-'+26x'W': base64-encoded key in note (base64_encoded_key_material); PEM header split across fields at a fresh split point (a='-----BEGIN PRIVA', b='TE KEY-----', refused pem_private_key on the joined view); key split across fields (a='sk-'+13x'W', b=13x'W', joined view); Bearer+U+200B zero-width with fresh token body (bearer_value). Payloads absent from responses and from state/vault after.","id":"K3-SPOT","name":"SPOT evasion battery — 4 fresh Verity shapes (of the 16-shape K3 family), incl. b64 key + split PEM header","verdict":"pass"},{"evidence":"Own slug 'verity-rebench4-northwest-descriptive-static-credential-reference-for-quarterly-rotation' (88 chars): lease_status -> LEASE_NOT_FOUND, no KEY_MATERIAL_REFUSED, slug nowhere in response; lease_list agent filter count 0. 257-char ref ('v'x257): REFERENCE_TOO_LONG by name at lease_status lease_id, lease_list secret_ref, redemption_audit lease_id — KEY_MATERIAL_REFUSED in none. 256-char ref: LEASE_NOT_FOUND (bound inclusive, lookup reached). D6 stays closed.","id":"K3-D6-SPOT","name":"SPOT D6 ref battery: own slug NOT_FOUND + 256/257-char ref bounds","verdict":"pass"}],"environment":"local","history":"append-only; prior runs retained in results/ (eclipse-keyhold-mcp-2026-10-25.json = FAILED 11/14; eclipse-keyhold-mcp-2026-10-25-2.json = FAILED scoped, D6 found; eclipse-keyhold-mcp-2026-10-25-3.json = FAILED scoped, D7/D8 found)","notes":["VERDICT: CERTIFIED (local) first-party scope — D7 and D8 are verified CLOSED byte-for-byte, the identity-based exemption survives every new PARSER-ADMITTED angle (aliasing-by-copy, self-nesting, list-wrapped envelope, list approval, signature-plus-extras, dotted keys on secret_enroll), legitimate exemptions hold end-to-end (valid enroll + full lifecycle via both executors), K3/D6 spot batteries pass, and the manifest's 78-test count is exact. No new defects. Combined with retained prior cards (K1/K2/K4-K14 pass, D1-D6 closed), the first-party local scope reaches the CERTIFIED(local) ceiling set at card 1. Scope remains: fixture/capturing transports only, no real credential, no live third-party redemption; third-party custody still HELD in code.","ACCEPTED OBSERVATION O1 (direct-API aliasing, wire-unreachable, no fix required for certification): server.scan_for_key_material retrieves the exempt node via obj.get('sealed_envelope') on secret_enroll and _collect_strings exempts by object identity (obj is exempt_envelope_node) — a caller invoking the Python API directly (not via MCP/JSON) with the SAME dict object aliased into a second argument position gets that object's ciphertext fields skipped at the aliased position too, and a self-referential structure raises RecursionError (no depth guard). Neither is constructible over the product's only transport (MCP over stdio, JSON): json.loads never aliases and cannot encode cycles; every wire equivalent probed refuses. No storage or echo path exists for an aliased occurrence (engine canonicalizes only the top-level sealed_envelope argument; extra args are never stored). OPTIONAL hardening for Silas (non-blocking, Verity does not fix): anchor the identity check to the top-level position as well (exempt only when the node's path is ()-rooted at the sealed_envelope arg), and add a recursion-depth cap in _collect_strings returning a named refusal.","ACCEPTED OBSERVATION O2 (fail-closed full-force scan of envelope-shaped blobs outside the envelope position): a verbatim real envelope (or its ephemeral_pubkey_pem) in any non-envelope position refuses KEY_MATERIAL_REFUSED (encoded_key_material) — the whitespace-stripped PEM body decodes non-printably at >=64 chars. This is the D6 scoping working as designed (encoded heuristic at full force on content positions); no tool accepts an envelope anywhere else, and no legitimate input is refused — the valid lifecycle passes. Recorded for transparency; claim wording already scopes the exemption to the envelope's allowlisted fields at their position.","No fix queue. D1-D8 all closed; residuals from card -2 (hex-encoded / interleaved scans) and observations O1/O2 above stand as adjudicated, claim wording matches observed behavior.","Builder suite regression: author suite independently re-executed by this bench — 78 tests, OK (was 73 at card -3; +5 are the D7/D8 regression tests, all passing under the builder's own shapes; this bench's NEW angles (a)-(e) are outside that suite and pass here).","Prior cards retained unmodified (append-only): eclipse-keyhold-mcp-2026-10-25.json (md5 47b8dc26c6742816fb2310b79e12cefc), eclipse-keyhold-mcp-2026-10-25-2.json (md5 ace57020bef90830a8a19e86cb88dfde), eclipse-keyhold-mcp-2026-10-25-3.json (md5 f236cca456a595ebe80409b894b8efeb)."],"qualifier":"local","scope":"Scoped re-bench #3 (Verity W-003, audit only, NO fixes): boundary battery D7/D8 (byte-for-byte from card -3) + legitimate-exemption controls + NEW identity-exemption angles (aliasing/self-nesting, list-wrapped envelope, list owner_approval, signature-plus-extra-material, dotted keys on secret_enroll) + K3/D6 spot batteries + manifest count. Other lines stand (closed on prior cards): K1/K2/K4-K14 from eclipse-keyhold-mcp-2026-10-25.json, D6/K3-refval from eclipse-keyhold-mcp-2026-10-25-3.json. 59/59 bench probes pass. All probes in bench-owned temp state/vault/keys with Verity's own fresh strings (except D7/D8 repros, byte-for-byte per task); workspace product state untouched (product data/ still empty).","tool":"eclipse-keyhold-mcp","verdict":"CERTIFIED","version":"0.1.0"}