{
 "tool": "eclipse-umbra-mcp",
 "version": "0.1.0",
 "bench_date": "2026-10-25",
 "bench_version": "0.1.0",
 "environment": "local",
 "qualifier": "local",
 "verdict": "CERTIFIED(local)",
 "checks": [
  {
   "id": "B1",
   "name": "similarity recall: bench lookalike battery (rn/m, 0/O domain, mixed-script, 1/I/l) flagged with skeleton evidence",
   "verdict": "pass",
   "evidence": "7/7 spec-class lookalikes flagged skeleton_equal with cited confusions: Bumham (rn/m fusion of Burnham), n0rthwind (0-for-o in the DOMAIN \u2014 confused char cited in confused_chars_domain), H\u0430lden (Cyrillic \u0430 cited in the name), Signa1s (1-for-l), SignaIs (I-for-l), SiIverline + SilverIine (I-for-l). OBSERVATION (non-failing): the punycode-shaped domain was NOT flagged \u2014 no IDNA decoding in the skeleton engine; see scorecard notes"
  },
  {
   "id": "B2",
   "name": "precision: honest near-names, genuine listings and another org's lookalike stay silent",
   "verdict": "pass",
   "evidence": "scan of the honest snapshot (6 genuine watched listings + 4 honest near-names sharing generic words / industry / substrings: Northwind Traders, Halden Group, Meridian Analytics, Quantum Relay Labs) yields ZERO candidates; in the attack snapshot the unchanged and endpoint-only-changed genuine listings and Gatep0st (a lookalike of an org NOT on this watchlist) stay silent. The key-rotated Meridian listing DOES surface as a candidate \u2014 name+domain skeleton-equal with ZERO confused chars: identity continuity is key-sensitive by design, since a rotated key is indistinguishable from a copied name+domain (the lapsed-domain silhouette); candidates are evidence, not verdicts"
  },
  {
   "id": "B3",
   "name": "case trap both directions: I-for-l is skeleton-equal + cited; i-to-l is NOT skeleton-equal, no confusion cited",
   "verdict": "pass",
   "evidence": "direction 1 (capital I where the watched name has lowercase l): SiIverline, SilverIine and SignaIs are skeleton-EQUAL with the confusion cited per position (in_subject 'l', in_candidate 'I') \u2014 the CopperIine class holds from the wire. Direction 2 (lowercase l where the watched name has lowercase i): Sllverline is NOT skeleton-equal, cites ZERO confused chars, and surfaces only as near_distance=1 evidence \u2014 i is not silently folded into l; the case-aware distinction is intact"
  },
  {
   "id": "B4",
   "name": "diff integrity: same-host path change, key rotation alone, remove + re-add",
   "verdict": "pass",
   "evidence": "same-host path change (api.halden.example /v1 -> /v2): endpoint_changed with the changes list naming ONLY the endpoint field (host untouched); key rotation with no endpoint change: key_changed only, and its diff finding issues CHANGED \u2014 the closed vocabulary has no key-specific state \u2014 with key_ids the sole change in the pack; lst-b-gone appears in removed for s1->s2 and in added for s2->s3; both snapshot hashes ride every diff"
  },
  {
   "id": "B5",
   "name": "evidence-pack law: header-only / missing / absent refused by name; single-snapshot claims downgrade honestly",
   "verdict": "pass",
   "evidence": "header-only snapshot, missing snapshot id, and absent listing each refuse EVIDENCE_PACK_INCOMPLETE by name (no -32603); a change-claim built on ONE snapshot compared with itself downgrades honestly to CONSISTENT with an empty change set \u2014 it does not manufacture a CHANGED; a plain single-snapshot observation of a healthy listing issues UNRESOLVED, the honest default"
  },
  {
   "id": "B6",
   "name": "rebuttal: original byte-unchanged, DISPUTED supersedes, subject-presented never merged",
   "verdict": "pass",
   "evidence": "rebuttal filed: the DISPUTED version supersedes, the original finding's canonical hash is byte-identical before and after in the state file, and the rebuttal renders as a separate linked record with provenance source 'subject-presented' / data_rights 'subject-provided' \u2014 its text is NOT merged into the finding's claims (and its banned word is quarantined hash-only at render, stored as data in the ledger)"
  },
  {
   "id": "B7",
   "name": "lint in new carriers: registry name field, endpoint URL string, rebuttal text quarantined hash-only",
   "verdict": "pass",
   "evidence": "three NEW lint carriers quarantined hash-only at render: (1) the registry NAME field itself ('bench-fraud-watch') is replaced by a quarantine marker in the import receipt and every downstream render; (2) a listing NAME carrying 'Verified Agent Partners' is quarantined in the diff added[] render; (3) an endpoint URL string containing 'verified agent' is quarantined in the watchlist echo. The banned phrases appear in NO output; markers carry class + sha256 only"
  },
  {
   "id": "B8",
   "name": "findings key: wrong key + tampered body fail; C1 key distinct from Herald RSA root",
   "verdict": "pass",
   "evidence": "finding verifies under the C1 findings public block (ECDSA P-256, key_id umbra-findings-*); a bench-generated wrong key fails, and a tampered body (state flipped) fails under the RIGHT key. Distinctness from the Herald root is structural: Herald's root is an RSA JWK (kty RSA, kid herald-488ece4a) in a different key file \u2014 a different key, algorithm family, and id namespace; a findings-key compromise cannot mint Herald identity"
  },
  {
   "id": "B9",
   "name": "T3 denylist enumerated + bench additions; surface is exactly the 11 spec tools",
   "verdict": "pass",
   "evidence": "tools/list is exactly the 11 spec tools (and matches the manifest); all 32 forbidden names \u2014 the builder's 16 T3 enforcement names plus 16 bench additions (remove_listing, mark_verified, trust_score, contact_registry, ban, block, freeze, escalate...) \u2014 refused -32602 by direct call. No delist, block, takedown, notify or scoring verb exists"
  },
  {
   "id": "B10",
   "name": "similarity record recomputed exactly from its carried algorithm + claim refs",
   "verdict": "pass",
   "evidence": "the wire record for Bumham recomputes EXACTLY from its carried algorithm (umbra-similarity/1.0) + both refs: skeleton equality, both distances, both confused-char lists and the signal all match an independent re-run of the published algorithm, and the record's candidate_claim_sha256 equals sha256 of the canonical raw listing the bench submitted \u2014 a subject can reproduce the evidence bit-for-bit"
  },
  {
   "id": "N1",
   "name": "novelty: distinguishing test vs a generic registry-diff tool (signed closed-vocabulary findings + reproducible packs + rebuttal linkage, not diffs)",
   "verdict": "pass",
   "evidence": "NOVELTY_PROVEN: mechanism = a closed-vocabulary witness \u2014 signed findings with reproducible evidence packs and rebuttal linkage, not diffs. Distinguishing test: to a generic registry-diff tool, Bumham is merely an 'added' row (proven: diff_run's added[] carries no signature, no state, no pack). Umbra's similarity finding over the same snapshot pair emits a DIVERGED finding in the closed vocabulary, signed under the C1 findings key (verified offline), backed by a pack whose snapshot hash and hash-pinned raw payload recompute exactly, and \u2014 per B6 \u2014 accepts a subject rebuttal that flips it to DISPUTED while the original stays byte-unchanged. A diff tool emits none of these four artifacts"
  },
  {
   "id": "U1",
   "name": "auth enforced",
   "verdict": "na",
   "evidence": "local stdio transport has no credential layer to test \u2014 process ownership is the boundary (the standing U1 note); the spec's tiers are declared in tool descriptions and enforced as surface shape, not auth"
  },
  {
   "id": "U2",
   "name": "schema valid",
   "verdict": "pass",
   "evidence": "tools/list: all 11 tools carry a name and an object inputSchema with properties/required"
  },
  {
   "id": "U3",
   "name": "no network surface (source-level)",
   "verdict": "pass",
   "evidence": "source scan of server.py + umbra_engine.py + findings_sig.py: zero network/process call sites (no urllib.request/urlopen/socket/requests/http.client/subprocess); the only urllib use is urlparse \u2014 string parsing of endpoint fields, no egress. Snapshot import is the only input seam, as specced"
  },
  {
   "id": "U4",
   "name": "no secret leak",
   "verdict": "pass",
   "evidence": "secret scan over all bench outputs + the bench state file: no credential patterns; the findings PRIVATE scalar never appears in any output or state (verification surfaces carry the public block only); the key file is chmod 600 and keys/ is gitignored"
  },
  {
   "id": "U5",
   "name": "sane errors",
   "verdict": "pass",
   "evidence": "unknown finding -> FINDING_NOT_FOUND; list-typed id -> INVALID_ID_TYPE; non-string registry_id -> INVALID_ID_TYPE; missing required argument -> -32602 transport schema error; zero -32603 and zero tracebacks across every bench probe"
  },
  {
   "id": "U6",
   "name": "process health",
   "verdict": "pass",
   "evidence": "ping answers; initialize latency 46 ms recorded at setup"
  },
  {
   "id": "U7",
   "name": "audit completeness (receipt chain + counts)",
   "verdict": "pass",
   "evidence": "fresh ledger: import + watchlist + import + finding + rebuttal produce exactly 5 receipts in order (snapshot_imported, watchlist_set, snapshot_imported, finding_issued, rebuttal_filed); receipt_list itself emits none; the hash chain verifies end to end"
  },
  {
   "id": "U8",
   "name": "bench isolation",
   "verdict": "pass",
   "evidence": "workspace product state byte-identical before/after bench"
  }
 ],
 "notes": [
  "Local scope: v0 imports snapshots only \u2014 all probes ran against bench-owned registry snapshots + watchlist in /tmp; no registry was contacted and no live crawl exists in the product (G-LIVE per registry is not_tested, carried verbatim in the manifest).",
  "U1 is na: local stdio transport \u2014 process ownership is the boundary; no credential layer exists to test.",
  "Non-failing observation (B1): a punycode-shaped lookalike domain (xn--...) whose NAME resembles nothing on the watchlist is NOT flagged \u2014 the skeleton engine does no IDNA decoding, so an encoded homoglyph domain is invisible unless the name betrays it. Punycode is outside the spec ch.7 fixture classes (capital-I/l, 0/O, rn/m, unicode confusables); IDNA-decode-before-skeleton is recommended before G-LIVE opens for any registry.",
  "Key-rotation finding state (B4): the closed vocabulary has no key-specific state \u2014 a key-only change issues CHANGED with the key_ids change named in the pack diff. That is the honest rendering, recorded here so the vocabulary gap is a decision, not a surprise.",
  "Run 1 PARTIAL card retained append-only: its B2 untestable was a bench-suite assertion defect \u2014 the suite assumed a key-rotated genuine listing stays silent in similarity_scan, but the engine's same-subject test is name+domain+KEYS, so a rotated key breaks identity continuity and the listing surfaces as a zero-confusion candidate. That is the designed witness posture (a rotated key is indistinguishable from a copied name+domain \u2014 the lapsed-domain silhouette; candidates are evidence, not verdicts). Suite fixed to assert the behavior explicitly; run 2 is the standing card. No product code touched."
 ],
 "history": "append-only; prior runs retained in results/"
}