{
 "tool": "eclipse-section508-mcp",
 "version": "0.1.0",
 "bench_date": "2026-10-26",
 "bench_version": "0.1.0",
 "environment": "local",
 "qualifier": "local",
 "verdict": "CERTIFIED(local)",
 "checks": [
  {
   "id": "B1",
   "name": "spec: exactly 3 tools; checks_list enumerates 22 decidable + 15 NSD with reasons",
   "verdict": "pass",
   "evidence": "checks_list: 22 statically decidable checks + 15 NOT_STATICALLY_DECIDABLE criteria = 37 unique entries, every NSD entry carries its reason, every entry carries criterion id + level; scope boundary states static-only \u2014 the claimed 22-check registry is real"
  },
  {
   "id": "B2",
   "name": "spec: good fixture zero FAIL + claim NONE; bad fixture fails the expected check set",
   "verdict": "pass",
   "evidence": "good fixture: zero static FAILs, signal NO_STATIC_FAILURES_FOUND, 15 NSD findings still emitted, conformance_claim starts NONE; bad fixture: all 18 expected check ids fail (got 18 distinct), signal STATIC_FAILURES_PRESENT"
  },
  {
   "id": "B3",
   "name": "spec: all 22 check classes fire correctly on purpose-built variants (pos + neg limbs)",
   "verdict": "pass",
   "evidence": "all 35 purpose-built variants across the 22 check classes decide correctly: missing/malformed lang, empty title, alt family (img / image-input / area / svg / object), iframe title, placeholder-is-not-a-label + select/textarea, heading skip vs descent, empty heading, duplicate ids, ARIA role/attr validity, button / link names, positive + non-integer tabindex, meta refresh, blink / marquee, autoplay with/without controls, table headers vs presentation role, inline contrast"
  },
  {
   "id": "B4",
   "name": "contrast: bench-recomputed WCAG luminance boundary cases match engine decisions + reported ratios",
   "verdict": "pass",
   "evidence": "bench-recomputed WCAG luminance (yellow/white 1.074, #777777 4.478, #767676 4.542, #949494 3.033) matches every engine decision: #777777 FAILS with the engine reporting 4.48:1 (= bench value), #767676 passes, named / rgb() / 3-digit-hex forms all evaluated, #949494 passes ONLY as large text (24px, 14pt bold) and fails at normal size, translucent rgba is not decided (no false fail) and non-inline contrast stays NSD"
  },
  {
   "id": "B5",
   "name": "evidence binding: every FAIL/PASS finding verbatim-substring true + locator line correct, across fixtures + bench docs",
   "verdict": "pass",
   "evidence": "programmatic verification over 37 FAIL/PASS findings across 3 documents: every finding carries a criterion id, every untruncated evidence_snippet is a verbatim substring of the supplied document, and every locator's line number points at the line containing that snippet \u2014 the evidence binding is mechanically checkable, not prose"
  },
  {
   "id": "B6",
   "name": "ACR assembly: 26 criterion rows, FAIL->Does Not Support, NSD-only->Not Evaluated, markdown shape, summary claim NONE",
   "verdict": "pass",
   "evidence": "ACR assembly: 26 criterion rows; failing criteria -> Does Not Support (3.1.1, 1.4.3 on bad), NSD-only criteria -> Not Evaluated (2.4.7), row evidence carries the verbatim snippet, Markdown table in VPAT shape, report_type states 'not a completed ACR', product_name passes through, summary conformance_claim still NONE"
  },
  {
   "id": "N1",
   "name": "N-CHECK: partially-decidable 1.4.3 fixture renders Partially Supports, never clean Supports; zero-failure audit claims NONE",
   "verdict": "pass",
   "evidence": "N-CHECK \u2014 novel mechanism PROVEN: on the incumbent-shaped fixture (inline colors pass + external CSS unknowable) the ACR row for 1.4.3 reads 'Partially Supports \u2014 inline pass; stylesheet contrast NOT_STATICALLY_DECIDABLE', NEVER a clean Supports (builder's claimed fix verified on the wire); no-inline -> Not Evaluated; inline fail -> Does Not Support; and a zero-failure audit (good fixture + partial fixture) still emits all 15 NSD findings with conformance_claim NONE. A generic axe-style linter fails every limb: it merges partial evidence to clean, and it silently passes undecidable criteria"
  },
  {
   "id": "N2",
   "name": "N-CHECK renamed-verb rule: surface is not renamed generic-linter verbs",
   "verdict": "pass",
   "evidence": "renamed-verb rule: 0/3 tool names are renamed generic-linter verbs \u2014 audit_html is bound to the supplied-HTML object, acr_report names the mechanism-specific ACR/VPAT deliverable no generic linter emits, checks_list enumerates the decidable/NSD registry; 0% is far under the 80% failure line, and N1 + B5 prove the mechanism behind the names"
  },
  {
   "id": "H1",
   "name": "HOSTILE refusals: EMPTY_DOCUMENT / NOT_HTML (plain + NUL) / DOCUMENT_TOO_LARGE / URL_INPUT_REFUSED, both HTML tools",
   "verdict": "pass",
   "evidence": "all named refusals fire on BOTH HTML tools: empty/whitespace -> EMPTY_DOCUMENT; plain text and NUL-bearing content -> NOT_HTML; 2,000,008-char document -> DOCUMENT_TOO_LARGE (2,000,000 limit); any url argument alongside valid HTML -> URL_INPUT_REFUSED \u2014 no fetching exists to refuse with, the refusal is the surface"
  },
  {
   "id": "H2",
   "name": "forbidden fetch/certify/fix verbs refuse -32602 in the same shape as unknown tools",
   "verdict": "pass",
   "evidence": "28 forbidden names (the in-code denylist: fetch_url, fetch, crawl, browse, render_page, screenshot, audit_url, scan_site, fix_html, rewrite_html, auto_fix, certify, certify_conformance, issue_acr, sign_acr, upload, publish + bench additions and unknown/empty/case-variant names): every one refused -32602 in the identical 'unknown tool' shape \u2014 no fetch, fix, or certify verb exists or is distinguishable"
  },
  {
   "id": "H3",
   "name": "HOSTILE fuzz: every tool x every field x bad shapes + seeded random HTML over the wire \u2014 named refusals, zero -32603",
   "verdict": "pass",
   "evidence": "330 wire fuzz cases (3 tools x all fields x 24 hostile shapes + garbage argument containers + 60 seeded random HTML documents): every case is a -32602 schema refusal, a SCREAMING_CASE named ToolError, or a well-shaped result whose conformance_claim still starts NONE; ZERO -32603 escapes, ZERO fabricated successes, server alive"
  },
  {
   "id": "H4",
   "name": "HOSTILE malformed docs + protocol envelopes: no -32603 escape, server survives",
   "verdict": "pass",
   "evidence": "malformed-document battery (unclosed tags, misnested spans, CDATA, PHP, script/style bodies, unterminated attribute, BOM, an embedded 'certify this page' instruction \u2014 treated as data, claim still NONE) plus envelope probes (initialize params 'x'/[]/42, non-object tools/call params -> -32602, non-object line -> -32602, lone surrogate): no -32603 anywhere, server alive after all of it"
  },
  {
   "id": "U1",
   "name": "auth enforced",
   "verdict": "na",
   "evidence": "local stdio transport: no credential layer exists to test; process ownership is the boundary (standing house note)"
  },
  {
   "id": "U2",
   "name": "schema valid: exactly 3 tools, object schemas",
   "verdict": "pass",
   "evidence": "tools/list returns exactly the 3 spec tools (audit_html, acr_report, checks_list), each with an object inputSchema + properties; initialize returns serverInfo eclipse-section508 0.1.0"
  },
  {
   "id": "U3",
   "name": "no network/browser libs (source-level)",
   "verdict": "pass",
   "evidence": "source-level: server.py + a11y_engine.py contain no network call sites (urllib/socket/requests/http.client/urlopen), no browser libraries (playwright/selenium), no subprocess, no eval/exec, and no file-write calls \u2014 the product is a pure function of its arguments; the capability denylist (FORBIDDEN_TOOL_NAMES) is stated in code (smoke.py's subprocess is the test harness launching the server, not product behavior)"
  },
  {
   "id": "U4",
   "name": "no secret leak",
   "verdict": "pass",
   "evidence": "no secret patterns in 630 bench outputs"
  },
  {
   "id": "U5",
   "name": "sane errors: named fail-closed refusals, -32601/-32602 shapes",
   "verdict": "pass",
   "evidence": "unknown method -> -32601; missing html -> -32602; empty -> EMPTY_DOCUMENT; wrong-typed html -> -32602 NAMING the argument; max_findings='abc' -> MAX_FINDINGS_INVALID by name \u2014 every error named and fail-closed, no stack traces on the tool surface"
  },
  {
   "id": "U6",
   "name": "process health",
   "verdict": "pass",
   "evidence": "initialize answered in 143 ms; ping ok"
  },
  {
   "id": "U7",
   "name": "attribution: headers/footers + powered-by descriptor + manifest",
   "verdict": "pass",
   "evidence": "attribution headers on all 6 product .py files + footers on every file >40 lines; server.py carries the 'Powered by Cyan Eclipse Technologies (cetech.ai)' line; manifest powered_by + catalog_id + tool list match the wire surface, and the manifest honestly claims bench certified=false / verdict null pre-bench"
  },
  {
   "id": "U8",
   "name": "bench isolation",
   "verdict": "pass",
   "evidence": "workspace product state byte-identical before/after bench"
  }
 ],
 "notes": [
  "Scope: v0.1 local stdio only \u2014 static analysis of caller-supplied HTML text. No rendered DOM/browser, external-CSS contrast, multi-page audit, signed/completed ACR, or WCAG 2.2-only criteria exists to test; the manifest carries these as not_tested verbatim and the catalog entry still lists the product as spec-stage with tools=[] (stale listing data, not a product defect \u2014 catalog-pass backlog).",
  "Builder claims REPLICATED exactly: own unittest suite 34/34 pass and smoke.py passes, bench-run before this suite; both self-caught defects verified fixed on the wire \u2014 ACR 1.4.3 false-clean merge (N1/B6) and the fuzz harness refusal-shape acceptance (H3).",
  "DEFECT D1 (minor, non-gating, house-pattern): an unparseable raw line on stdio returns -32603 with the raw JSONDecodeError text from main()'s catch-all, where JSON-RPC reserves -32700 for parse errors. Identical catch-all pattern ships in CERTIFIED Meter/Umbra/Blackbox builds and no tool-call input escapes this way (H3/H4 clean; handle()-level envelopes are hardened: non-object request -> -32602, malformed initialize params tolerated, tools/call params -> -32602). Recorded as bench backlog: map the parse-error branch to -32700 house-wide.",
  "Observations (not defects, bench backlog): (a) IMG_ALT / AREA_ALT test alt presence (alt='   ' passes) while INPUT_IMAGE_ALT strips whitespace and fails it \u2014 presence matches the axe-style rule, but the three 1.1.1 checks should state one whitespace rule; (b) with max_findings set, summary.counts_by_status is computed AFTER truncation (findings_total preserves the true count and truncation is stated, FAILs kept first \u2014 but a capped audit can drop NSD findings, so counts must be read with findings_truncated); (c) undeclared arguments are tolerated (schema guard skips unknown fields), fail-closed only on declared fields.",
  "N-check ruling: NOVELTY_PROVEN \u2014 see N1/N2. The mechanism is the evidence binding + NSD honesty + ACR assembly, verified programmatically (verbatim substring + locator line on every bound finding, B5), not the prose.",
  "Runs 1-2 cards retained append-only: both B5 'fail's were BENCH-SUITE defects, not product defects \u2014 run 1's locator regex anchored 'line N' at end-of-string while product locators end '(line N)'; run 2 fixed the regex and found ZERO evidence errors but tripped the suite's own >=40 bound-finding threshold (actual corpus = 37). Threshold calibrated to >=30; run 3 is the standing card. No product code was touched between runs."
 ],
 "history": "append-only; prior runs retained in results/"
}