{"bench_date":"2026-10-25","bench_version":"0.5.0-scoped-rebench-4","checks":[{"evidence":"live stdio (fresh state file, isolated copy): tools/list = exactly the 16 spec-10 names; tools/call payroll_run_submit -> {ok:false,error:'unknown tool'}; surface unchanged by fix round 4","id":"SURF","name":"surface diff vs spec 10: LIVE stdio tools/list == exactly the 16 specced tools","verdict":"pass"},{"evidence":"isolated copy at /tmp/wageward-bench5/work: builder suite 103/103 unittest OK (claim of 103 verified, grew from 94); self_check.py PASS (a/b/c); workspace tree sha256-identical before/after bench (12 source files, __pycache__ excluded per bench-3/-4 method)","id":"GEN","name":"builder suite independently re-run on isolated copy; workspace integrity","verdict":"pass"},{"evidence":"N-check STANDS PASS (per scoped instructions; not re-derived). Fix round 4 altered only the digit-joining rule (all separator-run/group/span caps removed) plus a field-grammar path for period-valued leaves in guards — mechanism shape unchanged, no CRUD verb added, read verbs remain 3/16","id":"NOV","name":"N-check: fused mechanism vs Gusto per-call confirmation","verdict":"pass"},{"evidence":"129,956 chars swept (tenant-alpha, all 16 tools exercised via dispatch, outputs + full state dump incl. candidates/fingerprints/packets/ledger/packs/flags): SSN-pattern hits=0, planted raw SSN/BANK/BANK_NEW/FUNDING value hits=0, normalized-digit hits=0 after stripping legit >=32-hex token material, raw-hash hits=0 (md5/sha1/sha256/sha512 of raw+normalized+class-prefixed constructions, every alpha employee/kind). Spot scope per instructions (one tenant); hostile planting is SEP/GRAM below","id":"H5-SPOT","name":"H5 spot regression: full-surface sweep on one tenant + persisted-state check, zero hits","verdict":"pass"},{"evidence":"CLOSED — defect D2-GEN2 (HIGH, bench -4) is fixed on every channel bench -4 condemned. All 7 bench-4 shapes byte-for-byte via LIVE stdio access_ledger_read period_range -> DATA_MINIMIZATION_REFUSED, value nowhere in response, persisted live state file clean (shapes absent; SSN digits absent after stripping legit >=32-hex token material): 975....298....326, 975....29....8326, 975....298326, 975____29____8326, four-space 975    29    8326, four-dash 975----29----8326, run-5 975.....29.....8326 (7/7). Persistence shapes 975....29....8326 + 975....298326 in employer_adjustments[].note -> DATA_MINIMIZATION_REFUSED via dispatch AND direct engine call, candidates=={} both paths (2/2). Signing shapes (both persistence shapes + 975....298....326 + run-5) as Herald approval nonces -> GuardError DATA_MINIMIZATION_REFUSED at signing, approvals=={} (4/4). Controls prove the paths themselves work: plain prose note persists (candidates=1), legit nonce signs (approvals=1). Fixture digits are alpha EMP-101's real synthetic SSN 975-29-8326, re-derived independently from synthetic.derive_identifier","id":"SEP-RUN4","name":"SEP-RUN4 byte-for-byte (bench -4 defect D2-GEN2): separator runs >=4 via LIVE stdio + persistence + signing paths","verdict":"pass"},{"evidence":"No surviving cap found (8/8 shapes x 3 paths): separator runs of length 8 (975........298326 and double-run 975........29........8326), 12 (975............298326), 20 (975....................298326); 5-group split 97-52-98-32-6; 9-group split 9-7-5-2-9-8-3-2-6; span-40 shape (975 + 31 dots + 298326, exactly 40 chars). Every shape -> DATA_MINIMIZATION_REFUSED with no echo via LIVE stdio period_range, via dispatch AND direct engine in adjustment notes (candidates=={}), and at Herald signing as approval nonce (approvals=={}). The uncapped joining rule is real: no run-length, group-count, or span ceiling remains reachable","id":"SEP-NOCAP","name":"no-new-cap probes: run-8/run-12/run-20, 5-group + 9-group splits, span-40 — echo/persist/sign all refuse","verdict":"pass"},{"evidence":"RULING: OBSERVATION, not a defect (check passes; no H5 hit scored). '975....x298326' passes the screen on all three paths — echoed verbatim as period_range via live stdio (ok:true), persists in an adjustment note (candidates=1), signs as an approval nonce (approvals=1). The letter 'x' breaks the digit window by the fix's stated rule (letters ALWAYS break a window — the same rule the FP battery depends on: 'T' breaks ISO timestamps); the value reaches each artifact only as two separate fragments of 3 and 6 digits, neither identifier-shaped alone, with no contiguous or separator-joined >=9-digit substring anywhere (contrast bench -2 E4: x975298326x was a hit because the 9 digits were CONTIGUOUS inside the wrap). Ruled per the split-across-fields precedent (bench -2 H5V, retained since): fragments that co-persist as separate fields/fragments — '975-29' + '8326' across adjustment entries — were recorded as observation, not a hit, because reassembly requires planter knowledge plus an out-of-band join across a boundary the screen lawfully treats as a break. Same standard applied within one leaf across a letter boundary. Recorded so a future bench can re-rule if the standard changes; it does not block certification under the standing precedent","id":"SEP-MIXED","name":"mixed shape ruling: '975....x298326' (letter breaks the window; 3+6 fragments)","verdict":"pass"},{"evidence":"period_range '2026-10-01..975298326' via LIVE stdio -> DATA_MINIMIZATION_REFUSED, no echo. Route verified at guards level: _is_valid_period_range=False (second segment is not a date/timestamp), so the leaf falls through to the full uncapped screen, whose joined window totals 17 digits. The grammar accepts only real date/timestamp segments — an identifier segment can never ride it","id":"GRAM-A","name":"grammar attack (a): SSN digits as a fake period segment ('2026-10-01..975298326')","verdict":"pass"},{"evidence":"period_range '2026-10-01....2026-10-31' via LIVE stdio -> DATA_MINIMIZATION_REFUSED, no echo. Route: split on '..' yields 3 parts (empty middle) -> grammar-invalid -> full uncapped screen joins 16 digits -> refuse. A 4-dot join cannot smuggle a range past the strict '..' grammar","id":"GRAM-B","name":"grammar attack (b): 4-dot join ('2026-10-01....2026-10-31')","verdict":"pass"},{"evidence":"period_range '9999-99-99..8888-88-88' via LIVE stdio -> DATA_MINIMIZATION_REFUSED, no echo. Route: date.fromisoformat rejects month/day 99/88 -> segments not real calendar dates -> grammar-invalid -> full screen refuses (16 joined digits). Shape alone (\\d{4}-\\d{2}-\\d{2}) does not satisfy the grammar; calendar validity is enforced","id":"GRAM-C","name":"grammar attack (c): invalid calendar dates ('9999-99-99..8888-88-88')","verdict":"pass"},{"evidence":"period_range '2026-10-25T12:00:00.975298326Z' via LIVE stdio -> DATA_MINIMIZATION_REFUSED, no echo. Route proven at guards level, and it is the PER-SEGMENT screen that catches it (not a grammar rejection): _is_valid_period_range=True (the segment parses as a tz-carrying timestamp) but identifier_shaped(segment)=True — the post-'T' window joins 12:00:00.975298326 across ':' and '.' separators to 15 digits >= 9 -> refused inside the grammar path exactly as scoped. A grammar-valid segment carrying a 9-digit run cannot pass","id":"GRAM-D","name":"grammar attack (d): timestamp segment with 9-digit fractional-second run as period_range","verdict":"pass"},{"evidence":"Name/path alone exempts nothing (3/3): period_range='975....298326' nested inside employer_adjustments[] extras via payroll_run_prepare -> DATA_MINIMIZATION_REFUSED, candidates=={}, state clean; period_range='975298326' nested in a change_request_open free area (verification_note) -> DATA_MINIMIZATION_REFUSED, change_packets=={}; pay_period='975....298326' nested in an extras bag via access_ledger_read -> DATA_MINIMIZATION_REFUSED. Every identifier-carrying period-named leaf fails the strict grammar and falls through to the full uncapped screen. Nuance recorded, judged no-hit: the grammar treatment is leaf-NAME-based at any depth, so a genuinely valid range string nested under a period_range/pay_period name in an odd context passes the screen — it carries no identifier, each segment is independently screened, and no value class the H5 sweep protects can ride it; the security property (no identifier exempted by name/path) holds","id":"GRAM-E","name":"grammar attack (e): period-named leaf in non-period contexts (adjustment extras / change-packet free areas) carrying an identifier","verdict":"pass"},{"evidence":"Builder claim verified (2/2): the legit range string '2026-10-01..2026-10-31' in a FREE field (employer_adjustments[].note) via payroll_run_prepare -> DATA_MINIMIZATION_REFUSED, candidates=={}, state clean — the raw uncapped rule applies (16 joined digits) because 'note' is not a period-valued leaf; the same string as a dict KEY in access_ledger_read args -> DATA_MINIMIZATION_REFUSED (keys get no grammar treatment, per guards law). Grammar relief exists only on period_range/pay_period value leaves","id":"GRAM-F","name":"grammar scope (f): legit range string in a free field (note) or as a dict key — raw rule applies","verdict":"pass"},{"evidence":"Real canonical payload pay_period='2026-10-16..2026-10-31' (draft range, system-generated) verifies end to end: pin_verify -> VALID via dispatch, via standalone function, and via standalone CLI (rc=0, verdict VALID, signature/scope/expiry all true); drift path with current_canonical_payload == payload via dispatch -> VALID, diff_classes=[], drift_detected=false. The grammar path legitimizes exactly the product's own period values without weakening the screen (see GRAM-A..F)","id":"GRAM-LEGIT","name":"grammar legit side (g): pay_period in real canonical payloads still verifies — pin_verify VALID (dispatch + standalone + CLI + drift)","verdict":"pass"},{"evidence":"Item-by-item, all pass: ISO timestamp as at_time (roster read ok:true) and as period_range '2026-10-25T12:00:00Z' echoed verbatim via live stdio; single date '2026-10-25' echoed verbatim; BOTH period ranges echoed verbatim — '2026-10-01..2026-10-31' and draft range '2026-10-16..2026-10-31'; hist range '2026-09-16..2026-09-30' echoed verbatim; prose period_range 'October payroll period' ok:true; genuine time '12:34:56' ok:true echoed (colon-joined window totals 6 < 9; contrast colon-SSN refused in bench -4 SEP-FRESH); amounts all prepare with candidates created — float 3000.00, string '3000.00', string '123456.78', int 12345678, string '12345678' (5/5); pin_verify VALID (see GRAM-LEGIT); adjusted-run SUBMITTED (see D4). Two fail-closed OBSERVATIONS recorded, outside the mandated battery and not scored: a fractional-second timestamp '2026-10-25T12:00:00.123Z' and an offset timestamp '2026-10-25T12:00:00+00:00' are identifier-shaped under the uncapped join (time window totals >= 9) and would refuse as period_range — the product emits whole-second 'Z' timestamps only (guards.format_time), and probe GRAM-D mandates refusing the 9-digit-fraction form, which is the same mechanism; noted for the spec author, not a defect against this scope","id":"FP","name":"FP re-verify item-by-item: timestamps, dates, period ranges, hist range, prose, genuine time, amounts, pin_verify, adjusted run","verdict":"pass"},{"evidence":"Unchanged, confirmed not reopened: '123,456,789' refuses BY NAME (DATA_MINIMIZATION_REFUSED, no echo) via LIVE stdio period_range and as adjustment gross_amount (candidates=={} afterward). Identical behavior to bench -4, where the tradeoff was ruled ACCEPTABLE fail-closed cost under spec 18.2 zero tolerance; this bench confirms the fix round did not silently reopen or widen it","id":"TRADEOFF","name":"FP tradeoff stands: comma-grouped '123,456,789' still refuses by name (accepted tradeoff, bench -4 ruling)","verdict":"pass"},{"evidence":"spot-check holds: approval_id as list and as dict -> APPROVAL_SCOPE_MISMATCH by name at simulated_run_submit and require_verified_approval (4/4, zero TypeError). Full D1 matrix not re-run per scope (closed in re-bench -3)","id":"RAIL","name":"D1 spot-check only (closed in re-bench -3): unhashable approval_id","verdict":"pass"},{"evidence":"spot-check holds: herald_sign_approval_simulated for approver_identity 'mallory-outsider' with valid role -> APPROVAL_SCOPE_MISMATCH, approvals=={} afterward. Full D3 matrix not re-run per scope (closed in re-bench -2)","id":"APRV","name":"D3 spot-check only (closed in re-bench -2): mallory-outsider at signing","verdict":"pass"},{"evidence":"spot-check holds (also an FP battery item): candidate prepared WITH employer_adjustments (EMP-102 gross 3000.00), approved at adjusted fingerprint -> SUBMITTED with readback_verified=true. Not re-derived beyond the spot-check per scope","id":"D4","name":"D4 spot-check only (closed in re-bench -2): adjusted run submits","verdict":"pass"},{"evidence":"lifecycle spot holds: packet opens OPENED, channel-of-record receipt issues, release after cooling-off by an independent human completes COMPLETED_IN_SOR. SPEC-GAP CARRY-FORWARD (not a defect, per instructions): receipt TTL still unimplemented after fix round 4 — mandate bank_change_policy keys remain {cooling_off_hours, verification_channel_class, near_payday_window_days, near_payday_heightened_review, near_payday_cooling_off_multiplier} with no TTL key, and behaviorally a release ~120 days post-verification (verified 2026-10-25, released 2027-02-22) still completes COMPLETED_IN_SOR. Carried as an open spec item for the owner/spec author, exactly as in benches -1/-2/-3/-4","id":"BANK","name":"bank-change lifecycle spot + receipt TTL carry-forward","verdict":"pass"}],"environment":"local","history":[{"bench_version":"0.1.0","note":"2 HIGH (D2 free-text ChangePacket channels, D1 RAIL TypeError) + 1 MEDIUM (D3 approver identity unbound) + LOW D4 phantom drift on adjusted runs + LOW CLI traceback on missing payload","scorecard":"eclipse-wageward-mcp-2026-10-25.json","verdict":"FAILED"},{"bench_version":"0.2.0-rebench","note":"D3 + D4 closed; D1 narrowed to MEDIUM residual (unhashable approval_id TypeError; totals-alias diff loss); D2 narrowed but still open (JSON-number and dot-separated identifier forms reach output/persisted state); CLI narrowed (malformed approval sub-object still tracebacks)","scorecard":"eclipse-wageward-mcp-2026-10-25-2.json","verdict":"FAILED"},{"bench_version":"0.3.0-scoped-rebench-2","note":"Scoped re-bench of fix round 2: D1 + CLI closed in full; D2 core residuals (E1-E4) closed; D3/D4 spot-checks hold; D2 still open at HIGH via separator generalization (underscore/slash et al. echo + persist) and the global hex-token exemption (crafted hex leaf embedding SSN digits echoes + persists) — H5 zero tolerance fails certification","scorecard":"eclipse-wageward-mcp-2026-10-25-3.json","verdict":"FAILED"},{"bench_version":"0.4.0-scoped-rebench-3","note":"Scoped re-bench of fix round 3: bench-3 separator shapes (10/10) + fresh-symbol generalization (12/12) closed on echo/persist/nonce channels; hex exemption correctly path-scoped (crafts refused in free fields, spoofs + dict keys refused, legit pin_verify VALID dispatch+CLI+drift); comma-grouped tradeoff ruled ACCEPTABLE fail-closed (non-blocking). D2 remains OPEN at HIGH via one surviving channel: separator runs of length >=4 echo verbatim via live stdio, persist in candidates, and sign+persist as approval nonces — ruled INSIDE the H5 threat model, not an acceptable boundary. H5 zero tolerance fails certification","scorecard":"eclipse-wageward-mcp-2026-10-25-4.json","verdict":"FAILED"}],"notes":["VERDICT CERTIFIED(local): 20/20 scoped checks pass. The scoped retest (SEP-RUN4 + FP re-verify) is clean, and per the bench instructions everything outside this scope was already closed in -2/-3/-4 with the comma-grouped tradeoff ruled acceptable — so a clean scope here certifies the product, and it lands: Eclipse Wageward P0 is CERTIFIED at the spec section 1 ceiling for P0 (simulator/local). Bench only — nothing fixed, committed, or published.","SEP-RUN4 CLOSE-OUT: defect D2-GEN2 (HIGH, bench -4) is CLOSED. Fix round 4 removed all three joining caps (separator run / group count / span) rather than raising them, and every bench-4 shape refuses byte-for-byte on the live-stdio echo channel (7/7), the adjustment-note persistence channel via dispatch and direct engine (2/2, candidates=={}), and the approval-nonce signing channel (4/4, approvals=={}), with the persisted live state file clean. Fresh no-cap probes (run-8/12/20, 5-group, 9-group, span-40) refuse on all three paths (8/8 shapes) — no successor cap was found. D2 is therefore closed in full: original channels (-1), JSON-number/dot/letter-wrap residuals (-2), separator generalization + hex exemption (-3), run>=4 (-4) all sealed.","GRAMMAR PATH (new surface from fix 4) — all probes pass: (a) identifier as fake segment refuses (grammar rejects segment, full screen joins 17); (b) 4-dot join refuses (3-part split is grammar-invalid); (c) invalid calendar dates refuse (calendar validity enforced, not shape); (d) 9-digit fractional-second timestamp refuses VIA THE PER-SEGMENT SCREEN inside the grammar path (segment is grammar-valid; its joined window totals 15) — the scoped attack surface behaves exactly as designed; (e) period-named leaves in non-period contexts refuse identifier payloads on all three probed nests — name/path alone exempts nothing; (f) the legit range string in a free note field and as a dict key refuses under the raw rule, as the builder claimed; (g) the product's own pay_period range verifies — pin_verify VALID via dispatch, standalone function, standalone CLI (rc=0), and the drift path (diff_classes=[], drift_detected=false).","FP BATTERY: every mandated item re-verified individually and passing (see FP check): ISO timestamp, single date, both period ranges + hist range echoed verbatim, prose, genuine 12:34:56 time, all five amount shapes incl. '12345678'/'123456.78', pin_verify VALID, adjusted-run SUBMITTED; comma-grouped '123,456,789' still refuses by name — tradeoff unchanged, not reopened. Two fail-closed observations (fractional-second and offset timestamps are identifier-shaped under the uncapped join) are recorded in the FP check: outside the mandated battery, product timestamps are whole-second 'Z' only, and the GRAM-D mandate is the same mechanism — noted for the spec author, not scored.","RULINGS RECORDED THIS BENCH: (1) SEP-MIXED '975....x298326' = OBSERVATION, not a defect — letter-broken 3+6 fragments reach artifacts as separate fragments only; ruled under the bench -2 split-across-fields precedent (fragments co-persisting across a lawful break = observation, no contiguous/joined >=9 window = no H5 hit). (2) Grammar treatment is leaf-name-based at any depth (GRAM-E nuance) — no identifier can ride it because acceptance requires strict calendar-valid date/timestamp segments, each screened per segment. (3) N-check stands PASS (stated per scope; fix round 4 touched screening parameters + one grammar path only). D1/D3/D4 spot-checks hold; H5 spot sweep (one tenant, full surface + persisted state) zero hits; surface exactly 16.","SPEC-GAP CARRY-FORWARD (not scored as a product defect): receipt TTL — confirmed still unimplemented in fix round 4 (no mandate key, no engine logic, ~120-day-post-verification release completes). Open spec item carried from benches -1/-2/-3/-4 for the owner/spec author; by instruction it does not block certification.","Builder claims independently verified this round: builder suite is now 103 tests (was 94), all passing on an isolated copy; self_check PASS; joining rule uncapped (run/group/span caps deleted — confirmed behaviorally to run-20, 9 groups, span-40, not by code reading alone); period grammar strict (calendar-valid ISO date | tz timestamp | two segments joined by '..') with per-segment screening; keys and free fields get no grammar relief. Workspace tree byte-identical before/after the bench."],"qualifier":"local","tool":"eclipse-wageward-mcp","verdict":"CERTIFIED","version":"0.1.0"}