test(v2): final benchmark campaign - real end-to-end characterization (EXP-0014)

Post-M10, user-requested final benchmark campaign: 6 realistic
workloads (16-256 independent neurons in a shared-input dense-layer
shape, a random-seeded 2-layer network with real cross-node PSRAM
forwarding, and a 6-node 2-hop dependency diamond) x 4 concurrency
levels (N_SLOTS=1/2/4/8) through the real, full neural_multiprocessor
system (real V1 PSRAM chain, real slot_mem_arbiter). 24/24 runs PASS
bit-exact against a software golden model (11,520 individual neuron/
node checks, zero mismatches).

Three real bugs found and fixed during the campaign itself (ERR-0009):
1. neural_director.v (M5) had a real RTL bug at N_SLOTS=1
   ($clog2(1)=0 makes a replication expression illegal) - never
   caught because M5-M10 only ever tested N_SLOTS=2/4/8. Fixed with
   a width-agnostic '0 literal; M5's own testbench re-verified
   unaffected.
2/3. Two testbench sizing bugs in tb_benchmark_suite.v itself
   (psram_model DEPTH too small for the Large workload's address
   range; N_NODES too small for the Stress workload's node-id range,
   causing a real deadlock via node-id wraparound colliding with an
   already-DISPATCHED node - a real, honest consequence of DEC-0008's
   own "no node-slot reclamation" design choice).

Headline finding: real parallel scaling is essentially flat beyond
N_SLOTS=2 - the single shared PSRAM port saturates at ~91%
utilization regardless of slot count, so memory-bound workloads gain
only 1.05-1.06x real speedup from N=1 to N=8. Once real POST-P&R Fmax
degradation is also factored in, N_SLOTS=4 is measurably 21% SLOWER
in real wall-clock time than N_SLOTS=1 for the largest workload
tested. N_SLOTS=2 is recommended as the default (DEC-0014,
superseding DEC-0012's resource-only "N_SLOTS=8 ceiling" framing for
general use).

Full 21-section report (every number classified THEORETICAL/
SIMULATED/POST-P&R MEASURED/DERIVED, per the user's own
methodology requirements): hardware/v2/docs/benchmarks/
final-benchmark.md

Logged: simulation/synthesis/timing/benchmark/decisions (DEC-0014)/
experiments (EXP-0014)/errors (ERR-0009)/development.log,
ROADMAP.md updated.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013xXuuRUWZScuo1DeYJxs3v
This commit is contained in:
2026-09-05 19:58:30 +02:00
co-authored by Claude Sonnet 5
parent 91bbbe2fe5
commit 3cdaeaee35
12 changed files with 1369 additions and 4 deletions
+73
View File
@@ -254,3 +254,76 @@ VERIFICATION: hardware/v2/sim/tb_neural_multiprocessor.v -- 4/4 PASS
entirely contained inside the new arbiter module.
STATUS: FIXED, verified end-to-end with the real (unmodified) V1
PSRAM backend chain and real concurrent multi-slot contention.
ERR-0009 (one real RTL bug + two testbench bugs, all FOUND AND FIXED
during the post-M10 final benchmark campaign -- hardware/v2/sim/
tb_benchmark_suite.v)
DATE: 2026-09-05
1. REAL RTL BUG in hardware/v2/rtl/neural_director.v (M5, previously
committed/synthesized, never before exercised at N_SLOTS=1):
SYMPTOM: Verilator compile error building neural_multiprocessor at
N_SLOTS=1 -- "%Error-ZEROREPL: Replication value of 0 is only
legal under a concatenation" at three sites.
ROOT CAUSE: at N_SLOTS=1, $clog2(1)=0, making
`{$clog2(N_SLOTS){1'b0}}` a ZERO-width replication (illegal
outside a concatenation, IEEE 1800 11.4.12.1). Every prior
milestone (M5-M10) only ever built/simulated/synthesized
neural_director at N_SLOTS=2/4/8 -- N_SLOTS=1 was never actually
exercised until this benchmark campaign asked for it as the
baseline for parallel-scaling measurement.
FIX: replaced the three `{$clog2(N_SLOTS){1'b0}}` reset/default
expressions with the width-agnostic `'0` literal, which self-sizes
correctly for any width including 0. No functional change for
N_SLOTS>1 (same reset value).
VERIFICATION: hardware/v2/sim/tb_neural_director.v (M5's own
testbench, N_SLOTS=2) re-run unchanged -- still 4/4 PASS. All 6
benchmark-suite workloads then verified bit-exact at N_SLOTS=1
through the real full neural_multiprocessor + real V1 PSRAM chain.
STATUS: FIXED, verified at both N_SLOTS=1 (newly working) and
N_SLOTS=2 (no regression).
2. TESTBENCH BUG (tb_benchmark_suite.v, not RTL): psram_model's own
DEPTH parameter (524288... originally 131072 words = 256KB) was
smaller than the byte address range some workloads actually use
(workload C-Large's own result region alone needs word address
~0x24000, beyond a 131072-word/0x20000 DEPTH) -- a silent
out-of-bounds array access, same bug CLASS already documented once
before (an M5 testbench bug, sim_byte_mem's too-small DEPTH).
SYMPTOM: every C-Large neuron read back real=0 (poison value never
overwritten) despite the RTL reporting all 128 jobs completed.
FIX: DEPTH raised to 524288 words (1MB byte-addressable), computed
to safely exceed the highest byte address used by any of the six
workloads' regions (~0xB2006, workload F).
VERIFICATION: C-Large re-run bit-exact PASS after the fix.
3. TESTBENCH BUG (tb_benchmark_suite.v, not RTL): N_NODES=512 was
smaller than the highest node_id actually used -- workload D-Stress
(node_base=400, 256 neurons) reaches node_id 655, which silently
WRAPS at the 9-bit node_id width (512 -> truncates to 0), colliding
with workload A's node_id 0, already permanently ST_DISPATCHED (M6's
own design never reclaims dispatched node slots, decisions.log
DEC-0008). register_node's blocking `while(!reg_ready)` wait then
deadlocks forever (reg_ready never returns true for an already-
occupied, non-EMPTY node_id).
SYMPTOM: simulation appeared to hang indefinitely partway through
D-Stress's node registration (confirmed via periodic progress
`$display` instrumentation added specifically to localize this --
registration silently stopped advancing past neuron index 112).
FIX: N_NODES raised to 1024, comfortably exceeding every workload's
own node_id range.
VERIFICATION: D-Stress re-run bit-exact PASS (256/256 neurons) after
the fix, no further hangs at any N_SLOTS configuration (1/2/4/8).
NOTE: this is a REAL, honest consequence of DEC-0008's own design
choice (no node-slot reclamation) -- a long-running system that
keeps registering new nodes without ever reusing old (DISPATCHED)
ids will eventually exhaust its node_id space and deadlock exactly
this way. Flagged in the final benchmark report's Limitations
section, not just fixed and forgotten.
DIAGNOSIS METHOD (bug 3): periodic `$display` progress heartbeats
added to the registration loop and the completion watchdog loop,
run under `stdbuf -oL` to force line-buffered (not block-buffered)
output for real-time visibility, isolating the exact neuron index
where progress stopped advancing -- the same "trace real signals,
don't guess" discipline used throughout this whole project.