test(v2): final benchmark campaign - real end-to-end characterization (EXP-0014)
Post-M10, user-requested final benchmark campaign: 6 realistic workloads (16-256 independent neurons in a shared-input dense-layer shape, a random-seeded 2-layer network with real cross-node PSRAM forwarding, and a 6-node 2-hop dependency diamond) x 4 concurrency levels (N_SLOTS=1/2/4/8) through the real, full neural_multiprocessor system (real V1 PSRAM chain, real slot_mem_arbiter). 24/24 runs PASS bit-exact against a software golden model (11,520 individual neuron/ node checks, zero mismatches). Three real bugs found and fixed during the campaign itself (ERR-0009): 1. neural_director.v (M5) had a real RTL bug at N_SLOTS=1 ($clog2(1)=0 makes a replication expression illegal) - never caught because M5-M10 only ever tested N_SLOTS=2/4/8. Fixed with a width-agnostic '0 literal; M5's own testbench re-verified unaffected. 2/3. Two testbench sizing bugs in tb_benchmark_suite.v itself (psram_model DEPTH too small for the Large workload's address range; N_NODES too small for the Stress workload's node-id range, causing a real deadlock via node-id wraparound colliding with an already-DISPATCHED node - a real, honest consequence of DEC-0008's own "no node-slot reclamation" design choice). Headline finding: real parallel scaling is essentially flat beyond N_SLOTS=2 - the single shared PSRAM port saturates at ~91% utilization regardless of slot count, so memory-bound workloads gain only 1.05-1.06x real speedup from N=1 to N=8. Once real POST-P&R Fmax degradation is also factored in, N_SLOTS=4 is measurably 21% SLOWER in real wall-clock time than N_SLOTS=1 for the largest workload tested. N_SLOTS=2 is recommended as the default (DEC-0014, superseding DEC-0012's resource-only "N_SLOTS=8 ceiling" framing for general use). Full 21-section report (every number classified THEORETICAL/ SIMULATED/POST-P&R MEASURED/DERIVED, per the user's own methodology requirements): hardware/v2/docs/benchmarks/ final-benchmark.md Logged: simulation/synthesis/timing/benchmark/decisions (DEC-0014)/ experiments (EXP-0014)/errors (ERR-0009)/development.log, ROADMAP.md updated. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013xXuuRUWZScuo1DeYJxs3v
This commit is contained in:
@@ -775,3 +775,85 @@ choices, not a mandate to re-synthesize already-logged results).
|
||||
STATUS:
|
||||
ACCEPTED
|
||||
|
||||
DEC-0014
|
||||
|
||||
DATE: 2026-09-05
|
||||
|
||||
DECISION:
|
||||
N_SLOTS=2 is the recommended default/shipped configuration for
|
||||
neural_multiprocessor.v, superseding DEC-0012's earlier "N_SLOTS=8 is
|
||||
the practical ceiling" framing for general use. N_SLOTS=8 remains a
|
||||
REAL, valid, synthesizable configuration (DEC-0012's DSP-budget
|
||||
ceiling finding stands), but the final benchmark campaign (EXP-0014)
|
||||
shows it is not a good DEFAULT given the system's real bottleneck.
|
||||
|
||||
WHY:
|
||||
EXP-0014's real, measured parallel-scaling data (6 workloads x 4
|
||||
configs, real V1 PSRAM chain, real slot_mem_arbiter, real POST-P&R
|
||||
Fmax) shows conclusively that the shared PSRAM port -- not slot count
|
||||
-- is this system's real bottleneck: memory-bound workloads
|
||||
(C-Large/D-Stress) get only 1.05-1.06x real cycle-count speedup from
|
||||
N_SLOTS=1 all the way to N_SLOTS=8 (PSRAM port utilization pegged at
|
||||
~91% regardless), and once real Fmax degradation from added slots is
|
||||
also factored in (152.46 -> 142.45 -> 113.38 MHz for N=1/2/4), the
|
||||
REAL WALL-CLOCK time for the Stress workload is actually 21% WORSE at
|
||||
N_SLOTS=4 than at N_SLOTS=1. More hardware parallelism made this
|
||||
workload class slower, not faster -- adding slots has a real Fmax cost
|
||||
with no compensating real throughput benefit once the shared PSRAM
|
||||
port saturates.
|
||||
|
||||
Small/bursty workloads (A-Small, E-Multilayer, F-DAG) DO show a real,
|
||||
if modest, benefit from N_SLOTS=2 (~1.2-1.3x real wall-clock speedup,
|
||||
from better overlap of per-job registration/scheduling latency across
|
||||
two independently-progressing jobs) -- this benefit already exists at
|
||||
N=2 and does not meaningfully grow at N=4/8 (see EXP-0014's efficiency
|
||||
table: efficiency collapses from 66% at N=2 to 15% at N=8 for exactly
|
||||
this workload class). N_SLOTS=2 is therefore the point that captures
|
||||
essentially all of the real, measured benefit this architecture can
|
||||
deliver from concurrency, without paying N=4/8's real Fmax tax for a
|
||||
benefit that does not materialize.
|
||||
|
||||
EVIDENCE:
|
||||
benchmark.log's EXP-0014 entry: the full 6-workload x 4-config real
|
||||
cycle-count table, the derived speedup/efficiency table, the real
|
||||
wall-clock (cycles / real POST-P&R Fmax) comparison for D-Stress, and
|
||||
the real per-slot tile-delivery imbalance data (slot 0/1 doing ~98% of
|
||||
C-Large's real work at N_SLOTS=4, slot 2/3 essentially idle until the
|
||||
tail) -- all real, Verilator-simulated + nextpnr-ecp5-measured, not
|
||||
assumed.
|
||||
|
||||
ALTERNATIVES:
|
||||
1. Recommend N_SLOTS=8 (DEC-0012's original framing, "practical DSP
|
||||
ceiling"). Rejected as a DEFAULT: DEC-0012 was correct that
|
||||
N_SLOTS=8 is the largest configuration that FITS the chip's DSP
|
||||
budget, but EXP-0014 shows fitting is not the same as being
|
||||
beneficial -- 8 slots deliver essentially the same real throughput
|
||||
as 1 slot for memory-bound work, at a real Fmax cost (92.63 MHz for
|
||||
dataflow_core-only, even lower once the real PSRAM chain is added).
|
||||
N_SLOTS=8 remains available/valid for a FUTURE system that also
|
||||
widens real memory bandwidth (see Alternative 2 below and the final
|
||||
report's Bottleneck Analysis/Limitations sections) but is not the
|
||||
right choice for THIS system as built.
|
||||
2. Solve the real bottleneck (widen/parallelize PSRAM bandwidth --
|
||||
e.g. multiple physical PSRAM banks, one per pair of slots) so that
|
||||
N_SLOTS=4/8 would actually deliver real throughput gains. Rejected
|
||||
for THIS decision: real hardware/board redesign, well beyond a
|
||||
measurement-driven RTL parameter choice -- flagged as the correct
|
||||
FUTURE direction if higher real concurrency is ever needed, not
|
||||
attempted here (§30: no invented results, no un-measured redesigns
|
||||
presented as decided).
|
||||
|
||||
RESULT:
|
||||
N_SLOTS=2 is the recommended default configuration, used as the
|
||||
reference configuration in the final benchmark report and (pending
|
||||
user confirmation) the datasheet. N_SLOTS=1 remains a real,
|
||||
competitive alternative for deployments that are purely large/
|
||||
sustained/memory-bound (equal or better real wall-clock throughput,
|
||||
lower resource cost, highest real Fmax). N_SLOTS=4/8 remain valid,
|
||||
synthesizable, functionally-correct configurations (all bit-exact
|
||||
verified in EXP-0014) but are NOT recommended as a default without a
|
||||
future memory-bandwidth-scaling architecture change.
|
||||
|
||||
STATUS:
|
||||
ACCEPTED
|
||||
|
||||
|
||||
Reference in New Issue
Block a user