micheleandClaude Sonnet 5
3cdaeaee35
Post-M10, user-requested final benchmark campaign: 6 realistic workloads (16-256 independent neurons in a shared-input dense-layer shape, a random-seeded 2-layer network with real cross-node PSRAM forwarding, and a 6-node 2-hop dependency diamond) x 4 concurrency levels (N_SLOTS=1/2/4/8) through the real, full neural_multiprocessor system (real V1 PSRAM chain, real slot_mem_arbiter). 24/24 runs PASS bit-exact against a software golden model (11,520 individual neuron/ node checks, zero mismatches). Three real bugs found and fixed during the campaign itself (ERR-0009): 1. neural_director.v (M5) had a real RTL bug at N_SLOTS=1 ($clog2(1)=0 makes a replication expression illegal) - never caught because M5-M10 only ever tested N_SLOTS=2/4/8. Fixed with a width-agnostic '0 literal; M5's own testbench re-verified unaffected. 2/3. Two testbench sizing bugs in tb_benchmark_suite.v itself (psram_model DEPTH too small for the Large workload's address range; N_NODES too small for the Stress workload's node-id range, causing a real deadlock via node-id wraparound colliding with an already-DISPATCHED node - a real, honest consequence of DEC-0008's own "no node-slot reclamation" design choice). Headline finding: real parallel scaling is essentially flat beyond N_SLOTS=2 - the single shared PSRAM port saturates at ~91% utilization regardless of slot count, so memory-bound workloads gain only 1.05-1.06x real speedup from N=1 to N=8. Once real POST-P&R Fmax degradation is also factored in, N_SLOTS=4 is measurably 21% SLOWER in real wall-clock time than N_SLOTS=1 for the largest workload tested. N_SLOTS=2 is recommended as the default (DEC-0014, superseding DEC-0012's resource-only "N_SLOTS=8 ceiling" framing for general use). Full 21-section report (every number classified THEORETICAL/ SIMULATED/POST-P&R MEASURED/DERIVED, per the user's own methodology requirements): hardware/v2/docs/benchmarks/ final-benchmark.md Logged: simulation/synthesis/timing/benchmark/decisions (DEC-0014)/ experiments (EXP-0014)/errors (ERR-0009)/development.log, ROADMAP.md updated. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013xXuuRUWZScuo1DeYJxs3v
278 lines
16 KiB
Plaintext
278 lines
16 KiB
Plaintext
# V2 benchmark log -- solo append, mai troncato/sovrascritto (vedi README.md)
|
|
# Nessuna entry ancora -- popolato incrementalmente man mano che avanza lo sviluppo V2.
|
|
|
|
[2026-09-05] M1 single Neural Processor, isolated (no array/director/
|
|
memory manager yet -- system-level numbers deferred to M9)
|
|
|
|
| Config | Fmax (POST-P&R) | LUT | FF | DSP | BRAM |
|
|
|----------------------|------------------|-----|-----|-----|------|
|
|
| P_IN=8, ACC_WIDTH=32 | 183.12 MHz | 55 | 533 | 8 | 0 |
|
|
| P_IN=8, ACC_WIDTH=24 | 176.21 MHz | 49 | 509 | 8 | 0 |
|
|
|
|
Reference (V1, hardware/v1/synthesis/p8/, isolated neuron_parallel,
|
|
PARALLEL=8, N_INPUTS=256): 61.71 MHz POST-P&R.
|
|
|
|
All Fmax figures above are POST-P&R (real nextpnr-ecp5), not
|
|
theoretical or simulated-only. MAC/cycle, cycles/neuron, neurons/s,
|
|
stall %, effective MAC/s: not yet meaningful at this milestone (single
|
|
isolated processor, no streaming benchmark harness yet -- deferred to
|
|
M2 once neural_processor_array.v exists and a real workload can be
|
|
timed end-to-end).
|
|
|
|
[2026-09-05] M2 Neural Processor Array, N_PROCESSORS sweep (P_IN=8,
|
|
ACC_WIDTH=32 each; synthesized via the timing harness, see errors.log
|
|
ERR-0005 for why)
|
|
|
|
| N_PROCESSORS | Fmax (POST-P&R) | LUT | FF | DSP (MULT18X18D) | DSP % of 72 |
|
|
|--------------|------------------|-----|------|-------------------|-------------|
|
|
| 1 | 159.11 MHz | 59 | 409 | 8 | 11% |
|
|
| 2 | 149.59 MHz | 106 | 786 | 16 | 22% |
|
|
| 4 | 151.01 MHz | 207 | 1540 | 32 | 44% |
|
|
| 8 | 134.70 MHz | 374 | 3048 | 64 | 88% |
|
|
|
|
All figures POST-P&R (real nextpnr-ecp5), all PASS at the 80MHz
|
|
target. LUT4 utilization stays under 6% of the device even at N=8;
|
|
DSP is the binding resource (see decisions.log DEC-0005), reaching 88%
|
|
at N=8 -- N_PROCESSORS=9 would already exceed the LFE5U-45F's 72
|
|
MULT18X18D budget at P_IN=8. Theoretical MAC/cycle (THEORETICAL, not
|
|
yet measured end-to-end -- no real workload/benchmark harness exists
|
|
until M9): N_PROCESSORS * P_IN MACs/cycle when all processors are
|
|
simultaneously streaming tiles (8, 16, 32, 64 for N=1/2/4/8 -- verified
|
|
achievable in principle by EXP-0003's concurrent/staggered simulation,
|
|
not yet measured as a sustained throughput number).
|
|
|
|
[2026-09-05] M3 buffers -- BRAM cost vs DEPTH (real Yosys synth_ecp5)
|
|
|
|
| Module | DEPTH | DP16KD | LUT4 | FF |
|
|
|--------------------|-------|--------|------|-----|
|
|
| activation_buffer | 4096 | 2 | 37 | 30 |
|
|
| activation_buffer | 256 | 1 | 21 | 26 |
|
|
| weight_buffer | 512 | 2 | 88 | 139 |
|
|
| weight_buffer | 64 | 2 | 73 | 136 |
|
|
| result_buffer | 4096 | 2 | 37 | 30 |
|
|
| result_buffer | 256 | 1 | 21 | 26 |
|
|
|
|
weight_buffer's DP16KD count is flat across an 8x depth reduction --
|
|
its 64-bit TILE_WIDTH (P_IN=8 * DATA_WIDTH=8), not DEPTH, determines
|
|
BRAM count for this module. activation_buffer/result_buffer (byte-
|
|
wide) scale as expected with depth. Confirms §14's warning literally:
|
|
"non assumere che buffer piu' grandi siano automaticamente migliori"
|
|
-- here, smaller was not cheaper either, because depth was the wrong
|
|
lever for this specific buffer's cost.
|
|
|
|
[2026-09-05] M4 Memory Manager + Prefetch Engine (standalone resource
|
|
count; Fmax via timing harness -- see errors.log ERR-0005)
|
|
|
|
| Module | Fmax (POST-P&R) | LUT | FF | DSP | CCU2C |
|
|
|----------------------------------|------------------|-----|-----|-----|-------|
|
|
| memory_manager + prefetch_engine | 165.86 MHz | 851 | 789 | 0 | 108 |
|
|
|
|
End-to-end (real PSRAM + real neural_processor, SIMULATED only, no
|
|
system-level P&R yet -- deferred to M7/M9): 3-tile job = 446 cycles,
|
|
1-tile job = 166 cycles, 5-tile job = 728 cycles. ~140-150 cycles/tile,
|
|
dominated by psram_model.v's real ~70ns TAA access latency, not by
|
|
memory_manager's own control overhead (its bank-swap turnaround is
|
|
documented as a fixed +1 cycle/tile in decisions.log DEC-0006, a small
|
|
fraction of the ~140-cycle PSRAM-dominated total).
|
|
|
|
[2026-09-05] M5 Neural Director (standalone resource count; Fmax via
|
|
timing harness -- see errors.log ERR-0005)
|
|
|
|
| Module | Fmax (POST-P&R) | LUT | FF | DSP | CCU2C |
|
|
|------------------|------------------|-----|-----|-----|-------|
|
|
| neural_director (N_SLOTS=4) | 250.50 MHz | 382 | 366 | 0 | 4 |
|
|
|
|
[2026-09-05] M6 Dependency Manager (real standalone synthesis + P&R,
|
|
no harness needed)
|
|
|
|
| Module | Fmax (POST-P&R) | LUT | FF | DSP | CCU2C |
|
|
|----------------------|------------------|-----|-----|-----|-------|
|
|
| dependency_manager (N_NODES=16) | 155.30 MHz | 763 | 474 | 0 | 0 |
|
|
|
|
[2026-09-05] M7 Dataflow Core (full M1-M6 integration; resources via
|
|
real standalone synthesis, Fmax via timing harness -- see errors.log
|
|
ERR-0005)
|
|
|
|
| Module (config) | Fmax (POST-P&R) | LUT4 | CCU2C | FF | DSP | BRAM |
|
|
|-------------------------------|------------------|------|-------|------|-----|------|
|
|
| dataflow_core (N_SLOTS=2) | 165.15 MHz | 2127 | 248 | 2505 | 16 | 0 |
|
|
| dataflow_core (N_SLOTS=4) | 133.19 MHz | 3953 | 500 | 4688 | 32 | 0 |
|
|
|
|
DSP budget on the LFE5U-45F is 72 MULT18X18D total: N_SLOTS=4 already
|
|
uses 32/72 (44%), consistent with DEC-0005's finding that DSP, not
|
|
LUT/FF, is the first resource to saturate as concurrency grows (M2's
|
|
own N_PROCESSORS=8 measurement: 88%). BRAM=0 on both is expected --
|
|
M3's buffers are not wired into dataflow_core yet (DEC-0009).
|
|
|
|
[2026-09-05] M8 Neural Multiprocessor top (real standalone synthesis +
|
|
P&R, no harness needed -- real PSRAM pins keep the bare top-level
|
|
pin count at 157, under the TRELLIS_IO budget)
|
|
|
|
| Module (config) | Fmax (POST-P&R) | LUT4 | CCU2C | FF | DSP | BRAM |
|
|
|--------------------------------------|------------------|------|-------|------|-----|------|
|
|
| neural_multiprocessor (N_SLOTS=2) | 142.45 MHz | 3145 | 388 | 3659 | 16 | 0 |
|
|
|
|
Compare to M7's dataflow_core alone (N_SLOTS=2): 165.15 MHz / LUT4=2127
|
|
/ CCU2C=248 / FF=2505 / DSP=16. Adding the real V1 PSRAM chain +
|
|
slot_mem_arbiter costs ~1000 LUT4/140 CCU2C/1150 FF and drops Fmax by
|
|
~23 MHz (165.15 -> 142.45) -- both real, measured costs of real PSRAM
|
|
integration, not assumed.
|
|
|
|
[2026-09-05] M9 -- Confronto finale V1 vs V2 (docs/v2-description.md
|
|
§32). Basi di confronto: entrambi i sistemi COMPLETI (full-system,
|
|
non moduli isolati), stessa larghezza di dot-product per neurone
|
|
(PARALLEL=8 / P_IN=8), stesso backend PSRAM reale V1 non modificato
|
|
in entrambi i casi.
|
|
V1 = hardware/v1's spi_neuron_top (post_fix_verify synthesis,
|
|
PARALLEL=8) + neuron_memory_tb.v TEST 5 (1 neuron, N_INPUTS
|
|
ridotto a 8, catena PSRAM reale) -- entrambi dati gia'
|
|
certificati/congelati in hardware/v1/.
|
|
V2 = hardware/v2/rtl/neural_multiprocessor.v (N_SLOTS=2, P_IN=8,
|
|
M8) -- EXP-0009 (1 tile/8 input via memory_manager standalone,
|
|
EXP-0005) + EXP-0009 stesso (catena PSRAM reale condivisa).
|
|
|
|
| Metrica | V1 | V2 | Classificazione |
|
|
|------------------------|---------------------------|------------------------------|-------------------|
|
|
| Fmax | 68.65 MHz (FAIL @80MHz) | 142.45 MHz (PASS @80MHz) | POST-P&R (reale, nextpnr-ecp5, entrambi full-system) |
|
|
| LUT (Total LUT4s) | 8907 | 4191 | POST-P&R (nextpnr "Total LUT4s", stessa metrica per entrambi) |
|
|
| FF (Total DFFs) | 4900 | 3659 | POST-P&R |
|
|
| DSP (MULT18X18D) | 16 | 16 | POST-P&R |
|
|
| BRAM (DP16KD) | 2 | 0 (M3 non ancora collegato, DEC-0009) | POST-P&R |
|
|
| MAC/cycle (picco) | 8 (PARALLEL=8, core singolo sequenziale) | 16 (N_SLOTS=2 x P_IN=8, concorrenti) | THEORETICAL (parametri architetturali noti, non ancora un picco sostenuto misurato con contesa reale su entrambi gli slot) |
|
|
| cycles/neuron | 209 (1 neurone, 8 input reali, PSRAM reale -- hardware/v1/docs/FPGA-NeuralNetwork-Engine.md, TEST 5 neuron_memory_tb.v, gia' certificato) | 166 (1 tile/8 input, PSRAM reale -- EXP-0005) | SIMULATED (Verilator per V2; V1 dato gia' certificato in hardware/v1/, non ri-simulato in questa sessione) |
|
|
| neurons/s | 328,469 (Fmax POST-P&R / cycles SIMULATED) | 858,133 (idem) | DERIVATO (Fmax POST-P&R reale x cycles/neuron SIMULATED reale -- non un numero misurato direttamente in un'unica prova, ma calcolato da due misure reali indipendenti, entrambe citate) |
|
|
| stall % | NON MISURATO questo milestone | NON MISURATO questo milestone | -- vedi decisions.log DEC-0011 |
|
|
| memory utilization | NON MISURATO questo milestone | NON MISURATO questo milestone | -- vedi decisions.log DEC-0011 |
|
|
| processor utilization | NON MISURATO questo milestone | NON MISURATO questo milestone | -- vedi decisions.log DEC-0011 |
|
|
| effective MAC/s | 2.63 MMAC/s (8 MAC/neurone / 209 cicli x 68.65MHz) | 6.87 MMAC/s (8 MAC/neurone / 166 cicli x 142.45MHz) | DERIVATO (stessa base di neurons/s) |
|
|
| effective MAC/s (picco teorico) | 549.2 MMAC/s (8 x 68.65MHz) | 2279.2 MMAC/s (16 x 142.45MHz) | THEORETICAL |
|
|
|
|
Osservazioni (tutte da dati reali sopra, nessun numero inventato):
|
|
- V2 e' 2.6x piu' veloce in wall-clock per un singolo neurone/8 input
|
|
attraverso la catena PSRAM reale (1.165us vs 3.044us), pur usando lo
|
|
STESSO backend PSRAM V1 non modificato -- il guadagno viene sia da
|
|
meno cicli (166 vs 209, pipeline M1 piu' efficiente) sia da un Fmax
|
|
POST-P&R piu' che doppio (142.45 vs 68.65 MHz).
|
|
- V1's full system FALLISCE il target 80MHz (68.65 MHz); V2's full
|
|
system lo supera con margine (142.45 MHz) -- confermato da vero
|
|
nextpnr-ecp5 place&route su entrambi.
|
|
- Il vantaggio "MAC/cycle di picco" di V2 (16 vs 8) viene dalla
|
|
concorrenza reale a livello di sistema (N_SLOTS=2, EXP-0009 lo
|
|
dimostra con vera contesa PSRAM tra 2 slot), non da un core piu'
|
|
largo -- V1 e' strutturalmente un acceleratore sequenziale (un solo
|
|
neuron_parallel attivo alla volta), esattamente il limite che
|
|
l'intero mandato V2 (docs/v2-description.md, titolo) si propone di
|
|
superare.
|
|
- BRAM=0 per V2 riflette una scelta di scope esplicita (M3 non ancora
|
|
collegato, DEC-0009), non un vantaggio architetturale reale -- un
|
|
confronto onesto lo nota piuttosto che nasconderlo.
|
|
|
|
[2026-09-05] M10 -- Optimization, closing M9's deferred gaps with real
|
|
data (§32/DEC-0011 follow-up)
|
|
|
|
Stall %/utilization (EXP-0013, real SIMULATED data, N_SLOTS=2, 3-node
|
|
DAG scenario, replacing M9's "NOT MEASURED" placeholder for V2 -- V1's
|
|
own equivalent instrumentation was not built this session, still NOT
|
|
MEASURED for V1):
|
|
| Metric | V2 (SIMULATED, real) |
|
|
|------------------------------------|------------------------|
|
|
| shared PSRAM port utilization | 81.7% (18.3% idle) |
|
|
| slot 0 (memory_manager) utilization| 95.2% |
|
|
| slot 1 (memory_manager) utilization| 65.2% |
|
|
|
|
N_SLOTS sweep, extended to the real DSP ceiling (EXP-0008/EXP-0011):
|
|
| N_SLOTS | Fmax (POST-P&R) | DSP used/total |
|
|
|-----------|-------------------|------------------|
|
|
| 2 | 165.15 MHz | 16/72 (22%) |
|
|
| 4 | 133.19 MHz | 32/72 (44%) |
|
|
| 8 | 92.63 MHz | 64/72 (89%) |
|
|
All three PASS at 80MHz; N_SLOTS=8 recommended as the practical
|
|
ceiling for P_IN=8 on this chip (DEC-0012).
|
|
|
|
ACC_WIDTH 6-seed real placement sweep (EXP-0012, resolves EXP-0002's
|
|
single-seed inconclusiveness):
|
|
| ACC_WIDTH | mean Fmax | min | max | stdev |
|
|
|-------------|-------------|-------|-------|---------|
|
|
| 32 | 170.12 MHz | 145.73| 183.96| 14.16 |
|
|
| 24 | 180.71 MHz | 175.16| 185.49| 4.21 |
|
|
ACC_WIDTH=24 recommended as the new default (DEC-0013): higher mean
|
|
Fmax, much tighter variance, fewer resources, same bit-exact
|
|
correctness.
|
|
|
|
[2026-09-05] EXP-0014 -- FINAL BENCHMARK CAMPAIGN: real end-to-end
|
|
characterization, 6 workloads x 4 configs (N_SLOTS=1/2/4/8), real V1
|
|
PSRAM chain + real slot_mem_arbiter, full report at
|
|
hardware/v2/docs/benchmarks/final-benchmark.md. Summary below; see
|
|
that report for the complete 21-section breakdown.
|
|
|
|
Real cycle counts (SIMULATED, Verilator, bit-exact verified against a
|
|
software golden model for every single neuron/node):
|
|
|
|
| Workload | Neurons | N=1 cycles | N=2 cycles | N=4 cycles | N=8 cycles |
|
|
|-----------------|---------|------------|------------|------------|------------|
|
|
| A-Small | 16 | 3546 | 2674 | 2743 | 2887 |
|
|
| B-Medium | 64 | 50442 | 46770 | 46765 | 46935 |
|
|
| C-Large | 128 | 390154 | 368210 | 368909 | 370835 |
|
|
| D-Stress | 256 | 780298 | 736402 | 736823 | 738751 |
|
|
| E-Multilayer | 10 | 2223 | 1689 | 1726 | 1723 |
|
|
| F-DAG | 6 | 1336 | 1014 | 1014 | 1014 |
|
|
|
|
PARALLEL SCALING (speedup(N)=cycles(1)/cycles(N),
|
|
efficiency(N)=speedup(N)/N -- NOT assumed, computed from the real
|
|
cycle counts above):
|
|
|
|
| Workload | spd N=2 | eff N=2 | spd N=4 | eff N=4 | spd N=8 | eff N=8 |
|
|
|--------------|---------|---------|---------|---------|---------|---------|
|
|
| A-Small | 1.33x | 66.3% | 1.29x | 32.3% | 1.23x | 15.4% |
|
|
| B-Medium | 1.08x | 53.9% | 1.08x | 27.0% | 1.08x | 13.4% |
|
|
| C-Large | 1.06x | 53.0% | 1.06x | 26.4% | 1.05x | 13.2% |
|
|
| D-Stress | 1.06x | 53.0% | 1.06x | 26.5% | 1.06x | 13.2% |
|
|
| E-Multilayer | 1.32x | 65.8% | 1.29x | 32.2% | 1.29x | 16.1% |
|
|
| F-DAG | 1.32x | 65.9% | 1.32x | 32.9% | 1.32x | 16.5% |
|
|
|
|
REAL WALL-CLOCK speedup (cycles / real POST-P&R Fmax, workload
|
|
D-Stress, the largest/most representative sustained workload):
|
|
|
|
| N_SLOTS | Fmax (POST-P&R) | cycles | wall-clock us | speedup vs N=1 |
|
|
|-----------|-------------------|---------|----------------|------------------|
|
|
| 1 | 152.46 MHz | 780298 | 5118.05 | 1.000x |
|
|
| 2 | 142.45 MHz | 736402 | 5169.55 | 0.990x (SLOWER) |
|
|
| 4 | 113.38 MHz | 736823 | 6498.70 | 0.788x (SLOWER) |
|
|
|
|
HEADLINE FINDING: real parallel scaling is essentially FLAT for
|
|
memory-bound workloads (C/D) regardless of N_SLOTS (1.05-1.06x speedup
|
|
from 1 to 8 slots) -- the shared PSRAM port (91% utilized regardless
|
|
of N_SLOTS>=2, see per-workload utilization data in the full report)
|
|
is the real bottleneck, not compute/slot count. Once real Fmax
|
|
degradation from added routing congestion is also accounted for
|
|
(POST-P&R, not assumed), N_SLOTS=4 is actually 21% SLOWER in real
|
|
wall-clock time than N_SLOTS=1 for the Stress workload -- more
|
|
hardware parallelism made this workload WORSE, not better. Small/
|
|
bursty workloads (A/E/F) DO see a real ~1.2-1.3x wall-clock benefit
|
|
from N_SLOTS=2 (better overlap of registration/scheduling latency),
|
|
but that benefit saturates immediately and does not extend to N=4/8.
|
|
|
|
Per-slot tile distribution (C-Large, N_SLOTS=4) also reveals real,
|
|
measured scheduling imbalance from the fixed lowest-index-priority
|
|
arbiter/director (decisions.log DEC-0010): slot 0/1 each delivered
|
|
1008/1008 real tiles while slot 2/3 delivered only 16/16 -- confirming
|
|
DEC-0010's original starvation concern with much stronger evidence
|
|
than EXP-0013's small-scale test could show (see decisions.log
|
|
DEC-0014 for the resulting recommendation).
|
|
|
|
Resource/Fmax summary (all real POST-P&R, full neural_multiprocessor
|
|
including the real V1 PSRAM chain):
|
|
|
|
| N_SLOTS | Fmax (POST-P&R) | LUT4 | FF | DSP | BRAM |
|
|
|-----------|-------------------|------|------|-----|------|
|
|
| 1 | 152.46 MHz | 2642 | 2240 | 8 | 0 |
|
|
| 2 | 142.45 MHz | 4191 | 3659 | 16 | 0 |
|
|
| 4 | 113.38 MHz | 7552 | 6495 | 32 | 0 |
|
|
|
|
Bit-exact functional verification: ALL 6 workloads x ALL 4
|
|
configurations (24 total workload/config combinations) PASSED
|
|
bit-exact against a software golden model, zero errors, zero
|
|
timeouts, zero deadlocks (after fixing the 3 issues in errors.log
|
|
ERR-0009). No node lost, no node duplicated, correct multi-hop
|
|
dependency wake-up verified (workload F's 2-hop diamond+fan-in graph).
|