docs: complete architecture analysis - DDR3 bandwidth is the real ceiling, not DSP count (EXP-0080)

Before building N=4/8/16 core scaling, did the requested full
analysis. Real finding: using measured DDR3 throughput (1.24 GB/s,
from the actual EXP-0079 JEDEC trace) against calculated compute-side
need (4.96 GB/s for one core at peak DSP throughput, given the
current activation memory layout's 2x byte overhead), the system is
memory-bandwidth-bound already at N=1/N=2, not DSP-bound (only 6.67%
DSP used). Scaling core count today would show no real throughput
gain.

docs/ARCHITECTURE_ANALYSIS.md: full module review + ranked
interventions -- result-writeback engine (blocker), denser activation
packing (highest-leverage bandwidth fix), then the user's own proposed
DDRManager/orchestrator-prefetch idea (design-sketched, grounded in
neural_director_packed.v's existing job queue, explicitly scoped as
complementary to denser packing, not a substitute for it), THEN
N-scaling with real per-N P&R signoff.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUG92aM9m68TRc4rG55BcC
This commit is contained in:
2026-09-20 09:44:10 +02:00
co-authored by Claude Sonnet 5
parent 43a12379a5
commit 27cf5f36da
3 changed files with 420 additions and 0 deletions
+5
View File
@@ -19,6 +19,11 @@ unmodified by v3, e.g. `layer_prefetch_ctrl.v`/`layer_weight_buffer.v`).
timing number, and memory-layout convention needed for the physical board timing number, and memory-layout convention needed for the physical board
and for host (ESP32) firmware. Keep it in sync with reality — if a pin and for host (ESP32) firmware. Keep it in sync with reality — if a pin
assignment or timing number changes, update this file in the same commit. assignment or timing number changes, update this file in the same commit.
- `docs/ARCHITECTURE_ANALYSIS.md` — real, measured bottleneck analysis
(DDR3 bandwidth is the real ceiling, not DSP count — see it before
proposing to scale core count) and ranked recommended interventions.
Update it whenever a recommendation from it gets built or a new real
bottleneck is found.
- `hardware/v2/logs/experiments.log` — the real project history, one - `hardware/v2/logs/experiments.log` — the real project history, one
`EXP-NNNN` entry per real experiment/change (context/method/result/ `EXP-NNNN` entry per real experiment/change (context/method/result/
decision/next_action). Read the tail before starting new work; append a decision/next_action). Read the tail before starting new work; append a
+349
View File
@@ -0,0 +1,349 @@
# FPGA-Neural V3 — Architecture Analysis: Timing, Bottlenecks, and Recommended Interventions
Scope: the current, real, P&R-verified V3 design (`hardware/v3/`, branch
`v3-artix7`), as of EXP-0079 (real activation-fetch engine, real DDR3 for
both weights and activations, real P&R: WNS +0.030ns). Every number in this
document is either directly measured (real simulation trace, real P&R
report) or a calculation built from directly-measured building blocks — the
two are labeled explicitly throughout. Nothing here is guessed.
---
## 1. Executive summary
The single most important finding of this analysis: **the system is DDR3
memory-bandwidth-bound, not DSP-bound, already at N=1 core** — and this is
true *before* considering any core-count scaling. Adding more compute cores
(N=4/8/16) without first addressing memory bandwidth would not increase real
throughput; it would only add more cores contending for the same saturated
DDR3 channel.
| Metric | Value | Source |
|---|---|---|
| Real DDR3 back-to-back burst bandwidth | **1.24 GB/s** (9.92 Gbps) | measured, real JEDEC trace (§3.1) |
| Real DDR3 bandwidth needed for ONE core at peak DSP throughput | **4.96 GB/s** | calculated from measured DSP rate + real memory layout (§3.2) |
| → DDR3 can sustain at best | **~25%** of one core's peak compute throughput | §3.2 |
| Real P&R timing margin (WNS) | **+0.030 ns** | measured, EXP-0079 real P&R |
| DSP48E1 headroom for scaling | 224/240 free (93%) | measured, real P&R utilization |
The DSP headroom is real and large. The memory-bandwidth ceiling is real and
small. **The highest-leverage next step is fixing the activation-fetch memory
waste (§5.1), not adding cores.** The user's own proposed DDRManager/
prefetch idea (§5.2) is valuable and complementary, but solves a *different*
problem (latency/stalling) than the bandwidth ceiling (§3.2) — both are
covered below, with the distinction made explicit.
---
## 2. Real signoff history (all real P&R runs to date)
| EXP | What changed | WNS (ns) | LUTs | DSP48E1 | Notes |
|---|---|---|---|---|---|
| 0059 | isolated packed core, out-of-context | n/a (isolated) | 507 | 8 | first real P&R, out-of-context only |
| 0074 | first real in-context P&R: DDR3 + pins + register file not yet added | +0.040 | 5140 | 16 | first trustworthy board-accurate number |
| 0076 | + register file, + pin constraints, + SPI physical-layer fix | +0.056 | 5173 | 16 | margin improved slightly (P&R is not perfectly monotonic run to run) |
| 0078 | + config-flash bridge (real STARTUPE2 placement) | +0.013 | 5213 | 16 | margin dropped — real added logic |
| 0079 | + real activation-fetch engine (`act_tile_fetch.v`) | **+0.030** | 5379 | 16 | current, final, trustworthy number |
**Observation**: WNS does not move monotonically with LUT count (0.056 →
0.013 → 0.030 while LUTs only ever grow) — this is normal P&R behavior
(placer/router heuristics find different solutions each run, small logic
changes can shift which path is critical). **Do not extrapolate a trend
line from 3-4 data points** — the only safe practice is a fresh real P&R
after every real change, which this project already does.
**DSP48E1 has stayed at 16 across every real change since EXP-0059's core
design was fixed** — confirms the packed-DSP MAC design (§4.1) is the
efficient, stable part of this architecture; all the margin pressure has
come from *control/glue logic* (arbitration, the flash bridge, the
activation engine's FSM), not from the compute datapath itself.
---
## 3. The real memory-bandwidth bottleneck (the analysis's central finding)
### 3.1 Real measured DDR3 throughput
From the actual `ddr3_model.sv` JEDEC command trace captured during the
EXP-0079 real simulation run (`tb_n2_system_ddr3.v` via `xsim`), two
back-to-back `Read` commands to the same open row:
```
Read bank 0 col 000 @ 7090615 ps
Read bank 0 col 000 @ 7103515 ps (delta = 12900 ps = 12.9 ns)
```
One `BURST_LEN=8` transaction moves 8 × 16 bits = 128 bits. Real measured
throughput for same-row back-to-back bursts:
```
128 bits / 12.9 ns = 9.92 Gbps = 1.24 GB/s
```
This exactly matches the *theoretical* peak for a 16-bit DDR3 interface at
310.078 MHz (16 bits × 2 (DDR) × 310.078 MHz = 9.92 Gbps) — confirming the
real controller achieves its theoretical ceiling for the best case
(same-row, no row switches). This is the **best-case** number; anything
requiring a row change (Activate/Precharge) is real-measured to cost more
(§3.3).
### 3.2 Real compute-side bandwidth requirement
Each packed core (EXP-0059's design, unchanged since) uses 8 DSP48E1, each
computing 2 packed INT8 MACs per cycle (lane A + lane B sharing one resident
weight) = 16 MACs/cycle/core. At the real measured 155.039 MHz compute clock:
```
16 MACs/cycle × 155.039 × 10^6 cycles/s = 2.48 GMAC/s per core (real, from measured Fmax)
```
Each compute cycle consumes 1 activation byte per MAC lane (16 bytes total:
8 for lane A, 8 for lane B). Under the current real memory layout
(`act_tile_fetch.v`, EXP-0079: one tile = one full `BURST_LEN=8` burst = 16
bytes moved, only 8 useful), feeding one compute cycle costs:
```
2 tiles (A+B) × 16 bytes/burst = 32 bytes moved DDR3 traffic → 16 MACs
= 2 real DDR3 bytes moved per MAC operation
```
Bandwidth needed to keep ONE core's DSPs fed at their real peak rate:
```
2.48 GMAC/s × 2 bytes/MAC = 4.96 GB/s
```
### 3.3 The real gap
```
1.24 GB/s available (measured, best case) vs 4.96 GB/s needed per core (calculated)
→ DDR3 can sustain at most ~25% of one core's peak DSP throughput,
even in the BEST case (zero row-switch overhead, one core, nothing else
sharing the bus).
```
This ceiling gets **worse**, not better, with:
- **Row switches**: real measured Activate→Read latency is 16.125 ns
(5 DDR3 clock cycles at 3.225 ns = real CAS-latency-5 timing, matches the
MIG's own configured CL=5). A tile fetch that requires a fresh row
activation costs ~16-30 ns instead of the 12.9 ns same-row case — a
real, measured 25-130% penalty per row switch.
- **More cores sharing the one DDR3 channel** (N=2 today, N=4/8/16
proposed): the 1.24 GB/s ceiling is shared across ALL active requesters,
not per-core. Adding cores divides an already-insufficient budget further.
- **Weight fetching** (amortized, but not free): each job pair's weight
prefetch (8 bursts for a 128-byte layer) adds real DDR3 traffic on top of
activation fetching, though this cost is shared across the M reuse
positions and becomes negligible for large M.
**Conclusion**: this is a genuine architectural ceiling, not a tuning
problem. The 2× byte-overhead from the current "1 tile = 1 full burst"
convention (chosen in EXP-0079 specifically to avoid a runtime-indexed
part-select, given the then-already-thin timing margin) is now confirmed,
with real numbers, to be the single most expensive design decision in the
current memory path. See §5.1 for the recommended fix.
---
## 4. Module-by-module review
### 4.1 Compute core (`mac2_dsp_packed.v`, `neural_processor_packed.v`)
Real, stable, efficient. 8 DSP48E1/core unchanged since EXP-0059. The
2-INT8-MAC-per-DSP48E1 packing technique is the correct choice for this INT8
workload — real utilization (16/240 DSP = 6.67% at N=2) confirms there is no
DSP-side pressure at all; **all scaling headroom is here, and all scaling
risk is elsewhere** (§3, §4.5).
No real change recommended here. This is the part of the design that is
*not* the bottleneck.
### 4.2 Weight-reuse path (`layer_prefetch_ctrl.v`, `layer_weight_buffer.v`,
`weight_tile_gather.v`)
Reused unmodified from V2 (ECP5 era), real and well-verified across many
EXPs (0057/0058/0061/0062 and every integration test since). Correctly
amortizes DDR3 traffic across M reuse positions — this part of the design
already does the "fetch once, use many times" optimization the activation
path currently lacks (§5.1's recommendation follows the SAME philosophy).
No real change recommended; this module is a good template for how the
activation path should evolve.
### 4.3 Activation-fetch path (`act_tile_fetch.v`, EXP-0079)
Real, correct (verified 3 levels deep, §2 of EXP-0079's own log entry), but
— per §3 above — the current real bottleneck. Two real design choices worth
re-examining now that real bandwidth numbers exist:
1. **1 tile = 1 full burst (2× byte overhead)**: chosen to avoid a
runtime-indexed part-select. §5.1 proposes a way to recover this
efficiency without reintroducing that risk.
2. **Lane A then lane B, sequential, per tile**: doubles the real number of
DDR3 transactions (and row-switch risk) versus a design that could fetch
both lanes in a single wider transaction when they happen to be adjacent
in memory. Not changed in this analysis pass — flagged for future work
if §5.1's fix doesn't fully close the gap.
### 4.4 Scheduling (`neural_director_packed.v`) and arbitration
(`sdram_arbiter_n.v`)
Real, correct, and — importantly for §5.2 — **`neural_director_packed.v`
already maintains a real queue of pending jobs** (`QUEUE_DEPTH=8`,
`q_x_base`/`q_w_base`/etc arrays). This is directly relevant to the
DDRManager/prefetch proposal (§5.2): the information needed to "know what's
coming next" already exists in this module, it just isn't currently used for
anything beyond pairing/dispatch decisions.
`sdram_arbiter_n.v`'s combinational-first-grant design (EXP-0066) is real,
proven, and already generalized to N-way (verified at N=3, EXP-0069) — no
real change needed to scale its own requester count for a DDRManager
addition or for N=4/8/16 scaling, though the arbiter's own **fairness
policy** (lowest-index-wins, a deliberate simplicity choice, EXP-0066) would
need re-examination if a DDRManager starts issuing speculative/anticipatory
requests that could starve a real, urgent request — see §5.2's own caveat.
### 4.5 Host interface (`spi_host_bridge_v3.v`, `flash_spi_master.v`,
`host_mem_bridge.v`)
Real, verified, low resource cost, not on any critical performance path
(host commands are inherently much slower than the internal compute/memory
loop). No bottleneck here. Not a scaling concern.
### 4.6 Missing: result-writeback engine
Still genuinely absent (disclosed since `packed_slot.v`'s own original
header, unchanged through EXP-0079). Currently `result_data_a/b` are literal
top-level pins — functional at N=2 (32 pins), but this is the **exact same
class of mistake already caught once** for activation data (EXP-0074: ~360
pins nearly exceeded the whole package's I/O budget). At N=16 this port
alone would need 8 bits × 2 lanes × 16 cores = 256 pins — **a real, hard
blocker for any scaling beyond a handful of cores**, independent of the
memory-bandwidth ceiling in §3. Recommended fix in §5.3.
---
## 5. Recommended interventions, ranked by real leverage
### 5.1 [Highest leverage] Denser activation packing — attack the real bandwidth ceiling directly
**What**: pack 2 tiles (lane A + lane B, or two consecutive tiles of the
same lane) into one `BURST_LEN=8` burst instead of one tile per burst,
halving real DDR3 bytes-per-MAC from 2 to 1. This alone would raise the
real achievable fraction of one core's peak throughput from ~25% to ~50%
(§3.2's math, halved).
**Why this was avoided in EXP-0079**: doing so naively requires a
runtime-indexed part-select (which half of the burst response to use,
selected by a runtime tile-index bit) — the same anti-pattern
`weight_tile_gather.v` (EXP-0061) already flagged as a real Fmax risk, and
the P&R margin was already thin (+0.013ns) when this design decision was
made.
**A safer path to the same efficiency gain** (not yet built, this is a
recommendation): register the tile-index LSB **one cycle ahead of** the
burst response arriving (it's already known at request time, not something
that needs to race the read data) — using it to select a *pre-registered*
mux input rather than gating the read-data path itself keeps the selection
off the critical timing path. This needs a real prototype and a real P&R
check before being trusted — proposed as the next concrete engineering task,
not asserted as already safe.
### 5.2 [Complementary, addresses latency not bandwidth] DDRManager with orchestrator-driven prefetch (user's proposal)
**The real problem this solves**: even within whatever bandwidth ceiling
§5.1 establishes, the CURRENT design only ever requests a tile the moment
`packed_slot.v`'s own FSM reaches `S_TILEREQ` for it — meaning the DSPs
stall waiting for that fetch's real latency (§3.3: 12.9-30+ ns) every
single tile, with no overlap between "fetching tile N+1" and "computing on
tile N". A DDRManager that issues tile N+1's fetch WHILE tile N is still
computing would hide that latency almost entirely (compute time per tile,
1/155.039MHz ≈ 6.4ns per cycle, vs a real fetch latency of 12.9-30+ns —
today's design is very likely stalling the DSPs for the majority of real
time, an real, additional cost on top of §3's raw bandwidth ceiling).
**What it does NOT solve**: §3.2's bandwidth ceiling is a hard physical
limit (bytes/second the DDR3 channel can physically move) — prefetching
earlier doesn't move more bytes per second, it only avoids IDLE gaps where
the channel is free but nothing is queued to use it. **§5.1 and §5.2 are
complementary, not alternatives** — §5.1 reduces bytes needed per MAC, §5.2
ensures the channel is never idle when there's real bandwidth budget
available and useful work queued. Do both, in this order (§5.1 first, since
it raises the ceiling §5.2 will then use more fully).
**Concrete design sketch** (informed by what already exists in this
codebase, not a from-scratch proposal):
- `neural_director_packed.v` already queues up to `QUEUE_DEPTH=8` pending
jobs, each with a known `x_base`/`w_base`/`n_tiles` — this is exactly the
"reservation" information a DDRManager needs. No new bookkeeping is
required at the Director level; a DDRManager would READ this existing
queue, not need the Director to change its own job-acceptance logic.
- A new module (name suggestion: `ddr_prefetch_mgr.v`) would sit between
`sdram_arbiter_n.v` and the per-slot `act_tile_fetch.v`/
`layer_prefetch_ctrl.v` instances, with a small staging buffer per slot
(double-buffered, matching `layer_weight_buffer.v`'s own already-proven
double-buffer pattern) — while `packed_slot.v` computes on the CURRENT
tile, the manager issues the request for the NEXT tile into the "other"
buffer, swapping on completion.
- **Real caveat, not glossed over**: this adds real arbitration complexity
— a prefetched-but-not-yet-consumed request competing with another slot's
genuinely urgent request needs a real priority policy, not just
`sdram_arbiter_n.v`'s current lowest-index-wins simplicity (§4.4). A
speculative prefetch that turns out to be wrong (e.g., the Director
reorders/never dispatches that queued job) also wastes real bandwidth —
needs a real cancellation/staleness mechanism, not assumed away.
- **Recommended validation before committing engineering time**: build a
minimal version scoped to ONE slot's OWN activation-tile look-ahead
(prefetch tile N+1 while computing tile N, using the double-buffer
pattern above) before attempting the full "reserve across the whole
Director queue" version — matches this project's own "one variable at a
time" discipline, and would give a real, measured stall-reduction number
to justify (or not) the added complexity of the full design.
### 5.3 [Blocking for any real scaling] Result-writeback engine
Must exist before N>2 is even attemptable (§4.6) — result data needs to go
into DDR3 (or through the SPI status/register path for small result sets),
never as N-scaled literal top-level pins again. Same architectural shape as
the weight-fetch path, in reverse (write instead of read) — a reasonable,
bounded scope, and a real prerequisite, not optional polish.
### 5.4 Scaling path recommendation (real numbers, not a guess)
Given §3's real bandwidth ceiling: **scaling core count alone, without
§5.1/§5.2, provides no real additional throughput past whatever N already
saturates the 1.24 GB/s ceiling** — back-of-envelope, using §3.2's numbers,
that's already close to N=1 in the worst case, and at most N≈2 in the best
(zero-row-switch) case. **Building N=4/8/16 today, before §5.1, would very
likely show near-IDENTICAL real throughput to N=2 in a real P&R'd, real-
DDR3-simulated test** — a real, wasted engineering cycle the analysis
recommends avoiding.
**Recommended real order of work**:
1. §5.3 (result-writeback) — genuine blocker, bounded scope.
2. §5.1 (denser activation packing) — highest real leverage on the actual
ceiling, needs a real P&R check given the thin margin.
3. Re-measure real achievable throughput at N=2 with §5.1 in place (real
simulation, real cycle counts) — THIS number, not a projection, should
decide whether N=4 is worth building next.
4. §5.2 (DDRManager/prefetch) — real, valuable, but its benefit is easier to
quantify and justify once §5.1 has already raised the ceiling it's
filling.
5. Only then: N=4/8/16, each with its own real P&R signoff (the margin is
thin, §2 — do not assume a prior N's timing closure predicts the next).
---
## 6. Summary table: what's real vs. what's a calculation
| Claim | Status |
|---|---|
| WNS/WHS/LUT/DSP numbers throughout | **Measured** (real Vivado P&R reports) |
| DDR3 back-to-back burst throughput (1.24 GB/s) | **Measured** (real `ddr3_model.sv` JEDEC trace) |
| Real Activate→Read latency (16.125 ns) | **Measured** (same trace) |
| Per-core compute throughput (2.48 GMAC/s) | **Calculated** from measured Fmax (155.039MHz) + known, fixed DSP-packing factor |
| Bandwidth needed per core (4.96 GB/s) | **Calculated** from the above + the real, as-built memory layout convention |
| "~25% of peak sustainable" | **Calculated** ratio of the two measured/calculated numbers above |
| Row-switch penalty as a fraction of real workloads | **Not measured** — depends on host-chosen memory layout, flagged as an open question, not asserted |
| DDRManager's real stall-reduction benefit | **Not measured** — no prototype exists yet; §5.2 recommends building a minimal version specifically to get this real number |
+66
View File
@@ -5031,3 +5031,69 @@ was already thin, +0.013ns, before this addition) -- must re-confirm
timing still closes before calling this "ready to use in physical timing still closes before calling this "ready to use in physical
hardware". Also: finalize and commit docs/PHYSICAL_REALIZATION.md hardware". Also: finalize and commit docs/PHYSICAL_REALIZATION.md
(drafted this session, real pin/part/protocol/layout data). (drafted this session, real pin/part/protocol/layout data).
EXP-0080 -- complete architecture analysis: real DDR3 bandwidth ceiling
found and quantified, before any N-scaling work (2026-09-20, same
autonomous continuation, user's own explicit request: "fai prima una
analisi completa", plus their own proposed DDRManager/prefetch idea)
CONTEXT: user asked for N=4/8/16 core scaling "sul prodotto finito"
(on the real hardware). Before spending real engineering/P&R time
building that blind, did the requested full analysis first -- and it
changed the recommended plan substantially.
REAL FINDING: using real measured numbers (DDR3 back-to-back burst
throughput from the actual JEDEC trace in EXP-0079's own real
simulation run: 128 bits / 12.9ns = 1.24 GB/s) against calculated
compute-side bandwidth need (one packed core's real 155.039MHz Fmax x
16 MACs/cycle = 2.48 GMAC/s, x 2 real DDR3 bytes/MAC under the current
"1 tile = 1 full burst" activation layout, EXP-0079 = 4.96 GB/s
needed) -- DDR3 can sustain at best ~25% of ONE core's peak DSP
throughput. The system is memory-bandwidth-bound, not DSP-bound,
already at N=1/N=2. Confirmed DSP headroom is real and large (16/240
used, 6.67%) but irrelevant until the memory ceiling is addressed --
scaling core count today would show near-identical real throughput to
N=2, wasting real P&R cycles building N=4/8/16 for no real gain.
Wrote docs/ARCHITECTURE_ANALYSIS.md: full module-by-module review,
the real signoff history table (WNS non-monotonic across EXP-0074/76/
78/79, confirmed NOT a trend to extrapolate from), and ranked
recommended interventions:
1. Result-writeback engine (real blocker for N>2 regardless of
bandwidth -- same pin-explosion risk already caught once for
activations, EXP-0074).
2. Denser activation packing (1 byte/MAC instead of 2 -- doubles the
real achievable throughput ceiling) -- proposed a safer approach
than the runtime part-select EXP-0079 deliberately avoided
(register the tile-index bit one cycle ahead of the burst
response, keeping selection off the critical path) -- NOT yet
built or verified, flagged as needing a real prototype + P&R
check.
3. Re-measure real N=2 throughput with (2) in place BEFORE deciding
if N=4 is worth building.
4. User's own DDRManager/orchestrator-prefetch idea -- real design
sketch grounded in what already exists (neural_director_packed.v
already queues up to 8 pending jobs with known x_base/w_base --
exactly the "reservation" data a prefetch manager needs, no
Director changes required). Explicitly scoped: this hides
LATENCY (stalls waiting for a fetch), it does NOT raise the
bandwidth CEILING (2) does -- presented as complementary to (2),
not a substitute, since conflating the two would overstate what
prefetching alone can fix. Recommended minimal validation: one
slot's own double-buffered look-ahead prefetch (mirrors layer_
weight_buffer.v's already-proven double-buffer pattern) before
attempting the full multi-job-queue version.
5. Only then: N=4/8/16, each with its own real P&R (the margin is
thin and non-monotonic, EXP-0074..0079 -- no N's timing closure
predicts the next).
DECISION: do not build N=4/8/16 yet. Real next engineering task is the
result-writeback engine (genuine blocker) followed by denser
activation packing (real bandwidth-ceiling fix, highest leverage found
in this analysis) -- both real, scoped, bounded pieces of work, not
speculative.
next_action: await user direction on which recommended intervention
to build first (result-writeback engine is the more clearly-scoped,
lower-risk starting point; denser packing needs more design care given
the thin timing margin).