FPGA-Neural/docs/ARCHITECTURE_ANALYSIS.md

FPGA-Neural V3 — Architecture Analysis: Timing, Bottlenecks, and Recommended Interventions

Scope: the current, real, P&R-verified V3 design (hardware/v3/, branch v3-artix7), as of EXP-0079 (real activation-fetch engine, real DDR3 for both weights and activations, real P&R: WNS +0.030ns). Every number in this document is either directly measured (real simulation trace, real P&R report) or a calculation built from directly-measured building blocks — the two are labeled explicitly throughout. Nothing here is guessed.

---

1. Executive summary

The single most important finding of this analysis: the system is DDR3 memory-bandwidth-bound, not DSP-bound, already at N=1 core — and this is true before considering any core-count scaling. Adding more compute cores (N=4/8/16) without first addressing memory bandwidth would not increase real throughput; it would only add more cores contending for the same saturated DDR3 channel.

Metric	Value	Source
Real DDR3 back-to-back burst bandwidth	1.24 GB/s (9.92 Gbps)	measured, real JEDEC trace (§3.1)
Real DDR3 bandwidth needed for ONE core at peak DSP throughput	4.96 GB/s	calculated from measured DSP rate + real memory layout (§3.2)
→ DDR3 can sustain at best	~25% of one core's peak compute throughput	§3.2
Real P&R timing margin (WNS)	+0.030 ns	measured, EXP-0079 real P&R
DSP48E1 headroom for scaling	224/240 free (93%)	measured, real P&R utilization

The DSP headroom is real and large. The memory-bandwidth ceiling is real and small. The highest-leverage next step is fixing the activation-fetch memory waste (§5.1), not adding cores. The user's own proposed DDRManager/ prefetch idea (§5.2) is valuable and complementary, but solves a different problem (latency/stalling) than the bandwidth ceiling (§3.2) — both are covered below, with the distinction made explicit.

---

2. Real signoff history (all real P&R runs to date)

EXP	What changed	WNS (ns)	LUTs	DSP48E1	Notes
0059	isolated packed core, out-of-context	n/a (isolated)	507	8	first real P&R, out-of-context only
0074	first real in-context P&R: DDR3 + pins + register file not yet added	+0.040	5140	16	first trustworthy board-accurate number
0076	+ register file, + pin constraints, + SPI physical-layer fix	+0.056	5173	16	margin improved slightly (P&R is not perfectly monotonic run to run)
0078	+ config-flash bridge (real STARTUPE2 placement)	+0.013	5213	16	margin dropped — real added logic
0079	+ real activation-fetch engine (act_tile_fetch.v)	+0.030	5379	16	current, final, trustworthy number

Observation: WNS does not move monotonically with LUT count (0.056 → 0.013 → 0.030 while LUTs only ever grow) — this is normal P&R behavior (placer/router heuristics find different solutions each run, small logic changes can shift which path is critical). Do not extrapolate a trend line from 3-4 data points — the only safe practice is a fresh real P&R after every real change, which this project already does.

DSP48E1 has stayed at 16 across every real change since EXP-0059's core design was fixed — confirms the packed-DSP MAC design (§4.1) is the efficient, stable part of this architecture; all the margin pressure has come from control/glue logic (arbitration, the flash bridge, the activation engine's FSM), not from the compute datapath itself.

---

3. The real memory-bandwidth bottleneck (the analysis's central finding)

3.1 Real measured DDR3 throughput

From the actual ddr3_model.sv JEDEC command trace captured during the EXP-0079 real simulation run (tb_n2_system_ddr3.v via xsim), two back-to-back Read commands to the same open row:

Read  bank 0 col 000 @ 7090615 ps
Read  bank 0 col 000 @ 7103515 ps   (delta = 12900 ps = 12.9 ns)

One BURST_LEN=8 transaction moves 8 × 16 bits = 128 bits. Real measured throughput for same-row back-to-back bursts:

128 bits / 12.9 ns = 9.92 Gbps = 1.24 GB/s

This exactly matches the theoretical peak for a 16-bit DDR3 interface at 310.078 MHz (16 bits × 2 (DDR) × 310.078 MHz = 9.92 Gbps) — confirming the real controller achieves its theoretical ceiling for the best case (same-row, no row switches). This is the best-case number; anything requiring a row change (Activate/Precharge) is real-measured to cost more (§3.3).

3.2 Real compute-side bandwidth requirement

Each packed core (EXP-0059's design, unchanged since) uses 8 DSP48E1, each computing 2 packed INT8 MACs per cycle (lane A + lane B sharing one resident weight) = 16 MACs/cycle/core. At the real measured 155.039 MHz compute clock:

16 MACs/cycle × 155.039 × 10^6 cycles/s = 2.48 GMAC/s per core (real, from measured Fmax)

Each compute cycle consumes 1 activation byte per MAC lane (16 bytes total: 8 for lane A, 8 for lane B). Under the current real memory layout (act_tile_fetch.v, EXP-0079: one tile = one full BURST_LEN=8 burst = 16 bytes moved, only 8 useful), feeding one compute cycle costs:

2 tiles (A+B) × 16 bytes/burst = 32 bytes moved DDR3 traffic → 16 MACs
= 2 real DDR3 bytes moved per MAC operation

Bandwidth needed to keep ONE core's DSPs fed at their real peak rate:

2.48 GMAC/s × 2 bytes/MAC = 4.96 GB/s

3.3 The real gap

1.24 GB/s available (measured, best case)  vs  4.96 GB/s needed per core (calculated)
→ DDR3 can sustain at most ~25% of one core's peak DSP throughput,
  even in the BEST case (zero row-switch overhead, one core, nothing else
  sharing the bus).

This ceiling gets worse, not better, with:

• Row switches: real measured Activate→Read latency is 16.125 ns (5 DDR3 clock cycles at 3.225 ns = real CAS-latency-5 timing, matches the MIG's own configured CL=5). A tile fetch that requires a fresh row activation costs ~16-30 ns instead of the 12.9 ns same-row case — a real, measured 25-130% penalty per row switch.
• More cores sharing the one DDR3 channel (N=2 today, N=4/8/16 proposed): the 1.24 GB/s ceiling is shared across ALL active requesters, not per-core. Adding cores divides an already-insufficient budget further.
• Weight fetching (amortized, but not free): each job pair's weight prefetch (8 bursts for a 128-byte layer) adds real DDR3 traffic on top of activation fetching, though this cost is shared across the M reuse positions and becomes negligible for large M.

Conclusion: this is a genuine architectural ceiling, not a tuning problem. The 2× byte-overhead from the current "1 tile = 1 full burst" convention (chosen in EXP-0079 specifically to avoid a runtime-indexed part-select, given the then-already-thin timing margin) is now confirmed, with real numbers, to be the single most expensive design decision in the current memory path. See §5.1 for the recommended fix.

---

4. Module-by-module review

4.1 Compute core (mac2_dsp_packed.v, neural_processor_packed.v)

Real, stable, efficient. 8 DSP48E1/core unchanged since EXP-0059. The 2-INT8-MAC-per-DSP48E1 packing technique is the correct choice for this INT8 workload — real utilization (16/240 DSP = 6.67% at N=2) confirms there is no DSP-side pressure at all; all scaling headroom is here, and all scaling risk is elsewhere (§3, §4.5).

No real change recommended here. This is the part of the design that is not the bottleneck.

4.2 Weight-reuse path (layer_prefetch_ctrl.v, layer_weight_buffer.v,

weight_tile_gather.v)

Reused unmodified from V2 (ECP5 era), real and well-verified across many EXPs (0057/0058/0061/0062 and every integration test since). Correctly amortizes DDR3 traffic across M reuse positions — this part of the design already does the "fetch once, use many times" optimization the activation path currently lacks (§5.1's recommendation follows the SAME philosophy).

No real change recommended; this module is a good template for how the activation path should evolve.

4.3 Activation-fetch path (act_tile_fetch.v, EXP-0079)

Real, correct (verified 3 levels deep, §2 of EXP-0079's own log entry), but — per §3 above — the current real bottleneck. Two real design choices worth re-examining now that real bandwidth numbers exist:

1. 1 tile = 1 full burst (2× byte overhead): chosen to avoid a runtime-indexed part-select. §5.1 proposes a way to recover this efficiency without reintroducing that risk.
2. Lane A then lane B, sequential, per tile: doubles the real number of DDR3 transactions (and row-switch risk) versus a design that could fetch both lanes in a single wider transaction when they happen to be adjacent in memory. Not changed in this analysis pass — flagged for future work if §5.1's fix doesn't fully close the gap.

4.4 Scheduling (neural_director_packed.v) and arbitration

(sdram_arbiter_n.v)

Real, correct, and — importantly for §5.2 — neural_director_packed.v already maintains a real queue of pending jobs (QUEUE_DEPTH=8, q_x_base/q_w_base/etc arrays). This is directly relevant to the DDRManager/prefetch proposal (§5.2): the information needed to "know what's coming next" already exists in this module, it just isn't currently used for anything beyond pairing/dispatch decisions.

sdram_arbiter_n.v's combinational-first-grant design (EXP-0066) is real, proven, and already generalized to N-way (verified at N=3, EXP-0069) — no real change needed to scale its own requester count for a DDRManager addition or for N=4/8/16 scaling, though the arbiter's own fairness policy (lowest-index-wins, a deliberate simplicity choice, EXP-0066) would need re-examination if a DDRManager starts issuing speculative/anticipatory requests that could starve a real, urgent request — see §5.2's own caveat.

4.5 Host interface (spi_host_bridge_v3.v, flash_spi_master.v,

host_mem_bridge.v)

Real, verified, low resource cost, not on any critical performance path (host commands are inherently much slower than the internal compute/memory loop). No bottleneck here. Not a scaling concern.

4.6 Missing: result-writeback engine

Still genuinely absent (disclosed since packed_slot.v's own original header, unchanged through EXP-0079). Currently result_data_a/b are literal top-level pins — functional at N=2 (32 pins), but this is the exact same class of mistake already caught once for activation data (EXP-0074: ~360 pins nearly exceeded the whole package's I/O budget). At N=16 this port alone would need 8 bits × 2 lanes × 16 cores = 256 pins — a real, hard blocker for any scaling beyond a handful of cores, independent of the memory-bandwidth ceiling in §3. Recommended fix in §5.3.

---

5. Recommended interventions, ranked by real leverage

5.1 [Highest leverage] Denser activation packing — attack the real bandwidth ceiling directly

What: pack 2 tiles (lane A + lane B, or two consecutive tiles of the same lane) into one BURST_LEN=8 burst instead of one tile per burst, halving real DDR3 bytes-per-MAC from 2 to 1. This alone would raise the real achievable fraction of one core's peak throughput from ~25% to ~50% (§3.2's math, halved).

Why this was avoided in EXP-0079: doing so naively requires a runtime-indexed part-select (which half of the burst response to use, selected by a runtime tile-index bit) — the same anti-pattern weight_tile_gather.v (EXP-0061) already flagged as a real Fmax risk, and the P&R margin was already thin (+0.013ns) when this design decision was made.

A safer path to the same efficiency gain (not yet built, this is a recommendation): register the tile-index LSB one cycle ahead of the burst response arriving (it's already known at request time, not something that needs to race the read data) — using it to select a pre-registered mux input rather than gating the read-data path itself keeps the selection off the critical timing path. This needs a real prototype and a real P&R check before being trusted — proposed as the next concrete engineering task, not asserted as already safe.

5.2 [Complementary, addresses latency not bandwidth] DDRManager with orchestrator-driven prefetch (user's proposal)

The real problem this solves: even within whatever bandwidth ceiling §5.1 establishes, the CURRENT design only ever requests a tile the moment packed_slot.v's own FSM reaches S_TILEREQ for it — meaning the DSPs stall waiting for that fetch's real latency (§3.3: 12.9-30+ ns) every single tile, with no overlap between "fetching tile N+1" and "computing on tile N". A DDRManager that issues tile N+1's fetch WHILE tile N is still computing would hide that latency almost entirely (compute time per tile, 1/155.039MHz ≈ 6.4ns per cycle, vs a real fetch latency of 12.9-30+ns — today's design is very likely stalling the DSPs for the majority of real time, an real, additional cost on top of §3's raw bandwidth ceiling).

What it does NOT solve: §3.2's bandwidth ceiling is a hard physical limit (bytes/second the DDR3 channel can physically move) — prefetching earlier doesn't move more bytes per second, it only avoids IDLE gaps where the channel is free but nothing is queued to use it. §5.1 and §5.2 are complementary, not alternatives — §5.1 reduces bytes needed per MAC, §5.2 ensures the channel is never idle when there's real bandwidth budget available and useful work queued. Do both, in this order (§5.1 first, since it raises the ceiling §5.2 will then use more fully).

Concrete design sketch (informed by what already exists in this codebase, not a from-scratch proposal):

• neural_director_packed.v already queues up to QUEUE_DEPTH=8 pending jobs, each with a known x_base/w_base/n_tiles — this is exactly the "reservation" information a DDRManager needs. No new bookkeeping is required at the Director level; a DDRManager would READ this existing queue, not need the Director to change its own job-acceptance logic.
• A new module (name suggestion: ddr_prefetch_mgr.v) would sit between sdram_arbiter_n.v and the per-slot act_tile_fetch.v/ layer_prefetch_ctrl.v instances, with a small staging buffer per slot (double-buffered, matching layer_weight_buffer.v's own already-proven double-buffer pattern) — while packed_slot.v computes on the CURRENT tile, the manager issues the request for the NEXT tile into the "other" buffer, swapping on completion.
• Real caveat, not glossed over: this adds real arbitration complexity — a prefetched-but-not-yet-consumed request competing with another slot's genuinely urgent request needs a real priority policy, not just sdram_arbiter_n.v's current lowest-index-wins simplicity (§4.4). A speculative prefetch that turns out to be wrong (e.g., the Director reorders/never dispatches that queued job) also wastes real bandwidth — needs a real cancellation/staleness mechanism, not assumed away.
• Recommended validation before committing engineering time: build a minimal version scoped to ONE slot's OWN activation-tile look-ahead (prefetch tile N+1 while computing tile N, using the double-buffer pattern above) before attempting the full "reserve across the whole Director queue" version — matches this project's own "one variable at a time" discipline, and would give a real, measured stall-reduction number to justify (or not) the added complexity of the full design.

5.3 [Blocking for any real scaling] Result-writeback engine

Must exist before N>2 is even attemptable (§4.6) — result data needs to go into DDR3 (or through the SPI status/register path for small result sets), never as N-scaled literal top-level pins again. Same architectural shape as the weight-fetch path, in reverse (write instead of read) — a reasonable, bounded scope, and a real prerequisite, not optional polish.

5.4 Scaling path recommendation (real numbers, not a guess)

Given §3's real bandwidth ceiling: scaling core count alone, without §5.1/§5.2, provides no real additional throughput past whatever N already saturates the 1.24 GB/s ceiling — back-of-envelope, using §3.2's numbers, that's already close to N=1 in the worst case, and at most N≈2 in the best (zero-row-switch) case. Building N=4/8/16 today, before §5.1, would very likely show near-IDENTICAL real throughput to N=2 in a real P&R'd, real- DDR3-simulated test — a real, wasted engineering cycle the analysis recommends avoiding.

Recommended real order of work:

1. §5.3 (result-writeback) — genuine blocker, bounded scope.
2. §5.1 (denser activation packing) — highest real leverage on the actual ceiling, needs a real P&R check given the thin margin.
3. Re-measure real achievable throughput at N=2 with §5.1 in place (real simulation, real cycle counts) — THIS number, not a projection, should decide whether N=4 is worth building next.
4. §5.2 (DDRManager/prefetch) — real, valuable, but its benefit is easier to quantify and justify once §5.1 has already raised the ceiling it's filling.
5. Only then: N=4/8/16, each with its own real P&R signoff (the margin is thin, §2 — do not assume a prior N's timing closure predicts the next).

---

6. Summary table: what's real vs. what's a calculation

Claim	Status
WNS/WHS/LUT/DSP numbers throughout	Measured (real Vivado P&R reports)
DDR3 back-to-back burst throughput (1.24 GB/s)	Measured (real ddr3_model.sv JEDEC trace)
Real Activate→Read latency (16.125 ns)	Measured (same trace)
Per-core compute throughput (2.48 GMAC/s)	Calculated from measured Fmax (155.039MHz) + known, fixed DSP-packing factor
Bandwidth needed per core (4.96 GB/s)	Calculated from the above + the real, as-built memory layout convention
"~25% of peak sustainable"	Calculated ratio of the two measured/calculated numbers above
Row-switch penalty as a fraction of real workloads	Not measured — depends on host-chosen memory layout, flagged as an open question, not asserted
DDRManager's real stall-reduction benefit	Not measured — no prototype exists yet; §5.2 recommends building a minimal version specifically to get this real number