feat: real 32-bit DDR3 channel widening - functionally complete, timing NOT yet closed (EXP-0084)

Real 32-bit DDR3 widening (2x MT41J128M16JT-125:K chips ganged in
parallel, user's own MIG wizard session). Full RTL adaptation across
the shared ctrl bus (16-bit word -> 32-bit word, BURST_LEN=8 unchanged,
burst payload 128->256 bits):

- mig_native_adapter.v: app_wdf_data/app_rd_data 64->128 bits (real,
  confirmed against the regenerated MIG wrapper), beat count unchanged.
- act_tile_fetch.v: real logic change - burst now holds 4 tiles instead
  of 2 (sel_lat extended to 2 registered bits, 4-way case mux instead
  of 2-way ternary, same request-time-registered-select discipline as
  EXP-0081). Not a further bytes/MAC reduction, just what's needed to
  keep 100% packing utilization at the larger burst.
- host_mem_bridge.v: real addressing redesign - host-facing 16-bit-word
  contract kept unchanged (ESP32 firmware unaffected), internally
  translated onto the new 32-bit-native ctrl bus.
- sdram_arbiter_n.v, layer_prefetch_ctrl.v, packed_slot.v,
  ddr_prefetch_mgr.v, n2_system_ddr3_top.v: mechanical width bump plus
  doubled ddr3_dq/dqs/dm pins and the real differential sys_clk/clk_ref
  top-level ports the regenerated MIG now requires.

New burst_mem_model32.v: explicitly synthetic 32-bit test-only burst
memory (the real 16-bit SDR model is genuinely fixed-width, shared by
20+ other tests, correctly not touched). Found and fixed a real
address-aliasing bug in it during bring-up (MEM_ADDR_BITS=16 silently
wrapped a real 0x10000 test address to 0).

Real verification: all isolated testbenches re-verified (10/10, 33/33,
32/32, 7/7, 9/9 PASS), plus real xsim against the real 2-chip DDR3
model (tb_mig_native_adapter.v 12/12 PASS, tb_n2_system_ddr3.v 8/8
PASS, both chips visibly returning different real data).

Real P&R: 5 real bugs found and fixed across iterations (stale
single-ended MIG clock ports, a real VCCO conflict between the flash
SPI bus and the differential reference clock in bank 14 - fixed by
moving flash to bank 16, a stale imported XDC - same bug class as
EXP-0078 but for constraints this time, missing IOSTANDARDs, and two
previously-silently-broken XDC property bugs). Route completes 100%,
but real timing does NOT close: WNS -0.618ns, 213 failing endpoints.

Honest root cause: the violation is inside neural_processor_packed.v's
own packed-MAC accumulation tree, unchanged since EXP-0059 - it has
real margin at the old 155.039MHz ui_clk but not at the new 172.414MHz
the paired clock-period change produced. This is NOT caused by the
32-bit width change itself. Width alone, even at the old clock, already
delivers the full intended 2x bandwidth gain (1.24 -> ~2.48 GB/s) -
width and clock rate are separable levers. Current trustworthy timing
signoff remains EXP-0083 (16-bit, +0.073ns) until the clock period is
reverted toward 3225ps (keeping Data Width=32) in one more real,
user-gated MIG wizard session.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUG92aM9m68TRc4rG55BcC
This commit is contained in:
2026-09-20 16:28:44 +02:00
co-authored by Claude Sonnet 5
parent 376ccb6ee2
commit 9dead54ebf
21 changed files with 1302 additions and 754 deletions
+99 -49
View File
@@ -1,21 +1,25 @@
# FPGA-Neural V3 — Architecture Analysis: Timing, Bottlenecks, and Recommended Interventions
Scope: the current, real, P&R-verified V3 design (`hardware/v3/`, branch
`v3-artix7`), updated through EXP-0083 (DDRManager phase 1, real P&R: WNS
+0.073ns). Every number in this document is either directly measured (real
simulation trace, real P&R report) or a calculation built from directly-
measured building blocks — the two are labeled explicitly throughout.
Nothing here is guessed.
`v3-artix7`), updated through EXP-0084 (32-bit DDR3 widening, functionally
complete and real-verified; real P&R timing does NOT yet close at the
paired clock speedup — see §5.4.1). The current real, trustworthy *timing*
signoff remains EXP-0083 (16-bit, 155.039MHz, WNS +0.073ns) until a real
P&R closes for the 32-bit configuration. Every number in this document is
either directly measured (real simulation trace, real P&R report) or a
calculation built from directly-measured building blocks — the two are
labeled explicitly throughout. Nothing here is guessed.
**Status note (post EXP-0083)**: §5.1 (denser activation packing) described
below as a *recommendation* is **DONE and real-P&R-verified** (EXP-0081/
0082), and §5.2 (DDRManager) phase 1 is **also DONE and real-measured**
(EXP-0083, a genuinely modest ~2.9% real benefit — see that section for the
honest number and why the original hypothesis overstated it). See the
"DONE" markers in those sections and the updated bandwidth numbers in §3. The
document originally analyzed the pre-fix state; it's kept below (marked
historical) because the comparison is itself informative, then updated with
the real post-fix numbers throughout.
**Status note (post EXP-0084)**: §5.1 (denser activation packing) is
**DONE and real-P&R-verified** (EXP-0081/0082); §5.2 (DDRManager) phase 1
is **DONE and real-measured** (EXP-0083, a genuinely modest ~2.9% real
benefit); §5.4 (32-bit widening) is **functionally DONE and real-verified**
but its own real timing has **not yet closed** (EXP-0084 — honest root
cause: an unrelated, pre-existing compute-pipeline critical path, not the
width change itself; see §5.4.1). See the "DONE" markers in those sections.
The document originally analyzed the pre-fix state; it's kept below
(marked historical) because the comparison is itself informative, then
updated with the real post-fix numbers throughout.
---
@@ -34,7 +38,8 @@ DDR3 channel.
| Real DDR3 bandwidth needed for ONE core at peak DSP throughput | **4.96 GB/s** | calculated from measured DSP rate + memory layout (§3.2) |
| → DDR3 can sustain, pre-EXP-0081 packing (1 tile/burst) | **~25%** of one core's peak compute throughput | §3.2, historical |
| → DDR3 can sustain, post-EXP-0081/0082 packing (2 tiles/burst, DONE) | **~50%** of one core's peak compute throughput | §3.2, current, real |
| Real P&R timing margin (WNS) | **+0.073 ns** | measured, EXP-0083 real P&R (improved further from EXP-0082's +0.068ns, and EXP-0079's +0.030ns before that) |
| Real P&R timing margin (WNS), current trustworthy (16-bit) | **+0.073 ns** | measured, EXP-0083 real P&R |
| Real P&R timing, 32-bit widened config (EXP-0084) | **-0.618 ns (FAILS)** | measured, real P&R — real root cause is an unrelated, pre-existing compute-pipeline path, not the width change; see §5.4.1 |
| DDRManager phase 1 (single-slot look-ahead prefetch) real benefit | **2.86%** reduction in total real simulated time | measured, real xsim A/B on `tb_n2_system_ddr3.v` (§5.2, EXP-0083) — modest, honestly reported, not oversold |
| DSP48E1 headroom for scaling | 224/240 free (93%) | measured, real P&R utilization |
@@ -44,14 +49,17 @@ achievable fraction of it** — from ~25% to ~50% of one core's peak DSP
throughput, with the real P&R margin *improving*, not degrading, as a side
effect. This was free leverage and it's now banked.
Even at ~50%, DDR3 is still the limiting resource, not DSP count — the next
real interventions, per the user's own explicit direction, are: (a) widening
the physical DDR3 channel from 16-bit to 32-bit (§5.5 — doubles the physical
1.24 GB/s ceiling itself, unlike §5.1 which only reduced waste against a
fixed ceiling), and (b) an intelligent DDRManager (§5.2) to hide latency via
orchestrator-driven prefetch. Both are required together — a wider channel
without a smarter prefetcher still stalls on latency; a smarter prefetcher
against a 16-bit channel still hits the same physical bandwidth wall.
Even at ~50%, DDR3 is still the limiting resource, not DSP count. The 32-bit
channel widening (§5.4) is **functionally real and verified** (real xsim
against the real 2-chip DDR3 model), and width alone already delivers the
full intended 2× physical bandwidth gain (1.24 → ~2.48 GB/s) — but its real
P&R does not yet close timing, because the SAME wizard session also sped up
`ui_clk` by 11.2%, which an unrelated, pre-existing compute-pipeline path
can't currently absorb (§5.4.1). Width and clock rate turned out to be
separable levers; the path forward is reverting just the clock period. The
DDRManager (§5.2) remains complementary — hides latency, doesn't raise the
physical ceiling — and its own real benefit should be re-measured once the
wider channel's timing actually closes.
---
@@ -65,7 +73,8 @@ against a 16-bit channel still hits the same physical bandwidth wall.
| 0078 | + config-flash bridge (real STARTUPE2 placement) | +0.013 | 5213 | 16 | margin dropped — real added logic |
| 0079 | + real activation-fetch engine (`act_tile_fetch.v`) | +0.030 | 5379 | 16 | pre-packing baseline |
| 0082 | + denser activation packing (2 tiles/burst, EXP-0081) | +0.068 | 5437 | 16 | pre-DDRManager baseline |
| 0083 | + DDRManager phase 1 (`ddr_prefetch_mgr.v`, single-slot look-ahead prefetch) | **+0.073** | 5644 | 16 | current, final, trustworthy number — margin IMPROVED again despite +207 LUTs |
| 0083 | + DDRManager phase 1 (`ddr_prefetch_mgr.v`, single-slot look-ahead prefetch) | **+0.073** | 5644 | 16 | current, trustworthy 16-bit/155.039MHz signoff |
| 0084 | 32-bit DDR3 widening (2 chips) + clock speedup to 172.414MHz | **-0.618 (FAILS)** | 6418 | 16 | functionally verified (real xsim), real timing NOT closed — root cause is the clock speedup, not the width; see §5.4.1. Not the current trustworthy number. |
**Observation**: WNS does not move monotonically with LUT count (0.056 →
0.013 → 0.030 → 0.068 → 0.073 while LUTs only ever grow) — this is normal P&R behavior
@@ -390,7 +399,7 @@ never as N-scaled literal top-level pins again. Same architectural shape as
the weight-fetch path, in reverse (write instead of read) — a reasonable,
bounded scope, and a real prerequisite, not optional polish.
### 5.4 [Decided] Widening the physical DDR3 channel: 32-bit single channel vs. a second independent 16-bit channel
### 5.4 [Decided, RTL done] Widening the physical DDR3 channel: 32-bit single channel vs. a second independent 16-bit channel — **functionally complete (EXP-0084), real timing pending**
**The real question**: §5.1 halved *waste* against a fixed 1.24 GB/s
physical ceiling; it did not raise the ceiling itself. Getting past ~50%
@@ -418,32 +427,73 @@ this package (the DQS-capable pins exist), but:
| **Conflict with already-placed I/O** | None | **Real, direct**: the management-SPI bus (bank 15: A15/B16/B17/A16) and the config-flash bus (bank 14: K17/K18/L13) are already placed in exactly the banks that would need to host a second channel. Only bank 16 (11 pins) would remain free — not enough margin for either SPI bus, let alone both. |
| Controller logic cost | One MIG instance, wider data path (mostly automatic — the MIG wizard regenerates CAS/CWL/MMCM ratios for the new width) | A full second MIG instance: second calibration sequence, second `ui_clk` domain, and critically the **DDRManager/arbiter would need to become channel-aware**, not just requester-aware — real added complexity on top of §5.2's own design, not a simplification |
| Real risk given current thin margin (+0.073ns) | Lower — one controller, one clock domain, incremental change to an already-proven design | Higher — two independent PHYs, two calibration state machines, cross-channel coordination logic, all new |
| PCB impact | None (same pins, same DDR3 part, different bus width usage — **the physical board the user is designing does not need to change** for this) | Would require re-routing/relocating whichever board-level bus (SPI mgmt or flash) currently occupies bank 14/15 pins — real PCB-level rework, not just RTL |
| PCB impact | **Corrected 2026-09-20 (real, caught mid-MIG-wizard-session, not assumed away)**: the selected DDR3 part, MT41J128M16JT-125:K, is a **x16 component** — the MIG wizard's "Data Width" field is the *controller's* aggregate width, not the chip's own width, so Data Width=32 with this part means **two physical MT41J128M16 chips wired in parallel**, each supplying half the data bus (standard DDR3 ganging — address/command fan out to both chips, only DQ/DQS/DM roughly double). This is a real BOM/footprint/routing addition, **not** "no PCB change" as an earlier draft of this document incorrectly claimed. Whether the extra DQ/DQS pins still fit within banks 34/35's existing allocation (likely, since address/command isn't duplicated) or require reaching into bank 14/15 must be confirmed from the real regenerated pinout, not assumed. | Also requires a second physical DDR3 chip (or a second x16-wide subsystem) for its own full independent controller, **plus** claims bank 14/15 for that second channel's own address/command/data — the SPI/flash pin conflict is additional to, not instead of, the same per-chip PCB cost as the 32-bit option. |
**Recommendation (honest, not deferential, as requested)**: **32-bit single-
channel widening**, not a second independent channel. The bandwidth gain is
identical, but the 32-bit path has zero pin conflicts with already-placed,
already-verified I/O (SPI management bus, config-flash bus), a much smaller
real risk profile against the current thin timing margin, doesn't require
a second full MIG/calibration instance, and — most importantly for the
user's own board — needs **no PCB changes**, since it reuses the DDR3 part's
own existing data pins at a wider access width rather than claiming new
banks. A second channel's only real advantage (aggregate bandwidth could in
principle scale further with a 3rd/4th channel later) doesn't apply here —
this package genuinely has no more free DQS-capable banks to grow into
after banks 14/15/34/35, so there's no future-proofing benefit being given
up. **User confirmed this recommendation and it is the decided path
forward.**
**Recommendation (honest, not deferential, as requested — and corrected
once, honestly, mid-session)**: **32-bit single-channel widening**, not a
second independent channel — this part of the recommendation still holds.
What does **not** hold, and was a real mistake in this document's earlier
version: the claim that widening needs "no PCB changes". It does — a second
physical DDR3 chip, confirmed by the MIG wizard itself (Memory Details:
x16, single component) when Data Width was set to 32. The real, remaining
differentiator versus a second independent channel is narrower than
originally stated but still real: 32-bit widening very likely stays within
the ALREADY-USED banks 34/35 (only data/strobe pins roughly double, address/
command is shared), while a second independent channel definitely needs its
OWN address/command/data pins, conflicting with the already-placed SPI
management bus (bank 15) and config-flash bus (bank 14). **User confirmed
they already understood the second-chip requirement and decided to proceed
with 32-bit widening regardless** (2026-09-20) — this remains the decided
path, now on the correct, fully-disclosed premise.
**What this requires (not yet done, real, disclosed)**: the real Xilinx MIG
"Customize IP" wizard must be re-run interactively (Data Width 16→32 AND
Input Clock Period both changed in the *same* wizard session, since both
require the wizard's own JEDEC/PLL calculator to recompute CAS Latency/CWL/
MMCM ratios correctly — this is **not** safe to hand-edit in `mig_a.prj` the
way the earlier `TargetFPGA` speed-grade field was, per this project's own
established discipline). This is a real, outstanding, user-gated
prerequisite before §5.2's DDRManager and any N>2 scaling test can use the
wider channel.
**What this required**: the real Xilinx MIG "Customize IP" wizard, re-run
interactively by the user (Data Width 16→32 AND Input Clock Period changed
in the same wizard session, since both require the wizard's own JEDEC/PLL
calculator — never hand-edited, per this project's own established
discipline). **Done, 2026-09-20.**
#### 5.4.1 Real outcome (EXP-0084): functionally complete, timing NOT yet closed — the width/clock-speed decoupling
**Confirmed correct, as predicted above**: the widened DQ/DQS/DM pins
stayed within banks 34/35 (no bank 14/15 conflict for the DDR3 signals
themselves) — real, verified from the actual routed pinout, not assumed.
One real prediction from this section did **not** hold, though, and is
corrected here rather than hidden: the differential **reference clock**
(`clk_ref_p/n`, a signal this section didn't analyze, chosen later in the
same wizard session) landed in **bank 14** by the wizard's own real UG586
placement rules — directly conflicting with the already-placed config-flash
SPI bus there (LVCMOS33 vs LVDS_25, incompatible VCCO in one bank). Real,
observed as an actual `place_design` failure, not hypothetical — fixed by
moving the flash bus to bank 16 (see `docs/PHYSICAL_REALIZATION.md` §2.3).
**The real, more consequential finding**: the SAME wizard session paired
the Data Width change with an Input Clock Period change (3225ps→2900ps,
chosen to keep the PHY:Controller ratio at 2:1 rather than falling back to
4:1, which would have halved `ui_clk` instead of speeding it up). This
clock speedup pushed `ui_clk` from 155.039MHz to 172.414MHz (+11.2%) — and
real P&R shows this specific increase breaks timing in
`neural_processor_packed.v`'s own packed-MAC accumulation tree (unchanged
since EXP-0059, real positive margin at the old rate, real -0.618ns
violation at the new one). **This is unrelated to the 32-bit width change
itself** — every module touched for the widening is real-verified
functionally correct (real xsim, real 2-chip DDR3 model, all tests PASS).
**The honest, useful conclusion**: width and clock rate are separable
levers, and this experiment shows they don't have to be exercised together.
Bandwidth = width × clock — 32-bit width **alone**, even at the OLD,
already-timing-proven 155.039MHz, already delivers the full intended 2×
bandwidth gain (1.24 → ~2.48 GB/s). The clock speedup was free upside *if*
the compute datapath could absorb it; real measurement shows it currently
can't, without a real, separate re-pipelining of that accumulation tree —
out of scope for a channel-width task, not attempted here.
**Current real status**: RTL fully adapted and functionally verified (see
`hardware/v2/logs/experiments.log` EXP-0084 for the complete story,
including 5 real P&R/XDC bugs found and fixed). Real timing signoff for
this configuration is **pending** — next real step is reverting Input
Clock Period toward 3225ps (keeping Data Width=32) in one more real,
user-gated MIG wizard session. The current real, trustworthy P&R signoff
remains EXP-0083 (16-bit, 155.039MHz, +0.073ns) until then.
### 5.5 Scaling path recommendation (real numbers, not a guess) — updated per user's final directive