docs: real timing closure for 32-bit DDR3 channel (EXP-0086)
Reverting Clock Period to 3225ps (keeping Data Width=32) closes real timing: WNS=+0.095707ns, WHS=+0.036275ns, 0 failing endpoints, all user-specified constraints met. This is the real final signoff for the 32-bit DDR3 channel widening effort (EXP-0083 through EXP-0086), replacing EXP-0083's 16-bit-era baseline as the project's current trustworthy number. Confirms the EXP-0084 root-cause analysis: the earlier -0.618ns failure was caused by a separate, stacked clock speedup, not the width change itself. Also found and fixed a second, wholesale real occurrence of the stale-import bug: the MIG wizard regeneration needed to revert Clock Period silently re-imported the entire v3 RTL tree (9 files) plus the top XDC back to pre-EXP-0084 stale copies, not just the file(s) the regeneration touched. CLAUDE.md's lesson extended accordingly.
This commit is contained in:
@@ -1,25 +1,26 @@
|
||||
# FPGA-Neural V3 — Architecture Analysis: Timing, Bottlenecks, and Recommended Interventions
|
||||
|
||||
Scope: the current, real, P&R-verified V3 design (`hardware/v3/`, branch
|
||||
`v3-artix7`), updated through EXP-0084 (32-bit DDR3 widening, functionally
|
||||
complete and real-verified; real P&R timing does NOT yet close at the
|
||||
paired clock speedup — see §5.4.1). The current real, trustworthy *timing*
|
||||
signoff remains EXP-0083 (16-bit, 155.039MHz, WNS +0.073ns) until a real
|
||||
P&R closes for the 32-bit configuration. Every number in this document is
|
||||
either directly measured (real simulation trace, real P&R report) or a
|
||||
calculation built from directly-measured building blocks — the two are
|
||||
labeled explicitly throughout. Nothing here is guessed.
|
||||
`v3-artix7`), updated through EXP-0086 (32-bit DDR3 widening, functionally
|
||||
complete AND real timing CLOSED). The current real, trustworthy *timing*
|
||||
signoff is now EXP-0086 (32-bit, 155.039MHz, WNS +0.096ns), replacing the
|
||||
prior EXP-0083 16-bit-era pointer. Every number in this document is either
|
||||
directly measured (real simulation trace, real P&R report) or a calculation
|
||||
built from directly-measured building blocks — the two are labeled
|
||||
explicitly throughout. Nothing here is guessed.
|
||||
|
||||
**Status note (post EXP-0084)**: §5.1 (denser activation packing) is
|
||||
**Status note (post EXP-0086)**: §5.1 (denser activation packing) is
|
||||
**DONE and real-P&R-verified** (EXP-0081/0082); §5.2 (DDRManager) phase 1
|
||||
is **DONE and real-measured** (EXP-0083, a genuinely modest ~2.9% real
|
||||
benefit); §5.4 (32-bit widening) is **functionally DONE and real-verified**
|
||||
but its own real timing has **not yet closed** (EXP-0084 — honest root
|
||||
cause: an unrelated, pre-existing compute-pipeline critical path, not the
|
||||
width change itself; see §5.4.1). See the "DONE" markers in those sections.
|
||||
The document originally analyzed the pre-fix state; it's kept below
|
||||
(marked historical) because the comparison is itself informative, then
|
||||
updated with the real post-fix numbers throughout.
|
||||
benefit, measured against the OLD 16-bit signoff — re-measurement against
|
||||
the now-closed 32-bit channel is a real next step); §5.4 (32-bit widening)
|
||||
is **fully DONE — real-verified AND real timing-closed** (EXP-0084
|
||||
functional + EXP-0086 timing closure; honest intermediate story: the first
|
||||
timing attempt failed because of a SEPARATE, stacked clock speedup, not the
|
||||
width change itself — see §5.4.1). See the "DONE" markers in those
|
||||
sections. The document originally analyzed the pre-fix state; it's kept
|
||||
below (marked historical) because the comparison is itself informative,
|
||||
then updated with the real post-fix numbers throughout.
|
||||
|
||||
---
|
||||
|
||||
@@ -38,9 +39,9 @@ DDR3 channel.
|
||||
| Real DDR3 bandwidth needed for ONE core at peak DSP throughput | **4.96 GB/s** | calculated from measured DSP rate + memory layout (§3.2) |
|
||||
| → DDR3 can sustain, pre-EXP-0081 packing (1 tile/burst) | **~25%** of one core's peak compute throughput | §3.2, historical |
|
||||
| → DDR3 can sustain, post-EXP-0081/0082 packing (2 tiles/burst, DONE) | **~50%** of one core's peak compute throughput | §3.2, current, real |
|
||||
| Real P&R timing margin (WNS), current trustworthy (16-bit) | **+0.073 ns** | measured, EXP-0083 real P&R |
|
||||
| Real P&R timing, 32-bit widened config (EXP-0084) | **-0.618 ns (FAILS)** | measured, real P&R — real root cause is an unrelated, pre-existing compute-pipeline path, not the width change; see §5.4.1 |
|
||||
| DDRManager phase 1 (single-slot look-ahead prefetch) real benefit | **2.86%** reduction in total real simulated time | measured, real xsim A/B on `tb_n2_system_ddr3.v` (§5.2, EXP-0083) — modest, honestly reported, not oversold |
|
||||
| Real P&R timing margin (WNS), current trustworthy (32-bit, CLOSED) | **+0.096 ns** | measured, EXP-0086 real P&R |
|
||||
| Real DDR3 physical bandwidth ceiling, 32-bit channel (closed timing) | **~2.48 GB/s** | measured, real P&R signoff, EXP-0086 — 2× EXP-0083's 16-bit ~1.24 GB/s, real not projected |
|
||||
| DDRManager phase 1 (single-slot look-ahead prefetch) real benefit | **2.86%** reduction in total real simulated time | measured, real xsim A/B on `tb_n2_system_ddr3.v` (§5.2, EXP-0083) — measured against the OLD 16-bit signoff; modest, honestly reported, not oversold; re-measurement against the closed 32-bit channel is a real next step |
|
||||
| DSP48E1 headroom for scaling | 224/240 free (93%) | measured, real P&R utilization |
|
||||
|
||||
The DSP headroom is real and large. The memory-bandwidth ceiling is real,
|
||||
@@ -50,16 +51,18 @@ throughput, with the real P&R margin *improving*, not degrading, as a side
|
||||
effect. This was free leverage and it's now banked.
|
||||
|
||||
Even at ~50%, DDR3 is still the limiting resource, not DSP count. The 32-bit
|
||||
channel widening (§5.4) is **functionally real and verified** (real xsim
|
||||
against the real 2-chip DDR3 model), and width alone already delivers the
|
||||
full intended 2× physical bandwidth gain (1.24 → ~2.48 GB/s) — but its real
|
||||
P&R does not yet close timing, because the SAME wizard session also sped up
|
||||
`ui_clk` by 11.2%, which an unrelated, pre-existing compute-pipeline path
|
||||
can't currently absorb (§5.4.1). Width and clock rate turned out to be
|
||||
separable levers; the path forward is reverting just the clock period. The
|
||||
DDRManager (§5.2) remains complementary — hides latency, doesn't raise the
|
||||
physical ceiling — and its own real benefit should be re-measured once the
|
||||
wider channel's timing actually closes.
|
||||
channel widening (§5.4) is now **fully real, verified, AND timing-closed**
|
||||
(real xsim against the real 2-chip DDR3 model, plus a real closed P&R at
|
||||
WNS=+0.096ns, EXP-0086) — width alone delivers the full intended 2×
|
||||
physical bandwidth gain (1.24 → ~2.48 GB/s), with real, closed timing, not
|
||||
a projection. The honest intermediate story: the first real P&R attempt
|
||||
stacked an ADDITIONAL clock speedup (`ui_clk` +11.2%) in the same wizard
|
||||
session, which an unrelated, pre-existing compute-pipeline path couldn't
|
||||
absorb (§5.4.1) — width and clock rate turned out to be separable levers,
|
||||
and reverting just the clock period closed timing with the width gain
|
||||
intact. The DDRManager (§5.2) remains complementary — hides latency,
|
||||
doesn't raise the physical ceiling — and its own real benefit should now be
|
||||
re-measured against this closed, wider channel (not yet done).
|
||||
|
||||
---
|
||||
|
||||
@@ -73,8 +76,9 @@ wider channel's timing actually closes.
|
||||
| 0078 | + config-flash bridge (real STARTUPE2 placement) | +0.013 | 5213 | 16 | margin dropped — real added logic |
|
||||
| 0079 | + real activation-fetch engine (`act_tile_fetch.v`) | +0.030 | 5379 | 16 | pre-packing baseline |
|
||||
| 0082 | + denser activation packing (2 tiles/burst, EXP-0081) | +0.068 | 5437 | 16 | pre-DDRManager baseline |
|
||||
| 0083 | + DDRManager phase 1 (`ddr_prefetch_mgr.v`, single-slot look-ahead prefetch) | **+0.073** | 5644 | 16 | current, trustworthy 16-bit/155.039MHz signoff |
|
||||
| 0084 | 32-bit DDR3 widening (2 chips) + clock speedup to 172.414MHz | **-0.618 (FAILS)** | 6418 | 16 | functionally verified (real xsim), real timing NOT closed — root cause is the clock speedup, not the width; see §5.4.1. Not the current trustworthy number. |
|
||||
| 0083 | + DDRManager phase 1 (`ddr_prefetch_mgr.v`, single-slot look-ahead prefetch) | +0.073 | 5644 | 16 | 16-bit/155.039MHz — superseded by EXP-0086 below |
|
||||
| 0084 | 32-bit DDR3 widening (2 chips) + clock speedup to 172.414MHz | -0.618 (FAILED) | 6418 | 16 | functionally verified (real xsim), real timing NOT closed — root cause is the clock speedup, not the width; see §5.4.1. |
|
||||
| 0086 | 32-bit DDR3 widening, clock period reverted to 3225ps/155.039MHz | **+0.096 (CLOSED)** | 6382 | 16 | **current, trustworthy signoff** — real, closed timing, full 2× real bandwidth (§5.4.1). Also uncovered and fixed a real recurrence of the stale-import bug (whole-tree, triggered by MIG regeneration) — see CLAUDE.md. |
|
||||
|
||||
**Observation**: WNS does not move monotonically with LUT count (0.056 →
|
||||
0.013 → 0.030 → 0.068 → 0.073 while LUTs only ever grow) — this is normal P&R behavior
|
||||
@@ -399,7 +403,7 @@ never as N-scaled literal top-level pins again. Same architectural shape as
|
||||
the weight-fetch path, in reverse (write instead of read) — a reasonable,
|
||||
bounded scope, and a real prerequisite, not optional polish.
|
||||
|
||||
### 5.4 [Decided, RTL done] Widening the physical DDR3 channel: 32-bit single channel vs. a second independent 16-bit channel — **functionally complete (EXP-0084), real timing pending**
|
||||
### 5.4 [DONE] Widening the physical DDR3 channel: 32-bit single channel vs. a second independent 16-bit channel — **functionally complete AND real timing closed (EXP-0084 → EXP-0086)**
|
||||
|
||||
**The real question**: §5.1 halved *waste* against a fixed 1.24 GB/s
|
||||
physical ceiling; it did not raise the ceiling itself. Getting past ~50%
|
||||
@@ -452,7 +456,7 @@ in the same wizard session, since both require the wizard's own JEDEC/PLL
|
||||
calculator — never hand-edited, per this project's own established
|
||||
discipline). **Done, 2026-09-20.**
|
||||
|
||||
#### 5.4.1 Real outcome (EXP-0084): functionally complete, timing NOT yet closed — the width/clock-speed decoupling
|
||||
#### 5.4.1 Real outcome (EXP-0084 → EXP-0086): functionally complete AND timing closed — the width/clock-speed decoupling
|
||||
|
||||
**Confirmed correct, as predicted above**: the widened DQ/DQS/DM pins
|
||||
stayed within banks 34/35 (no bank 14/15 conflict for the DDR3 signals
|
||||
@@ -483,17 +487,31 @@ levers, and this experiment shows they don't have to be exercised together.
|
||||
Bandwidth = width × clock — 32-bit width **alone**, even at the OLD,
|
||||
already-timing-proven 155.039MHz, already delivers the full intended 2×
|
||||
bandwidth gain (1.24 → ~2.48 GB/s). The clock speedup was free upside *if*
|
||||
the compute datapath could absorb it; real measurement shows it currently
|
||||
can't, without a real, separate re-pipelining of that accumulation tree —
|
||||
out of scope for a channel-width task, not attempted here.
|
||||
the compute datapath could absorb it; real measurement showed it couldn't
|
||||
without a real, separate re-pipelining of that accumulation tree — so it
|
||||
was reverted rather than pursued, keeping the task scoped to channel width.
|
||||
|
||||
**Current real status**: RTL fully adapted and functionally verified (see
|
||||
`hardware/v2/logs/experiments.log` EXP-0084 for the complete story,
|
||||
including 5 real P&R/XDC bugs found and fixed). Real timing signoff for
|
||||
this configuration is **pending** — next real step is reverting Input
|
||||
Clock Period toward 3225ps (keeping Data Width=32) in one more real,
|
||||
user-gated MIG wizard session. The current real, trustworthy P&R signoff
|
||||
remains EXP-0083 (16-bit, 155.039MHz, +0.073ns) until then.
|
||||
**Real closure (EXP-0086)**: the user reverted Input Clock Period back to
|
||||
3225ps (keeping Data Width=32) via a second real MIG wizard session. This
|
||||
regeneration itself triggered a real, wholesale recurrence of the
|
||||
stale-import bug — Vivado silently re-imported the entire v3 RTL tree (9
|
||||
files) plus the top XDC back to their pre-EXP-0084 state, reproducing the
|
||||
already-fixed VCCO/PROHIBIT failures until root-caused (via the actual P&R
|
||||
log, not guessed) and fixed again by re-establishing direct-reference
|
||||
sources (see CLAUDE.md's stale-import lesson, now extended to cover this).
|
||||
Once fixed, the real P&R closed cleanly: **WNS=+0.095707ns, WHS=+0.036275ns,
|
||||
0 failing endpoints, "All user specified timing constraints are met."** The
|
||||
`clk_pll_i`/155.039MHz domain — the exact domain that failed at -0.618ns in
|
||||
the 2900ps attempt — closes at +0.096ns across 24522 endpoints, matching
|
||||
EXP-0083's own 16-bit-era margin (+0.073ns) closely, confirming the width
|
||||
change itself was never the real problem.
|
||||
|
||||
**Current real status**: fully DONE — RTL functionally verified (real xsim)
|
||||
AND real P&R timing-closed (EXP-0086). This is now the project's current,
|
||||
trustworthy signoff, replacing EXP-0083. `data_ready_n` (EXP-0085) also
|
||||
confirmed placed and routed cleanly (D14/LVCMOS33) in this same run. See
|
||||
`hardware/v2/logs/experiments.log` EXP-0084 and EXP-0086 for the complete
|
||||
story, including all real P&R/XDC bugs found and fixed along the way.
|
||||
|
||||
### 5.5 Scaling path recommendation (real numbers, not a guess) — updated per user's final directive
|
||||
|
||||
@@ -515,16 +533,17 @@ specifically to document where/how it breaks rather than to succeed):
|
||||
**DONE** (EXP-0081/0082). §5.3 remains a genuine blocker for N>2 and must
|
||||
land before any scaling test that needs real result data out of more than
|
||||
2 cores' worth of pins.
|
||||
2. §5.4 (32-bit channel widening) — **decided**, user-gated on a real
|
||||
interactive MIG wizard session (Data Width + Input Clock Period changed
|
||||
together). Doubles the physical ceiling itself, which §5.1 alone could
|
||||
not do.
|
||||
2. §5.4 (32-bit channel widening) — **DONE** (EXP-0084 functional +
|
||||
EXP-0086 real timing closure, WNS=+0.096ns). Doubles the physical
|
||||
ceiling itself (real, closed, ~2.48 GB/s), which §5.1 alone could not do.
|
||||
3. §5.2 (DDRManager) — phase 1 (single-slot look-ahead prefetch) is
|
||||
**DONE** (EXP-0083, real but modest ~2.9% benefit on the still-16-bit
|
||||
channel). Re-measure this SAME real A/B once §5.4 lands, since a wider
|
||||
channel may change how much idle-channel time there is left to fill —
|
||||
only build the larger multi-slot scheduler version if that re-measurement
|
||||
justifies it, not on the original (now-corrected) hypothesis alone.
|
||||
**DONE** (EXP-0083, real but modest ~2.9% benefit, measured on the OLD
|
||||
16-bit channel). **Next real step**: re-measure this SAME real A/B
|
||||
against the now-closed 32-bit channel (EXP-0086), since a wider channel
|
||||
may change how much idle-channel time there is left to fill — only
|
||||
build the larger multi-slot scheduler version if that re-measurement
|
||||
justifies it, not on the original (now-corrected) hypothesis alone. Not
|
||||
yet done.
|
||||
4. Real N=2/4/8/16 tests, each with its own real P&R signoff (margin is
|
||||
thin, §2 — do not assume a prior N's timing closure predicts the next).
|
||||
N=8 is the real target configuration; N=16 is expected to expose real
|
||||
@@ -669,7 +688,7 @@ design task, not yet scoped, and out of this repository's own RTL scope
|
||||
|
||||
| Claim | Status |
|
||||
|---|---|
|
||||
| WNS/WHS/LUT/DSP numbers throughout | **Measured** (real Vivado P&R reports, current: EXP-0083) |
|
||||
| WNS/WHS/LUT/DSP numbers throughout | **Measured** (real Vivado P&R reports, current: EXP-0086) |
|
||||
| DDR3 back-to-back burst throughput (1.24 GB/s) | **Measured** (real `ddr3_model.sv` JEDEC trace) — physical channel limit, unchanged by §5.1's packing fix |
|
||||
| Real Activate→Read latency (16.125 ns) | **Measured** (same trace) |
|
||||
| Per-core compute throughput (2.48 GMAC/s) | **Calculated** from measured Fmax (155.039MHz) + known, fixed DSP-packing factor |
|
||||
@@ -678,6 +697,6 @@ design task, not yet scoped, and out of this repository's own RTL scope
|
||||
| "~25% of peak sustainable" (pre-packing) / "~50%" (post-packing, current) | **Calculated** ratios; post-packing figure re-verified against real P&R (EXP-0082) and real simulation (§5.1) |
|
||||
| I/O bank/DQS pin counts for XC7A100T-CSG324 (banks 14/15/16/34/35) | **Measured** — queried directly from the real Vivado part database for this exact part/package, used in §5.4's dual-channel-vs-widening analysis |
|
||||
| Row-switch penalty as a fraction of real workloads | **Not measured** — depends on host-chosen memory layout, flagged as an open question, not asserted |
|
||||
| DDRManager phase-1 real stall-reduction benefit (2.86%) | **Measured** — real xsim A/B on `tb_n2_system_ddr3.v`, real DDR3 backend, before vs after `ddr_prefetch_mgr.v` (EXP-0083). Modest, not the larger figure the original hypothesis (§5.2) suggested before it was built. |
|
||||
| DDRManager phase-1 real stall-reduction benefit (2.86%) | **Measured** — real xsim A/B on `tb_n2_system_ddr3.v`, real DDR3 backend, before vs after `ddr_prefetch_mgr.v` (EXP-0083), against the OLD 16-bit channel. Modest, not the larger figure the original hypothesis (§5.2) suggested before it was built. Re-measurement against the closed 32-bit channel (EXP-0086) is a real next step, not yet done. |
|
||||
| Full multi-slot DDRManager's real benefit | **Not measured** — not built; §5.2 recommends re-measuring phase 1 against the 32-bit-widened channel before deciding whether to build it |
|
||||
| 32-bit widening's real post-change bandwidth/timing numbers | **Not measured** — requires the user's own interactive MIG wizard session (§5.4); this document's ~2.48 GB/s figure is a doubling projection, not yet re-verified by real P&R/simulation |
|
||||
| 32-bit widening's real post-change bandwidth/timing numbers | **Measured** — real, closed P&R signoff (EXP-0086): WNS=+0.096ns, WHS=+0.036ns, 0 failing endpoints. The ~2.48 GB/s figure is now a real, closed-timing result, not a projection. |
|
||||
|
||||
Reference in New Issue
Block a user