Adds neural_director_grouped.v, a direct extension of neural_director_ packed.v's own already-proven 2-position pairing discipline to 8-position octets (matching systolic_group.v's fixed 4 PEs x 2 lanes). Real, deliberate finding: the host-facing SPI/WRITE_JOB submission protocol needs zero changes -- the host just submits 8 jobs sharing a weight base instead of 2, the same real pattern already required today. Real out-of-context synthesis of one systolic_group.v: 32 DSP48E1 (13.3%), confirming the original brainstorm's own DSP projection exactly. Found and fixed two real bugs: (1) a wraparound-arithmetic width bug in the octet index computation (same class already flagged for address math elsewhere in this project -- needs N+1 bits before the mod-reduce compare, not N); (2) a real, generalizable testbench race -- driving stimulus on the same clock edge the DUT samples on works fine with a natural gap between pulses (every prior testbench in this project has one) but silently double-registers data when called back-to-back with zero gap, confirmed via real signal tracing. Fixed with @(negedge clk) stimulus; CLAUDE.md's existing blocking/nonblocking testbench-race lesson extended to cover this new trigger. Verified via tb_neural_director_grouped.v: 4/4 PASS (octet dispatch + per-PE addressing, stall-not-mis-dispatch on a mismatched octet, queue wraparound). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUG92aM9m68TRc4rG55BcC
838 lines
52 KiB
Markdown
838 lines
52 KiB
Markdown
# FPGA-Neural V3 — Architecture Analysis: Timing, Bottlenecks, and Recommended Interventions
|
||
|
||
Scope: the current, real, P&R-verified V3 design (`hardware/v3/`, branch
|
||
`v3-artix7`), updated through EXP-0088 (result-writeback engine, real,
|
||
closed timing). The current real, trustworthy *timing* signoff is now
|
||
EXP-0088 (32-bit, 155.039MHz, WNS +0.100ns), replacing EXP-0086's own
|
||
pointer (which itself replaced EXP-0083's 16-bit-era one). Every number
|
||
in this document is either
|
||
directly measured (real simulation trace, real P&R report) or a calculation
|
||
built from directly-measured building blocks — the two are labeled
|
||
explicitly throughout. Nothing here is guessed.
|
||
|
||
**Status note (post EXP-0087)**: §5.1 (denser activation packing) is
|
||
**DONE and real-P&R-verified** (EXP-0081/0082); §5.2 (DDRManager) phase 1
|
||
is **DONE, built, and RE-MEASURED against the closed 32-bit channel**
|
||
(EXP-0083's original modest ~2.9% real benefit was measured only at the
|
||
OLD 16-bit signoff — real re-measurement, EXP-0087, found the benefit is
|
||
now ~0%/a wash at 32-bit, kept wired in for correctness/timing-neutrality
|
||
only, not performance); §5.4 (32-bit widening)
|
||
is **fully DONE — real-verified AND real timing-closed** (EXP-0084
|
||
functional + EXP-0086 timing closure; honest intermediate story: the first
|
||
timing attempt failed because of a SEPARATE, stacked clock speedup, not the
|
||
width change itself — see §5.4.1). See the "DONE" markers in those
|
||
sections. The document originally analyzed the pre-fix state; it's kept
|
||
below (marked historical) because the comparison is itself informative,
|
||
then updated with the real post-fix numbers throughout.
|
||
|
||
---
|
||
|
||
## 1. Executive summary
|
||
|
||
The single most important finding of this analysis: **the system is DDR3
|
||
memory-bandwidth-bound, not DSP-bound, already at N=1 core** — and this is
|
||
true *before* considering any core-count scaling. Adding more compute cores
|
||
(N=4/8/16) without first addressing memory bandwidth would not increase real
|
||
throughput; it would only add more cores contending for the same saturated
|
||
DDR3 channel.
|
||
|
||
| Metric | Value | Source |
|
||
|---|---|---|
|
||
| Real DDR3 back-to-back burst bandwidth (physical channel) | **1.24 GB/s** (9.92 Gbps) | measured, real JEDEC trace (§3.1) — unchanged by packing, this is a physical-channel limit |
|
||
| Real DDR3 bandwidth needed for ONE core at peak DSP throughput | **4.96 GB/s** | calculated from measured DSP rate + memory layout (§3.2) |
|
||
| → DDR3 can sustain, pre-EXP-0081 packing (1 tile/burst) | **~25%** of one core's peak compute throughput | §3.2, historical |
|
||
| → DDR3 can sustain, post-EXP-0081/0082 packing (2 tiles/burst, DONE) | **~50%** of one core's peak compute throughput | §3.2, current, real |
|
||
| Real P&R timing margin (WNS), current trustworthy (32-bit, CLOSED) | **+0.096 ns** | measured, EXP-0086 real P&R |
|
||
| Real DDR3 physical bandwidth ceiling, 32-bit channel (closed timing) | **~2.48 GB/s** | measured, real P&R signoff, EXP-0086 — 2× EXP-0083's 16-bit ~1.24 GB/s, real not projected |
|
||
| DDRManager phase 1 (single-slot look-ahead prefetch) real benefit | **~0% (0.0064% regression)** at the closed 32-bit channel | measured, real xsim A/B on `tb_n2_system_ddr3.v` (§5.2, EXP-0087) — the old 16-bit-only 2.86% figure (EXP-0083) does not survive the wider channel; kept wired in (timing-neutral, real P&R already signs off with it) but no longer a real performance win |
|
||
| DSP48E1 headroom for scaling | 224/240 free (93%) | measured, real P&R utilization |
|
||
|
||
The DSP headroom is real and large. The memory-bandwidth ceiling is real,
|
||
and **denser activation packing (§5.1) has already doubled the real
|
||
achievable fraction of it** — from ~25% to ~50% of one core's peak DSP
|
||
throughput, with the real P&R margin *improving*, not degrading, as a side
|
||
effect. This was free leverage and it's now banked.
|
||
|
||
Even at ~50%, DDR3 is still the limiting resource, not DSP count. The 32-bit
|
||
channel widening (§5.4) is now **fully real, verified, AND timing-closed**
|
||
(real xsim against the real 2-chip DDR3 model, plus a real closed P&R at
|
||
WNS=+0.096ns, EXP-0086) — width alone delivers the full intended 2×
|
||
physical bandwidth gain (1.24 → ~2.48 GB/s), with real, closed timing, not
|
||
a projection. The honest intermediate story: the first real P&R attempt
|
||
stacked an ADDITIONAL clock speedup (`ui_clk` +11.2%) in the same wizard
|
||
session, which an unrelated, pre-existing compute-pipeline path couldn't
|
||
absorb (§5.4.1) — width and clock rate turned out to be separable levers,
|
||
and reverting just the clock period closed timing with the width gain
|
||
intact. The DDRManager (§5.2) was re-measured against this now-closed,
|
||
wider channel (EXP-0087): its real benefit is gone (~0%, a 0.0064%
|
||
regression) — the wider channel's own lower per-tile latency already
|
||
absorbed the gap the look-ahead used to hide.
|
||
|
||
---
|
||
|
||
## 2. Real signoff history (all real P&R runs to date)
|
||
|
||
| EXP | What changed | WNS (ns) | LUTs | DSP48E1 | Notes |
|
||
|---|---|---|---|---|---|
|
||
| 0059 | isolated packed core, out-of-context | n/a (isolated) | 507 | 8 | first real P&R, out-of-context only |
|
||
| 0074 | first real in-context P&R: DDR3 + pins + register file not yet added | +0.040 | 5140 | 16 | first trustworthy board-accurate number |
|
||
| 0076 | + register file, + pin constraints, + SPI physical-layer fix | +0.056 | 5173 | 16 | margin improved slightly (P&R is not perfectly monotonic run to run) |
|
||
| 0078 | + config-flash bridge (real STARTUPE2 placement) | +0.013 | 5213 | 16 | margin dropped — real added logic |
|
||
| 0079 | + real activation-fetch engine (`act_tile_fetch.v`) | +0.030 | 5379 | 16 | pre-packing baseline |
|
||
| 0082 | + denser activation packing (2 tiles/burst, EXP-0081) | +0.068 | 5437 | 16 | pre-DDRManager baseline |
|
||
| 0083 | + DDRManager phase 1 (`ddr_prefetch_mgr.v`, single-slot look-ahead prefetch) | +0.073 | 5644 | 16 | 16-bit/155.039MHz — superseded by EXP-0086 below |
|
||
| 0084 | 32-bit DDR3 widening (2 chips) + clock speedup to 172.414MHz | -0.618 (FAILED) | 6418 | 16 | functionally verified (real xsim), real timing NOT closed — root cause is the clock speedup, not the width; see §5.4.1. |
|
||
| 0086 | 32-bit DDR3 widening, clock period reverted to 3225ps/155.039MHz | +0.096 (CLOSED) | 6382 | 16 | real, closed timing, full 2× real bandwidth (§5.4.1). Also uncovered and fixed a real recurrence of the stale-import bug (whole-tree, triggered by MIG regeneration) — see CLAUDE.md. Superseded by 0088 below. |
|
||
| 0088 | + result-writeback engine (`result_writeback.v`, removes the last hard N-scaling pin blocker, §4.6/§5.3) | **+0.100 (CLOSED)** | 6642 | 16 | **current, trustworthy signoff** — essentially unchanged margin, zero real timing cost for closing the pin-scaling blocker. |
|
||
|
||
**Observation**: WNS does not move monotonically with LUT count (0.056 →
|
||
0.013 → 0.030 → 0.068 → 0.073 while LUTs only ever grow) — this is normal P&R behavior
|
||
(placer/router heuristics find different solutions each run, small logic
|
||
changes can shift which path is critical). **Do not extrapolate a trend
|
||
line from 3-4 data points** — the only safe practice is a fresh real P&R
|
||
after every real change, which this project already does.
|
||
|
||
**DSP48E1 has stayed at 16 across every real change since EXP-0059's core
|
||
design was fixed** — confirms the packed-DSP MAC design (§4.1) is the
|
||
efficient, stable part of this architecture; all the margin pressure has
|
||
come from *control/glue logic* (arbitration, the flash bridge, the
|
||
activation engine's FSM), not from the compute datapath itself.
|
||
|
||
---
|
||
|
||
## 3. The real memory-bandwidth bottleneck (the analysis's central finding)
|
||
|
||
### 3.1 Real measured DDR3 throughput
|
||
|
||
From the actual `ddr3_model.sv` JEDEC command trace captured during the
|
||
EXP-0079 real simulation run (`tb_n2_system_ddr3.v` via `xsim`), two
|
||
back-to-back `Read` commands to the same open row:
|
||
|
||
```
|
||
Read bank 0 col 000 @ 7090615 ps
|
||
Read bank 0 col 000 @ 7103515 ps (delta = 12900 ps = 12.9 ns)
|
||
```
|
||
|
||
One `BURST_LEN=8` transaction moves 8 × 16 bits = 128 bits. Real measured
|
||
throughput for same-row back-to-back bursts:
|
||
|
||
```
|
||
128 bits / 12.9 ns = 9.92 Gbps = 1.24 GB/s
|
||
```
|
||
|
||
This exactly matches the *theoretical* peak for a 16-bit DDR3 interface at
|
||
310.078 MHz (16 bits × 2 (DDR) × 310.078 MHz = 9.92 Gbps) — confirming the
|
||
real controller achieves its theoretical ceiling for the best case
|
||
(same-row, no row switches). This is the **best-case** number; anything
|
||
requiring a row change (Activate/Precharge) is real-measured to cost more
|
||
(§3.3).
|
||
|
||
### 3.2 Real compute-side bandwidth requirement
|
||
|
||
Each packed core (EXP-0059's design, unchanged since) uses 8 DSP48E1, each
|
||
computing 2 packed INT8 MACs per cycle (lane A + lane B sharing one resident
|
||
weight) = 16 MACs/cycle/core. At the real measured 155.039 MHz compute clock:
|
||
|
||
```
|
||
16 MACs/cycle × 155.039 × 10^6 cycles/s = 2.48 GMAC/s per core (real, from measured Fmax)
|
||
```
|
||
|
||
Each compute cycle consumes 1 activation byte per MAC lane (16 bytes total:
|
||
8 for lane A, 8 for lane B).
|
||
|
||
**Historical (EXP-0079, "1 tile = 1 burst")**: each tile fetch moved a full
|
||
16-byte burst for only 8 useful bytes:
|
||
|
||
```
|
||
2 tiles (A+B) × 16 bytes/burst = 32 bytes moved DDR3 traffic → 16 MACs
|
||
= 2 real DDR3 bytes moved per MAC operation
|
||
2.48 GMAC/s × 2 bytes/MAC = 4.96 GB/s needed per core
|
||
```
|
||
|
||
**Current (EXP-0081/0082, "2 tiles = 1 burst", DONE and real-P&R-verified)**:
|
||
two tiles now share one 16-byte burst, halving the moved-bytes-per-useful-byte
|
||
ratio:
|
||
|
||
```
|
||
2 tiles (A+B) × 16 bytes/burst ÷ 2 tiles-per-burst = 16 bytes moved DDR3 traffic → 16 MACs
|
||
= 1 real DDR3 byte moved per MAC operation (halved)
|
||
2.48 GMAC/s × 1 byte/MAC = 2.48 GB/s needed per core (halved)
|
||
```
|
||
|
||
### 3.3 The real gap
|
||
|
||
```
|
||
Historical: 1.24 GB/s available vs 4.96 GB/s needed per core → ~25% sustainable
|
||
Current: 1.24 GB/s available vs 2.48 GB/s needed per core → ~50% sustainable
|
||
```
|
||
|
||
The physical channel ceiling (1.24 GB/s, §3.1) did **not** change — §5.1's
|
||
fix reduced waste against a fixed ceiling, it did not raise the ceiling
|
||
itself. Even at ~50%, DDR3 remains the binding constraint, not DSP count,
|
||
even in the BEST case (zero row-switch overhead, one core, nothing else
|
||
sharing the bus). Raising the *physical* ceiling requires a wider channel
|
||
(§5.5) or a second channel — both analyzed below.
|
||
|
||
This ceiling gets **worse**, not better, with:
|
||
- **Row switches**: real measured Activate→Read latency is 16.125 ns
|
||
(5 DDR3 clock cycles at 3.225 ns = real CAS-latency-5 timing, matches the
|
||
MIG's own configured CL=5). A tile fetch that requires a fresh row
|
||
activation costs ~16-30 ns instead of the 12.9 ns same-row case — a
|
||
real, measured 25-130% penalty per row switch.
|
||
- **More cores sharing the one DDR3 channel** (N=2 today, N=4/8/16
|
||
proposed): the 1.24 GB/s ceiling is shared across ALL active requesters,
|
||
not per-core. Adding cores divides an already-insufficient budget further.
|
||
- **Weight fetching** (amortized, but not free): each job pair's weight
|
||
prefetch (8 bursts for a 128-byte layer) adds real DDR3 traffic on top of
|
||
activation fetching, though this cost is shared across the M reuse
|
||
positions and becomes negligible for large M.
|
||
|
||
**Conclusion**: this was a genuine architectural ceiling, not a tuning
|
||
problem, and half of it has now been recovered for free. The 2× byte-
|
||
overhead from the original "1 tile = 1 full burst" convention (chosen in
|
||
EXP-0079 specifically to avoid a runtime-indexed part-select, given the
|
||
then-already-thin timing margin) was confirmed, with real numbers, to be the
|
||
single most expensive design decision in the memory path — and has since
|
||
been fixed (§5.1, DONE, EXP-0081/0082) by registering the select bit at
|
||
request time instead of avoiding the select entirely. The remaining gap
|
||
(~50% sustainable, not 100%) is now a *physical channel width* problem, not
|
||
a packing-waste problem — see §5.5.
|
||
|
||
---
|
||
|
||
## 4. Module-by-module review
|
||
|
||
### 4.1 Compute core (`mac2_dsp_packed.v`, `neural_processor_packed.v`)
|
||
|
||
Real, stable, efficient. 8 DSP48E1/core unchanged since EXP-0059. The
|
||
2-INT8-MAC-per-DSP48E1 packing technique is the correct choice for this INT8
|
||
workload — real utilization (16/240 DSP = 6.67% at N=2) confirms there is no
|
||
DSP-side pressure at all; **all scaling headroom is here, and all scaling
|
||
risk is elsewhere** (§3, §4.5).
|
||
|
||
No real change recommended here. This is the part of the design that is
|
||
*not* the bottleneck.
|
||
|
||
### 4.2 Weight-reuse path (`layer_prefetch_ctrl.v`, `layer_weight_buffer.v`,
|
||
`weight_tile_gather.v`)
|
||
|
||
Reused unmodified from V2 (ECP5 era), real and well-verified across many
|
||
EXPs (0057/0058/0061/0062 and every integration test since). Correctly
|
||
amortizes DDR3 traffic across M reuse positions — this part of the design
|
||
already does the "fetch once, use many times" optimization the activation
|
||
path currently lacks (§5.1's recommendation follows the SAME philosophy).
|
||
|
||
No real change recommended; this module is a good template for how the
|
||
activation path should evolve.
|
||
|
||
### 4.3 Activation-fetch path (`act_tile_fetch.v`, EXP-0079, updated EXP-0081/0082)
|
||
|
||
Real, correct (verified 3 levels deep, §2 of EXP-0079's own log entry; the
|
||
EXP-0081 packing change re-verified at all 3 levels again — `tb_act_tile_fetch.v`
|
||
8/8, `tb_packed_slot.v` 9/9, `tb_n2_system_ddr3.v` 8/8, plus real P&R). Two
|
||
real design choices identified in this analysis:
|
||
|
||
1. **1 tile = 1 full burst (2× byte overhead)** — **FIXED (EXP-0081/0082,
|
||
DONE)**: now 2 tiles share 1 burst via a request-time-registered select
|
||
bit (`sel_lat`), avoiding the runtime-indexed-part-select Fmax risk while
|
||
still halving DDR3 waste. Real P&R confirms margin improved, not
|
||
degraded (+0.030ns → +0.068ns). See §5.1.
|
||
2. **Lane A then lane B, sequential, per tile**: still doubles the real
|
||
number of DDR3 transactions (and row-switch risk) versus a design that
|
||
could fetch both lanes in a single wider transaction when they happen to
|
||
be adjacent in memory. Not changed — flagged for future work; the
|
||
DDRManager (§5.2) and 32-bit channel widening (§5.5) are higher-leverage
|
||
and were prioritized first per the user's explicit direction.
|
||
|
||
### 4.4 Scheduling (`neural_director_packed.v`) and arbitration
|
||
(`sdram_arbiter_n.v`)
|
||
|
||
Real, correct, and — importantly for §5.2 — **`neural_director_packed.v`
|
||
already maintains a real queue of pending jobs** (`QUEUE_DEPTH=8`,
|
||
`q_x_base`/`q_w_base`/etc arrays). This is directly relevant to the
|
||
DDRManager/prefetch proposal (§5.2): the information needed to "know what's
|
||
coming next" already exists in this module, it just isn't currently used for
|
||
anything beyond pairing/dispatch decisions.
|
||
|
||
`sdram_arbiter_n.v`'s combinational-first-grant design (EXP-0066) is real,
|
||
proven, and already generalized to N-way (verified at N=3, EXP-0069) — no
|
||
real change needed to scale its own requester count for a DDRManager
|
||
addition or for N=4/8/16 scaling, though the arbiter's own **fairness
|
||
policy** (lowest-index-wins, a deliberate simplicity choice, EXP-0066) would
|
||
need re-examination if a DDRManager starts issuing speculative/anticipatory
|
||
requests that could starve a real, urgent request — see §5.2's own caveat.
|
||
|
||
### 4.5 Host interface (`spi_host_bridge_v3.v`, `flash_spi_master.v`,
|
||
`host_mem_bridge.v`)
|
||
|
||
Real, verified, low resource cost, not on any critical performance path
|
||
(host commands are inherently much slower than the internal compute/memory
|
||
loop). No bottleneck here. Not a scaling concern.
|
||
|
||
### 4.6 [DONE, EXP-0088] Result-writeback engine
|
||
|
||
**No longer absent.** `result_writeback.v` (new module) is instantiated
|
||
inside each `packed_slot.v`, sharing that slot's own ctrl port with
|
||
`layer_prefetch_ctrl.v`/`ddr_prefetch_mgr.v` exactly the way those two
|
||
already share it with each other (mutually exclusive in time by FSM
|
||
construction — writeback only starts in `S_RESULT`, strictly after the
|
||
tile loop has finished). Each completed job's result is written directly
|
||
into DDR3 at the job's own `result_addr_a`/`result_addr_b` — the SAME
|
||
architectural shape as the weight-fetch path, in reverse, exactly as this
|
||
section used to recommend before it was built. `job_done` now means "the
|
||
result is durably in DDR3", not merely "captured in a register only a
|
||
literal top-level pin could see". `n2_system_ddr3_top.v`'s own
|
||
`s0_result_data_a/b`/`s1_result_data_a/b` top-level PACKAGE PINS are
|
||
**removed** — the real, hard N-scaling blocker this section used to flag
|
||
(8 bits × 2 lanes × N cores → 256 pins at N=16) no longer exists at any N.
|
||
See §5.3 for the full real design/verification story.
|
||
|
||
---
|
||
|
||
## 5. Recommended interventions, ranked by real leverage
|
||
|
||
### 5.1 [Highest leverage] Denser activation packing — attack the real bandwidth ceiling directly — **DONE (EXP-0081/0082)**
|
||
|
||
**What was built**: 2 tiles (even/odd tile index) now share one
|
||
`BURST_LEN=8` burst instead of one tile per burst, halving real DDR3
|
||
bytes-per-MAC from 2 to 1. Real, measured effect: the achievable fraction
|
||
of one core's peak DSP throughput rose from ~25% to ~50% (§3.2/§3.3,
|
||
current numbers).
|
||
|
||
**Why this was avoided in EXP-0079**: doing so naively requires a
|
||
runtime-indexed part-select (which half of the burst response to use,
|
||
selected by a runtime tile-index bit) — the same anti-pattern
|
||
`weight_tile_gather.v` (EXP-0061) already flagged as a real Fmax risk, and
|
||
the P&R margin was already thin (+0.013ns) when that original design
|
||
decision was made.
|
||
|
||
**The safe path that was actually built**: the tile-index LSB is registered
|
||
into `sel_lat` at *request* time (`S_IDLE`, the same cycle `tcnt` is
|
||
latched) — many `ui_clk` cycles before the real DDR3 round-trip completes
|
||
and `ctrl_rdata` becomes valid. The eventual data-select mux therefore
|
||
selects on an already-long-stable registered bit, never one racing the read
|
||
data. **Real P&R confirms this is genuinely timing-safe, not just
|
||
functionally correct**: margin *improved* from +0.030ns to +0.068ns
|
||
(EXP-0082), despite the added mux logic (LUTs 5379→5437). Full detail:
|
||
`hardware/v3/rtl/act_tile_fetch.v` header, `docs/PHYSICAL_REALIZATION.md` §4,
|
||
`hardware/v2/logs/experiments.log` EXP-0081/EXP-0082.
|
||
|
||
**Verification**: `tb_act_tile_fetch.v` (8/8 PASS, covers even/odd-in-same-
|
||
burst, new-burst crossing, back-to-back alternation), `tb_packed_slot.v`
|
||
(9/9 PASS, bit-identical numeric results to the pre-change run),
|
||
`tb_n2_system_ddr3.v` (8/8 PASS, real xsim against real `ddr3_model.sv`,
|
||
JEDEC trace confirmed to show no more half-burst zero-padding).
|
||
|
||
### 5.2 [Complementary, addresses latency not bandwidth] DDRManager with orchestrator-driven prefetch (user's proposal) — **phase 1 DONE (EXP-0083); real benefit at the closed 32-bit channel is now ZERO (EXP-0087)**
|
||
|
||
**The original hypothesis** (written before building anything, now corrected
|
||
by real measurement — kept here so the correction is visible, not silently
|
||
edited away): the CURRENT design only ever requests a tile the moment
|
||
`packed_slot.v`'s own FSM reaches `S_TILEREQ` for it, so a DDRManager that
|
||
issues tile N+1's fetch WHILE tile N is still being consumed should hide
|
||
"today's design is very likely stalling ... for the majority of real time".
|
||
|
||
**What the real EXP-0083 measurement actually found**: this hypothesis
|
||
overstated the achievable benefit, for a specific, now-confirmed reason —
|
||
`neural_processor_packed.v`'s own pipeline accepts one operand **per cycle**
|
||
whenever it's in `NP_WAIT_OPERANDS` (`operand_ready` is state-only, not
|
||
gated on any internal pipeline stall). The old design's real "dead time"
|
||
between one tile's fetch completing and the next one's fetch being issued
|
||
was therefore only the ~2-cycle request/consume handshake overhead
|
||
(`S_TILEREQ` + `S_OPERAND`), not a large compute-bound stall — and that
|
||
small overhead is what look-ahead prefetch can actually remove, not the
|
||
DDR3 fetch latency itself (which is dominated by row activation/precharge,
|
||
§3.3, and look-ahead cannot make a single fetch faster, only start it
|
||
earlier).
|
||
|
||
**Real, measured result at the OLD 16-bit channel** (`ddr_prefetch_mgr.v`,
|
||
real P&R WNS +0.073ns, up from EXP-0082's +0.068ns, LUTs 5644, DSP48E1 16
|
||
unchanged):
|
||
- Real apples-to-apples comparison on the **real DDR3 backend**
|
||
(`tb_n2_system_ddr3.v` via real xsim, same N=2/8-position workload,
|
||
before vs after, same `ddr3_model.sv`): **2.86% reduction in total real
|
||
simulated time** (108370.88ns → 105268.43ns) — EXP-0083's own headline
|
||
number, but see EXP-0087 below: this figure does NOT survive the 32-bit
|
||
channel and should not be quoted as the current real benefit.
|
||
- On the fast SDR placeholder backend (used for isolated glue-logic
|
||
testing, `tb_ddr_prefetch_mgr.v`): 0.9% reduction in a row-switch-heavy
|
||
scenario, and -1.4% (i.e. not faster) in an isolated same-row best case —
|
||
that placeholder model's own per-fetch cost turned out to be dominated by
|
||
a near-fixed protocol cost regardless of address locality, so it doesn't
|
||
cleanly isolate the mechanism the real DDR3 backend's own row/bank timing
|
||
does. Full detail: EXP-0083 in `hardware/v2/logs/experiments.log`.
|
||
|
||
**Real RE-measurement at the closed 32-bit channel (EXP-0087) — the
|
||
benefit is now GONE**: EXP-0083's 2.86% figure was only ever measured
|
||
against the OLD 16-bit/155MHz channel. Once the 32-bit channel's own real
|
||
timing closed (EXP-0086), this project's own standing plan
|
||
("re-measure once the wider channel's timing actually closes") was
|
||
carried out: a real, fair A/B on the SAME 32-bit/3225ps config, comparing
|
||
the current, committed `packed_slot.v` (with `ddr_prefetch_mgr.v`) against
|
||
a measurement-only fork reproducing the pre-EXP-0083 direct-fetch
|
||
sequencing (`hardware/v3/sim/packed_slot_noprefetch.v`, not part of the
|
||
real synthesis target). Real result, both runs 8/8 PASS, 0 errors,
|
||
identical golden data:
|
||
- WITH `ddr_prefetch_mgr.v`: $finish at 100663.1335 ns
|
||
- WITHOUT `ddr_prefetch_mgr.v`: $finish at 100656.6835 ns
|
||
- **WITH is 6.45ns SLOWER — a 0.0064% real regression**, statistically a
|
||
wash, definitively NOT a benefit any more.
|
||
|
||
**Real, honest interpretation**: the 32-bit widening already halves the
|
||
real per-tile DDR3 round-trip latency. EXP-0083's own real finding
|
||
already showed the achievable benefit was capped by
|
||
`neural_processor_packed.v`'s fixed one-operand-per-cycle consumption
|
||
rate, not DDR3 latency, even at 16-bit — the wider channel shrinks the
|
||
real per-tile wait below whatever small gap the look-ahead could still
|
||
hide, leaving effectively nothing left to overlap. `ddr_prefetch_mgr.v`
|
||
stays wired into the real, committed `packed_slot.v` (real P&R already
|
||
signs off with it included, EXP-0086, and it causes zero real harm) —
|
||
but its justification is now "real, correct, timing-neutral", not "real
|
||
performance win". Building the larger multi-slot scheduler sketched below
|
||
is **not justified** by this real result.
|
||
|
||
**What it does NOT solve** (this part of the original reasoning holds):
|
||
§3.2's bandwidth ceiling is a hard physical limit (bytes/second the DDR3
|
||
channel can physically move) — prefetching earlier doesn't move more bytes
|
||
per second, it only avoids idle gaps. §5.1 (done) reduced bytes needed per
|
||
MAC; §5.4 (32-bit widening, decided, pending) raises the physical ceiling
|
||
itself; this phase-1 DDRManager only removes a small, now-quantified,
|
||
per-tile dead-time — real, free (zero timing cost, margin still improving),
|
||
but genuinely modest, not the larger win a first-principles estimate
|
||
suggested before it was actually built and measured.
|
||
|
||
**What was built** (`ddr_prefetch_mgr.v`, real RTL, not a sketch): wraps
|
||
`act_tile_fetch.v` (unmodified) with a depth-2 ping-pong buffer scoped to
|
||
ONE slot's own activation-tile look-ahead, exactly the validated,
|
||
scoped-first approach recommended below before this experiment ran. Bank
|
||
selection uses a registered index bit at both fill and read time, same
|
||
"known long before the data it gates" discipline as `act_tile_fetch.v`'s
|
||
own EXP-0081 layout — confirmed timing-safe by real P&R, not asserted.
|
||
|
||
**Full multi-slot / whole-Director-queue scheduler — still NOT built, and
|
||
now real-disconfirmed, not just deferred**: the re-examination against the
|
||
32-bit channel's real numbers (EXP-0087, above) has now happened, and the
|
||
result is that phase 1's own benefit is gone entirely at this core count
|
||
— there is no real cost/benefit case left for the larger design below
|
||
until a NEW real bottleneck reintroduces DDR3-latency-bound behavior
|
||
(e.g. a higher core count where the shared bus is contended again, or a
|
||
workload with heavier row-switching than this experiment's own test
|
||
pattern). Revisit only if/when N=4/8/16 scaling (§5.5) reveals DDR3
|
||
latency, not `neural_processor_packed.v`'s own consumption rate, as the
|
||
real limiting factor again.
|
||
|
||
**Concrete design sketch for the full version** (informed by what already
|
||
exists in this codebase; kept for when it's revisited):
|
||
|
||
- `neural_director_packed.v` already queues up to `QUEUE_DEPTH=8` pending
|
||
jobs, each with a known `x_base`/`w_base`/`n_tiles` — this is exactly the
|
||
"reservation" information a full DDRManager needs. No new bookkeeping is
|
||
required at the Director level; it would READ this existing queue, not
|
||
need the Director to change its own job-acceptance logic.
|
||
- A cross-slot manager would sit between `sdram_arbiter_n.v` and each
|
||
slot's own `ddr_prefetch_mgr.v`/`layer_prefetch_ctrl.v` instances,
|
||
scheduling across slots (not just within one slot's own tile loop as
|
||
phase 1 does) — e.g. prioritizing requests that share an already-open
|
||
DDR3 row across DIFFERENT slots, which phase 1 cannot see or exploit.
|
||
- **Real caveat, not glossed over**: this adds real arbitration complexity
|
||
— a prefetched-but-not-yet-consumed request competing with another slot's
|
||
genuinely urgent request needs a real priority policy, not just
|
||
`sdram_arbiter_n.v`'s current lowest-index-wins simplicity (§4.4). A
|
||
speculative prefetch that turns out to be wrong (e.g., the Director
|
||
reorders/never dispatches that queued job) also wastes real bandwidth —
|
||
needs a real cancellation/staleness mechanism, not assumed away.
|
||
|
||
### 5.3 [DONE, EXP-0088] Result-writeback engine
|
||
|
||
**Built and real-verified.** New module `result_writeback.v`, one instance
|
||
per `packed_slot.v` (matching how `layer_prefetch_ctrl.v`/`act_tile_
|
||
fetch.v`/`ddr_prefetch_mgr.v` are already one-per-slot, not a new
|
||
arbiter-requester count as N scales). On job completion (`S_RESULT`), it
|
||
writes BOTH lanes' results into DDR3 at the job's own `result_addr_a`/
|
||
`result_addr_b` and only THEN asserts `job_done` — real, correctness-first
|
||
sequencing (`job_done` now means "durably in DDR3", not "captured in a
|
||
register" the way it used to).
|
||
|
||
**Real addressing** (verified against `act_tile_fetch.v`'s/`layer_
|
||
prefetch_ctrl.v`'s own real address-computation code, not guessed):
|
||
`result_addr_a/b` arrive in `packed_slot.v`'s own `JOB_ADDR_WIDTH=26`-bit
|
||
convention; the low `ADDR_WIDTH=25` bits (dropping the unused top/MSB
|
||
headroom bit) are used directly as a ctrl-bus-native 32-bit-word address
|
||
— the exact same address space `x_base_a`/`w_base` already live in. One
|
||
full 32-bit ctrl-word is written per lane: `{node_id[15:0], 8'h00,
|
||
result_data[7:0]}`. The host reads results back via the ALREADY-EXISTING
|
||
`READ_MEM` (0x02) SPI opcode — no new protocol. Real, disclosed
|
||
host-firmware implication (not yet built, same class of gap as this
|
||
project's other disclosed firmware work, e.g. JTAG bit-banging): reading
|
||
a result back needs `mem_addr = result_addr[24:0]*2` for the value and
|
||
`+1` for node_id (2 host reads per lane), since `READ_MEM`'s own
|
||
`mem_addr` is 16-bit-word-granular while this engine writes a native
|
||
32-bit ctrl-word — the same real halving `host_mem_bridge.v`'s own header
|
||
already discloses for the debug raw-access path.
|
||
|
||
**Shared-bus discipline** (mirrors `act_tile_fetch.v`'s own real,
|
||
proven pattern, not reinvented): waits for `mem_grant` before ever
|
||
issuing `ctrl_req` (EXP-0066's own established rule — an early/blind
|
||
request on a shared bus can lose the request permanently); a real
|
||
`S_GAP` state waits for `!ctrl_busy` between lane A's write and lane
|
||
B's own, since `mig_native_adapter.v`'s own `busy` stays asserted one
|
||
cycle past `ctrl_ready`. `wmask` polarity matches `host_mem_bridge.v`'s
|
||
own real, already-working convention exactly (0 = write this byte, 1 =
|
||
masked, the same DQM-style polarity this project's whole memory stack
|
||
already uses end to end).
|
||
|
||
**Real, hard scaling blocker removed**: `n2_system_ddr3_top.v`'s own
|
||
`s0_result_data_a/b`/`s1_result_data_a/b` top-level PACKAGE PINS are
|
||
gone — each slot's `packed_slot.v` still exposes `result_data_a/b` etc.
|
||
as plain output ports (for debug/testbench visibility, unchanged), but
|
||
these are no longer wired to literal FPGA package pins at any N.
|
||
|
||
**Verification**: `tb_packed_slot.v` extended with a new real
|
||
read-after-write check (`verify_writeback` task) — after each job's
|
||
`job_done`, the testbench independently reads back the exact DDR3
|
||
location `result_writeback.v` should have written and confirms both the
|
||
result value AND node_id match, closing the loop (not just checking
|
||
`job_done` eventually pulses). **9/9 PASS, 0 errors**, real Icarus xsim
|
||
against `burst_mem_model32.v`. Also re-verified at the full N=2 system
|
||
level (`tb_n2_system_ddr3.v`, real xsim against the real, closed
|
||
32-bit/3225ps DDR3 model) to confirm the writeback engine behaves
|
||
correctly under real shared-bus arbitration contention between 2 slots
|
||
— **8/8 PASS, 0 errors**.
|
||
|
||
**Real P&R signoff**: **WNS = +0.099962ns, WHS = +0.036275ns**, 0
|
||
failing endpoints, "All user specified timing constraints are met" —
|
||
essentially unchanged from EXP-0086's own +0.095707ns (real, closed,
|
||
not a regression). 6642 LUTs (+260 vs EXP-0086, expected), 16 DSP48E1
|
||
(unchanged). This is now the current, trustworthy P&R signoff,
|
||
superseding EXP-0086/0087. See `hardware/v2/logs/experiments.log`
|
||
EXP-0088 for the full real result.
|
||
|
||
### 5.4 [DONE] Widening the physical DDR3 channel: 32-bit single channel vs. a second independent 16-bit channel — **functionally complete AND real timing closed (EXP-0084 → EXP-0086)**
|
||
|
||
**The real question**: §5.1 halved *waste* against a fixed 1.24 GB/s
|
||
physical ceiling; it did not raise the ceiling itself. Getting past ~50%
|
||
sustained DSP utilization requires more physical bytes/second, which means
|
||
either (a) widening the existing channel from 16-bit to 32-bit data width,
|
||
or (b) adding a second, independent 16-bit DDR3 channel. Both roughly
|
||
double the real 1.24 GB/s ceiling to ~2.48 GB/s. The user asked for an
|
||
honest comparison, not a diplomatically-balanced non-answer — here it is,
|
||
based on **real device data**, not guessed.
|
||
|
||
**Real device data** (queried directly from the actual Vivado part database
|
||
for this exact part/package, XC7A100T-**CSG324**): this package has only
|
||
**5 total I/O banks** — 14 (56 pins), 15 (56 pins), 16 (11 pins), 34 (56
|
||
pins), 35 (56 pins). All report `BANK_TYPE=BT_HIGH_RANGE` (Artix-7 has no
|
||
separate "HP" bank class the way some other families do). Banks **14 and
|
||
15** each expose **8 DQS-capable pin pairs** — the same memory-PHY
|
||
signature already used by the real, placed DDR3 controller on banks 34/35.
|
||
This means a second, independent DDR3 channel is *physically plausible* on
|
||
this package (the DQS-capable pins exist), but:
|
||
|
||
| Factor | 32-bit single channel | Second independent 16-bit channel |
|
||
|---|---|---|
|
||
| Real ceiling gain | ~2× (1.24 → ~2.48 GB/s) | ~2× (aggregate, same total) |
|
||
| Pin cost | Reuses/extends the existing MIG's own bank(s); no new bank claimed | Would claim banks 14 **and/or** 15 (the only banks with free DQS-capable pins) |
|
||
| **Conflict with already-placed I/O** | None | **Real, direct**: the management-SPI bus (bank 15: A15/B16/B17/A16) and the config-flash bus (bank 14: K17/K18/L13) are already placed in exactly the banks that would need to host a second channel. Only bank 16 (11 pins) would remain free — not enough margin for either SPI bus, let alone both. |
|
||
| Controller logic cost | One MIG instance, wider data path (mostly automatic — the MIG wizard regenerates CAS/CWL/MMCM ratios for the new width) | A full second MIG instance: second calibration sequence, second `ui_clk` domain, and critically the **DDRManager/arbiter would need to become channel-aware**, not just requester-aware — real added complexity on top of §5.2's own design, not a simplification |
|
||
| Real risk given current thin margin (+0.073ns) | Lower — one controller, one clock domain, incremental change to an already-proven design | Higher — two independent PHYs, two calibration state machines, cross-channel coordination logic, all new |
|
||
| PCB impact | **Corrected 2026-09-20 (real, caught mid-MIG-wizard-session, not assumed away)**: the selected DDR3 part, MT41J128M16JT-125:K, is a **x16 component** — the MIG wizard's "Data Width" field is the *controller's* aggregate width, not the chip's own width, so Data Width=32 with this part means **two physical MT41J128M16 chips wired in parallel**, each supplying half the data bus (standard DDR3 ganging — address/command fan out to both chips, only DQ/DQS/DM roughly double). This is a real BOM/footprint/routing addition, **not** "no PCB change" as an earlier draft of this document incorrectly claimed. Whether the extra DQ/DQS pins still fit within banks 34/35's existing allocation (likely, since address/command isn't duplicated) or require reaching into bank 14/15 must be confirmed from the real regenerated pinout, not assumed. | Also requires a second physical DDR3 chip (or a second x16-wide subsystem) for its own full independent controller, **plus** claims bank 14/15 for that second channel's own address/command/data — the SPI/flash pin conflict is additional to, not instead of, the same per-chip PCB cost as the 32-bit option. |
|
||
|
||
**Recommendation (honest, not deferential, as requested — and corrected
|
||
once, honestly, mid-session)**: **32-bit single-channel widening**, not a
|
||
second independent channel — this part of the recommendation still holds.
|
||
What does **not** hold, and was a real mistake in this document's earlier
|
||
version: the claim that widening needs "no PCB changes". It does — a second
|
||
physical DDR3 chip, confirmed by the MIG wizard itself (Memory Details:
|
||
x16, single component) when Data Width was set to 32. The real, remaining
|
||
differentiator versus a second independent channel is narrower than
|
||
originally stated but still real: 32-bit widening very likely stays within
|
||
the ALREADY-USED banks 34/35 (only data/strobe pins roughly double, address/
|
||
command is shared), while a second independent channel definitely needs its
|
||
OWN address/command/data pins, conflicting with the already-placed SPI
|
||
management bus (bank 15) and config-flash bus (bank 14). **User confirmed
|
||
they already understood the second-chip requirement and decided to proceed
|
||
with 32-bit widening regardless** (2026-09-20) — this remains the decided
|
||
path, now on the correct, fully-disclosed premise.
|
||
|
||
**What this required**: the real Xilinx MIG "Customize IP" wizard, re-run
|
||
interactively by the user (Data Width 16→32 AND Input Clock Period changed
|
||
in the same wizard session, since both require the wizard's own JEDEC/PLL
|
||
calculator — never hand-edited, per this project's own established
|
||
discipline). **Done, 2026-09-20.**
|
||
|
||
#### 5.4.1 Real outcome (EXP-0084 → EXP-0086): functionally complete AND timing closed — the width/clock-speed decoupling
|
||
|
||
**Confirmed correct, as predicted above**: the widened DQ/DQS/DM pins
|
||
stayed within banks 34/35 (no bank 14/15 conflict for the DDR3 signals
|
||
themselves) — real, verified from the actual routed pinout, not assumed.
|
||
One real prediction from this section did **not** hold, though, and is
|
||
corrected here rather than hidden: the differential **reference clock**
|
||
(`clk_ref_p/n`, a signal this section didn't analyze, chosen later in the
|
||
same wizard session) landed in **bank 14** by the wizard's own real UG586
|
||
placement rules — directly conflicting with the already-placed config-flash
|
||
SPI bus there (LVCMOS33 vs LVDS_25, incompatible VCCO in one bank). Real,
|
||
observed as an actual `place_design` failure, not hypothetical — fixed by
|
||
moving the flash bus to bank 16 (see `docs/PHYSICAL_REALIZATION.md` §2.3).
|
||
|
||
**The real, more consequential finding**: the SAME wizard session paired
|
||
the Data Width change with an Input Clock Period change (3225ps→2900ps,
|
||
chosen to keep the PHY:Controller ratio at 2:1 rather than falling back to
|
||
4:1, which would have halved `ui_clk` instead of speeding it up). This
|
||
clock speedup pushed `ui_clk` from 155.039MHz to 172.414MHz (+11.2%) — and
|
||
real P&R shows this specific increase breaks timing in
|
||
`neural_processor_packed.v`'s own packed-MAC accumulation tree (unchanged
|
||
since EXP-0059, real positive margin at the old rate, real -0.618ns
|
||
violation at the new one). **This is unrelated to the 32-bit width change
|
||
itself** — every module touched for the widening is real-verified
|
||
functionally correct (real xsim, real 2-chip DDR3 model, all tests PASS).
|
||
|
||
**The honest, useful conclusion**: width and clock rate are separable
|
||
levers, and this experiment shows they don't have to be exercised together.
|
||
Bandwidth = width × clock — 32-bit width **alone**, even at the OLD,
|
||
already-timing-proven 155.039MHz, already delivers the full intended 2×
|
||
bandwidth gain (1.24 → ~2.48 GB/s). The clock speedup was free upside *if*
|
||
the compute datapath could absorb it; real measurement showed it couldn't
|
||
without a real, separate re-pipelining of that accumulation tree — so it
|
||
was reverted rather than pursued, keeping the task scoped to channel width.
|
||
|
||
**Real closure (EXP-0086)**: the user reverted Input Clock Period back to
|
||
3225ps (keeping Data Width=32) via a second real MIG wizard session. This
|
||
regeneration itself triggered a real, wholesale recurrence of the
|
||
stale-import bug — Vivado silently re-imported the entire v3 RTL tree (9
|
||
files) plus the top XDC back to their pre-EXP-0084 state, reproducing the
|
||
already-fixed VCCO/PROHIBIT failures until root-caused (via the actual P&R
|
||
log, not guessed) and fixed again by re-establishing direct-reference
|
||
sources (see CLAUDE.md's stale-import lesson, now extended to cover this).
|
||
Once fixed, the real P&R closed cleanly: **WNS=+0.095707ns, WHS=+0.036275ns,
|
||
0 failing endpoints, "All user specified timing constraints are met."** The
|
||
`clk_pll_i`/155.039MHz domain — the exact domain that failed at -0.618ns in
|
||
the 2900ps attempt — closes at +0.096ns across 24522 endpoints, matching
|
||
EXP-0083's own 16-bit-era margin (+0.073ns) closely, confirming the width
|
||
change itself was never the real problem.
|
||
|
||
**Current real status**: fully DONE — RTL functionally verified (real xsim)
|
||
AND real P&R timing-closed (EXP-0086). This is now the project's current,
|
||
trustworthy signoff, replacing EXP-0083. `data_ready_n` (EXP-0085) also
|
||
confirmed placed and routed cleanly (D14/LVCMOS33) in this same run. See
|
||
`hardware/v2/logs/experiments.log` EXP-0084 and EXP-0086 for the complete
|
||
story, including all real P&R/XDC bugs found and fixed along the way.
|
||
|
||
### 5.5 Scaling path recommendation (real numbers, not a guess) — updated per user's final directive
|
||
|
||
Given §3's real bandwidth ceiling: **scaling core count alone, without
|
||
§5.2/§5.4, provides no real additional throughput past whatever N already
|
||
saturates the (post-§5.1) ~2.48 GB/s-equivalent demand ceiling** —
|
||
back-of-envelope, using §3.2's current numbers (2.48 GB/s needed per core
|
||
at peak, 1.24 GB/s physically available today pre-widening), that's already
|
||
close to N≈1 in the worst case and at most N≈2 in the best (zero-row-switch)
|
||
case on the current 16-bit channel. **Building N=4/8/16 before §5.2/§5.4 are
|
||
in place would very likely show near-IDENTICAL real throughput to N=2** — a
|
||
real, wasted engineering cycle the analysis recommends avoiding.
|
||
|
||
**Decided real order of work** (§5.1 and §5.2-phase-1 already done; this
|
||
reflects the user's own explicit final direction — 32-bit widening, then
|
||
the DDRManager, then N=2/4/8/16 tests, real target N=8, N=16 built
|
||
specifically to document where/how it breaks rather than to succeed):
|
||
1. ~~§5.3 (result-writeback)~~ / ~~§5.1 (denser activation packing)~~ — §5.1
|
||
**DONE** (EXP-0081/0082). §5.3 remains a genuine blocker for N>2 and must
|
||
land before any scaling test that needs real result data out of more than
|
||
2 cores' worth of pins.
|
||
2. §5.4 (32-bit channel widening) — **DONE** (EXP-0084 functional +
|
||
EXP-0086 real timing closure, WNS=+0.096ns). Doubles the physical
|
||
ceiling itself (real, closed, ~2.48 GB/s), which §5.1 alone could not do.
|
||
3. §5.2 (DDRManager) — phase 1 (single-slot look-ahead prefetch) is
|
||
**DONE and RE-MEASURED** (EXP-0083's original ~2.9% benefit was
|
||
measured on the OLD 16-bit channel; EXP-0087 re-measured the same real
|
||
A/B against the now-closed 32-bit channel and found the benefit is
|
||
gone, ~0%/a 0.0064% regression — the wider channel's lower per-tile
|
||
latency already absorbed the gap the look-ahead used to hide). Kept
|
||
wired in for correctness/timing-neutrality; the larger multi-slot
|
||
scheduler is **not justified** by this real result — revisit only if
|
||
N=4/8/16 scaling below reintroduces DDR3-latency-bound behavior.
|
||
4. Real N=2/4/8/16 tests, each with its own real P&R signoff (margin is
|
||
thin, §2 — do not assume a prior N's timing closure predicts the next).
|
||
N=8 is the real target configuration; N=16 is expected to expose real
|
||
bus/arbitration/timing limits and is built specifically to document that
|
||
breakdown, not to be a viable production configuration. With DDR3
|
||
latency no longer the real bottleneck at N=2 (EXP-0087), this is now
|
||
the more promising real lever to pursue next.
|
||
|
||
---
|
||
|
||
### 5.6 [Steps 1–2 DONE, EXP-0089/0090] Hybrid systolic scaling: 4 groups × 4-PE weight-stationary chains
|
||
|
||
Captured from a 2026-09-20 brainstorming session as a purely exploratory
|
||
idea; the same day, per the user's own explicit reprioritization, the real
|
||
open design question below (shared-weight broadcast vs. a literal PE-to-PE
|
||
systolic shift register) was resolved with the user directly (not guessed).
|
||
**Real progress so far**:
|
||
- `systolic_group.v` + `packed_pe.v` (EXP-0089) — one real group of 4 PEs
|
||
sharing a single broadcast weight fetch, real barrier-synchronized, real
|
||
xsim-verified (8/8 PASS across 2 consecutive group jobs).
|
||
- Real out-of-context synthesis of one group (EXP-0090): **32 DSP48E1**
|
||
(13.3%), confirming the original brainstorm's own quantified DSP
|
||
projection exactly (8 DSP/PE × 4 PEs = 32; scaled to 4 groups, 128/240 =
|
||
53%, matching the doc's own earlier estimate).
|
||
- `neural_director_grouped.v` (EXP-0090) — real Director extension
|
||
dispatching 8-position octets to free groups, a direct extension of
|
||
`neural_director_packed.v`'s own already-proven 2-position pairing
|
||
discipline. Real, deliberate finding: the host-facing SPI/`WRITE_JOB`
|
||
submission protocol needs **zero changes** — the host just submits 8
|
||
jobs sharing a weight base instead of 2. Real xsim-verified (4/4 PASS:
|
||
correct octet dispatch + per-PE addressing, correct stall-not-mis-
|
||
dispatch on a mismatched octet, correct queue wraparound). Also
|
||
surfaced a real, generalizable testbench-race lesson (tight back-to-
|
||
back stimulus pulses on the same edge the DUT samples on — fixed with
|
||
`@(negedge clk)` stimulus — see CLAUDE.md).
|
||
|
||
**Not yet done**: a real N=16 (4-group) top-level module + a real,
|
||
appropriately-sized arbiter, and real in-context P&R for the whole system
|
||
— see EXP-0090's own `next_action`.
|
||
|
||
**The problem it targets**: plain N=16 independent cores (§5.5's own
|
||
"documentary, expected to break" framing) means 16 independent DDR3
|
||
requesters contending for one arbitrated channel — real congestion that
|
||
neither §5.1 (packing) nor §5.4 (32-bit widening) alone removes, since both
|
||
attack bytes-per-MAC or raw bandwidth, not the *number of independent
|
||
consumers*.
|
||
|
||
**The idea**: instead of 16 flat, independent `packed_slot.v` instances,
|
||
group them into **4 systolic chains of 4 PEs each**. Within a chain: the
|
||
weight tile stays resident (loaded once, same "weight-stationary" pattern
|
||
`layer_prefetch_ctrl.v`/`layer_weight_buffer.v` already implement for the
|
||
existing A/B lane reuse — this is a direct extension to 4 positions instead
|
||
of 2, not a new mechanism), and activation data streams through the chain
|
||
position by position, fetched from DDR3 once per chain rather than once per
|
||
PE. Between the 4 chains (groups), full task-level parallelism is
|
||
preserved — each group can run an independent job, same as today's model.
|
||
|
||
**Why weight-stationary specifically (not activation-broadcast)**: chosen
|
||
because it generalizes to any layer type (FC, conv-via-im2col, attention —
|
||
anything reducible to "same weight matrix, many activation vectors") without
|
||
assuming a specific model's channel count or convolution overlap pattern —
|
||
important since this is a general-purpose accelerator, not built for one
|
||
fixed network.
|
||
|
||
**Real, quantifiable rationale** (order-of-magnitude, not yet measured —
|
||
flagged explicitly as a projection):
|
||
- DSP48E1: 16 cores × 8 DSP/core = 128/240 (53%) — real, fits with margin.
|
||
- DDR3 requesters: drops from 16 (flat) to 4 (one per chain) — a real 4×
|
||
reduction in the number of independent contenders for the arbitrated
|
||
channel, on top of (not instead of) §5.1's packing and §5.4's widening.
|
||
- Combined with §5.4's 32-bit widening (2× physical ceiling), the
|
||
available-bandwidth-to-demand ratio improves by roughly 8× versus the
|
||
naive flat-N=16 baseline — a real, meaningful, but NOT a full solve by
|
||
itself; still needs real measurement once anything is built.
|
||
|
||
**Real open question, RESOLVED (EXP-0089)**: intra-chain dataflow — the
|
||
user was asked directly (`AskUserQuestion`, concrete side-by-side preview
|
||
of both real topologies) rather than guessed, given how different their
|
||
real risk/complexity profiles are. Chosen: **shared-weight broadcast** (one
|
||
real weight fetch per group of 4 PEs, no inter-PE result/data propagation,
|
||
each PE computes its own independent activation positions in parallel) —
|
||
**not** a literal PE-to-PE systolic shift register. This achieves the real,
|
||
quantified rationale above (4× reduction in redundant weight-fetch DDR3
|
||
traffic per group) without the real complexity/risk of genuine pipeline
|
||
fill/drain at chain boundaries, which the literal-systolic alternative
|
||
would have required. Real result-writeback stays exactly as §5.3 already
|
||
built it — `packed_pe.v` reuses `result_writeback.v` completely unmodified,
|
||
one real DDR3 write per PE, not a new "4 chain-output events" model (that
|
||
framing, from the original brainstorm, doesn't apply to the broadcast
|
||
design actually built). Arbitration: confirmed real — `sdram_arbiter_n.v`'s
|
||
own `NUM_REQ` parameter generalized to a 5-way arbiter (1 group weight-fetch
|
||
+ 4 PE activation/writeback) in `tb_systolic_group.v` with zero changes to
|
||
that module.
|
||
|
||
**Decision**: first real step DONE (EXP-0089) — build and verify the
|
||
isolated group mechanism before any Director/SPI/top-level integration,
|
||
per this project's own "one variable at a time" discipline. Full N=16
|
||
integration (Director-level group job dispatch, SPI protocol extension,
|
||
real top-level, real P&R) is real, disclosed, deliberately deferred work,
|
||
not yet done — see EXP-0089's own `next_action` in
|
||
`hardware/v2/logs/experiments.log`.
|
||
|
||
#### 5.6.1 [EXPLORATORY] Opportunistic BRAM cache for activation tiles
|
||
|
||
Smaller and more incremental than §5.6's systolic restructuring — doesn't
|
||
require knowing anything about the target network's structure in advance.
|
||
Artix-7 100T's Block RAM is real and currently **0% utilized** (§3, every
|
||
real P&R signoff to date) — real, free, unused capacity.
|
||
|
||
**The idea**: a small direct-mapped or low-associativity cache, in BRAM,
|
||
remembering the last few activation tiles fetched from DDR3 (address +
|
||
data). Before `act_tile_fetch.v` (or `ddr_prefetch_mgr.v`, EXP-0083) issues
|
||
a real DDR3 request, check the cache first — on a hit (e.g. two jobs, on
|
||
the same or different slots, requesting overlapping/adjacent tiles, common
|
||
in convolution with sliding-window overlap), skip the DDR3 round-trip
|
||
entirely.
|
||
|
||
**Why it's attractive**: catches real reuse the design doesn't have to
|
||
predict or assume in advance — unlike §5.6's systolic chains (which commit
|
||
to a specific reuse *pattern*, weight-stationary), a cache opportunistically
|
||
exploits WHATEVER locality the real workload happens to have, including
|
||
patterns nobody designed for. Composable with everything else already
|
||
built or decided (§5.1 packing, §5.4 widening, §5.6 systolic groups if that
|
||
direction is taken) — it's a cache in front of the existing fetch path, not
|
||
a replacement for it.
|
||
|
||
**Real open questions**: cache size vs. real hit rate is workload-dependent
|
||
and NOT measured — would need a real trace-driven estimate (or a real
|
||
simulation with representative test data) before sizing it, not guessed.
|
||
Coherency is simple here (activation data in DDR3 is written once by the
|
||
host before a job runs and never modified during compute, per the current
|
||
protocol) — no cache-invalidation problem to solve, a real simplification
|
||
versus a general-purpose cache design.
|
||
|
||
#### 5.6.2 [EXPLORATORY] Host-side (ESP32) job-queue optimization — a "software DDRManager"
|
||
|
||
A different kind of lever than anything else in this section: instead of
|
||
adding hardware intelligence inside the FPGA, exploit the fact that the
|
||
**ESP32 already has full visibility of the whole job queue before
|
||
submitting it** — `neural_director_packed.v`'s own `QUEUE_DEPTH=8` hardware
|
||
queue only sees jobs one at a time as they're written over SPI; the ESP32
|
||
firmware, upstream of that, could see and reorder ALL pending jobs at once.
|
||
|
||
**The idea**: the ESP32's own job-submission firmware groups/reorders jobs
|
||
before writing them over the management SPI bus (§2.2 of
|
||
`docs/PHYSICAL_REALIZATION.md`), so that jobs whose DDR3 addresses are
|
||
close together (same or adjacent rows) are submitted close together in
|
||
time — directly reducing the real row-switch cost (§3.3) that dominates
|
||
per-tile latency, WITHOUT any new RTL at all. A real, software-only
|
||
"DDRManager" living in ESP32 firmware, upstream of and complementary to
|
||
`ddr_prefetch_mgr.v` (EXP-0083, which only looks ahead within one already-
|
||
submitted job).
|
||
|
||
**Why this is worth capturing seriously, not just as a curiosity**: it's
|
||
the cheapest possible lever in this whole list — zero RTL, zero real P&R
|
||
risk, zero timing-margin cost (the project's real margin is thin, §2, and
|
||
every RTL addition risks it; this doesn't touch RTL at all) — and it's far
|
||
faster to iterate on than Verilog (the user's own established preference
|
||
for where complexity is easiest to absorb). It doesn't compete with any
|
||
other idea in this section — it can be built independently, at any time,
|
||
by whoever writes the ESP32-side firmware, and composes with all of them.
|
||
|
||
**Real open question**: requires the host firmware to know DDR3 addresses
|
||
well enough to group by row locality (`ROW_BITS`/`COL_BITS`/`BANK_BITS`
|
||
convention, §2 of `docs/PHYSICAL_REALIZATION.md`) — a real firmware-side
|
||
design task, not yet scoped, and out of this repository's own RTL scope
|
||
(ESP32 firmware isn't part of `hardware/v3/`).
|
||
|
||
---
|
||
|
||
## 6. Summary table: what's real vs. what's a calculation
|
||
|
||
| Claim | Status |
|
||
|---|---|
|
||
| WNS/WHS/LUT/DSP numbers throughout | **Measured** (real Vivado P&R reports, current: EXP-0086) |
|
||
| DDR3 back-to-back burst throughput (1.24 GB/s) | **Measured** (real `ddr3_model.sv` JEDEC trace) — physical channel limit, unchanged by §5.1's packing fix |
|
||
| Real Activate→Read latency (16.125 ns) | **Measured** (same trace) |
|
||
| Per-core compute throughput (2.48 GMAC/s) | **Calculated** from measured Fmax (155.039MHz) + known, fixed DSP-packing factor |
|
||
| Bandwidth needed per core, pre-packing (4.96 GB/s) | **Calculated**, historical (EXP-0079 layout) |
|
||
| Bandwidth needed per core, post-packing (2.48 GB/s) | **Calculated** from the real, as-built EXP-0081/0082 memory layout — current |
|
||
| "~25% of peak sustainable" (pre-packing) / "~50%" (post-packing, current) | **Calculated** ratios; post-packing figure re-verified against real P&R (EXP-0082) and real simulation (§5.1) |
|
||
| I/O bank/DQS pin counts for XC7A100T-CSG324 (banks 14/15/16/34/35) | **Measured** — queried directly from the real Vivado part database for this exact part/package, used in §5.4's dual-channel-vs-widening analysis |
|
||
| Row-switch penalty as a fraction of real workloads | **Not measured** — depends on host-chosen memory layout, flagged as an open question, not asserted |
|
||
| DDRManager phase-1 real stall-reduction benefit | **Measured** — real xsim A/B on `tb_n2_system_ddr3.v`, real DDR3 backend, before vs after `ddr_prefetch_mgr.v`. **2.86%** at the OLD 16-bit channel (EXP-0083); **~0% (0.0064% regression)** re-measured at the closed 32-bit channel (EXP-0087) — the benefit does not survive the wider channel, real and honestly reported, not oversold either direction. |
|
||
| Full multi-slot DDRManager's real benefit | **Not measured, not built, and now real-disconfirmed as a priority** — phase 1's own re-measurement against the closed 32-bit channel (EXP-0087) found ~0% benefit; §5.2 recommends against building the larger version unless N=4/8/16 scaling reintroduces DDR3-latency-bound behavior |
|
||
| 32-bit widening's real post-change bandwidth/timing numbers | **Measured** — real, closed P&R signoff (EXP-0086): WNS=+0.096ns, WHS=+0.036ns, 0 failing endpoints. The ~2.48 GB/s figure is now a real, closed-timing result, not a projection. |
|