michele/FPGA-Neural
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

docs: sync physical/architecture docs with EXP-0081/0082 reality, add 32-bit vs dual-channel analysis

Browse Source 
- PHYSICAL_REALIZATION.md: replace stale "1 tile = 1 burst" layout
  description with the real EXP-0081/0082 "2 tiles = 1 burst" convention;
  add EXP-0082 signoff row and history table.
- ARCHITECTURE_ANALYSIS.md: mark §5.1 (denser activation packing) DONE with
  real re-measured numbers (bandwidth ceiling fraction 25%->50%, WNS
  +0.030->+0.068ns); add §5.4, the real device-data-backed comparison of
  32-bit single-channel widening vs a second independent DDR3 channel
  (decided: 32-bit widening, per real DQS/bank pin-conflict analysis);
  update scaling-path recommendation to reflect the user's final directive
  (widen channel -> build DDRManager -> N=2/4/8/16 tests, N=8 target, N=16
  documentary).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUG92aM9m68TRc4rG55BcC
This commit is contained in:
michele
2026-09-20 10:07:16 +02:00
co-authored by Claude Sonnet 5
parent e6768623d6
commit cbd16dd727
2 changed files with 237 additions and 98 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ARCHITECTURE_ANALYSIS.md
+194 -83
View File
@@ -1,11 +1,18 @@
# FPGA-Neural V3 — Architecture Analysis: Timing, Bottlenecks, and Recommended Interventions
Scope: the current, real, P&R-verified V3 design (`hardware/v3/`, branch
`v3-artix7`), as of EXP-0079 (real activation-fetch engine, real DDR3 for
both weights and activations, real P&R: WNS +0.030ns). Every number in this
document is either directly measured (real simulation trace, real P&R
report) or a calculation built from directly-measured building blocks — the
two are labeled explicitly throughout. Nothing here is guessed.
`v3-artix7`), updated through EXP-0082 (denser activation packing, real
P&R: WNS +0.068ns). Every number in this document is either directly
measured (real simulation trace, real P&R report) or a calculation built
from directly-measured building blocks — the two are labeled explicitly
throughout. Nothing here is guessed.
**Status note (post EXP-0082)**: §5.1 (denser activation packing) described
below as a *recommendation* is now **DONE and real-P&R-verified** — see the
"DONE" marker in that section and the updated bandwidth numbers in §3. The
document originally analyzed the pre-fix state; it's kept below (marked
historical) because the comparison is itself informative, then updated with
the real post-fix numbers throughout.
---
@@ -20,18 +27,27 @@ DDR3 channel.
| Metric | Value | Source |
|---|---|---|
| Real DDR3 back-to-back burst bandwidth | **1.24 GB/s** (9.92 Gbps) | measured, real JEDEC trace (§3.1) |
| Real DDR3 bandwidth needed for ONE core at peak DSP throughput | **4.96 GB/s** | calculated from measured DSP rate + real memory layout (§3.2) |
| → DDR3 can sustain at best | **~25%** of one core's peak compute throughput | §3.2 |
| Real P&R timing margin (WNS) | **+0.030 ns** | measured, EXP-0079 real P&R |
| Real DDR3 back-to-back burst bandwidth (physical channel) | **1.24 GB/s** (9.92 Gbps) | measured, real JEDEC trace (§3.1) — unchanged by packing, this is a physical-channel limit |
| Real DDR3 bandwidth needed for ONE core at peak DSP throughput | **4.96 GB/s** | calculated from measured DSP rate + memory layout (§3.2) |
| → DDR3 can sustain, pre-EXP-0081 packing (1 tile/burst) | **~25%** of one core's peak compute throughput | §3.2, historical |
| → DDR3 can sustain, post-EXP-0081/0082 packing (2 tiles/burst, DONE) | **~50%** of one core's peak compute throughput | §3.2, current, real |
| Real P&R timing margin (WNS) | **+0.068 ns** | measured, EXP-0082 real P&R (improved from +0.030ns pre-packing) |
| DSP48E1 headroom for scaling | 224/240 free (93%) | measured, real P&R utilization |
The DSP headroom is real and large. The memory-bandwidth ceiling is real and
small. **The highest-leverage next step is fixing the activation-fetch memory
waste (§5.1), not adding cores.** The user's own proposed DDRManager/
prefetch idea (§5.2) is valuable and complementary, but solves a *different*
problem (latency/stalling) than the bandwidth ceiling (§3.2) — both are
covered below, with the distinction made explicit.
The DSP headroom is real and large. The memory-bandwidth ceiling is real,
and **denser activation packing (§5.1) has already doubled the real
achievable fraction of it** — from ~25% to ~50% of one core's peak DSP
throughput, with the real P&R margin *improving*, not degrading, as a side
effect. This was free leverage and it's now banked.
Even at ~50%, DDR3 is still the limiting resource, not DSP count — the next
real interventions, per the user's own explicit direction, are: (a) widening
the physical DDR3 channel from 16-bit to 32-bit (§5.5 — doubles the physical
1.24 GB/s ceiling itself, unlike §5.1 which only reduced waste against a
fixed ceiling), and (b) an intelligent DDRManager (§5.2) to hide latency via
orchestrator-driven prefetch. Both are required together — a wider channel
without a smarter prefetcher still stalls on latency; a smarter prefetcher
against a 16-bit channel still hits the same physical bandwidth wall.
---
@@ -43,10 +59,11 @@ covered below, with the distinction made explicit.
| 0074 | first real in-context P&R: DDR3 + pins + register file not yet added | +0.040 | 5140 | 16 | first trustworthy board-accurate number |
| 0076 | + register file, + pin constraints, + SPI physical-layer fix | +0.056 | 5173 | 16 | margin improved slightly (P&R is not perfectly monotonic run to run) |
| 0078 | + config-flash bridge (real STARTUPE2 placement) | +0.013 | 5213 | 16 | margin dropped — real added logic |
| 0079 | + real activation-fetch engine (`act_tile_fetch.v`) | **+0.030** | 5379 | 16 | current, final, trustworthy number |
| 0079 | + real activation-fetch engine (`act_tile_fetch.v`) | +0.030 | 5379 | 16 | pre-packing baseline |
| 0082 | + denser activation packing (2 tiles/burst, EXP-0081) | **+0.068** | 5437 | 16 | current, final, trustworthy number — margin IMPROVED despite added mux logic |
**Observation**: WNS does not move monotonically with LUT count (0.056 →
0.013 → 0.030 while LUTs only ever grow) — this is normal P&R behavior
0.013 → 0.030 → 0.068 while LUTs only ever grow) — this is normal P&R behavior
(placer/router heuristics find different solutions each run, small logic
changes can shift which path is critical). **Do not extrapolate a trend
line from 3-4 data points** — the only safe practice is a fresh real P&R
@@ -98,30 +115,41 @@ weight) = 16 MACs/cycle/core. At the real measured 155.039 MHz compute clock:
```
Each compute cycle consumes 1 activation byte per MAC lane (16 bytes total:
8 for lane A, 8 for lane B). Under the current real memory layout
(`act_tile_fetch.v`, EXP-0079: one tile = one full `BURST_LEN=8` burst = 16
bytes moved, only 8 useful), feeding one compute cycle costs:
8 for lane A, 8 for lane B).
**Historical (EXP-0079, "1 tile = 1 burst")**: each tile fetch moved a full
16-byte burst for only 8 useful bytes:
```
2 tiles (A+B) × 16 bytes/burst = 32 bytes moved DDR3 traffic → 16 MACs
= 2 real DDR3 bytes moved per MAC operation
2.48 GMAC/s × 2 bytes/MAC = 4.96 GB/s needed per core
```
Bandwidth needed to keep ONE core's DSPs fed at their real peak rate:
**Current (EXP-0081/0082, "2 tiles = 1 burst", DONE and real-P&R-verified)**:
two tiles now share one 16-byte burst, halving the moved-bytes-per-useful-byte
ratio:
```
2.48 GMAC/s × 2 bytes/MAC = 4.96 GB/s
2 tiles (A+B) × 16 bytes/burst ÷ 2 tiles-per-burst = 16 bytes moved DDR3 traffic → 16 MACs
= 1 real DDR3 byte moved per MAC operation (halved)
2.48 GMAC/s × 1 byte/MAC = 2.48 GB/s needed per core (halved)
```
### 3.3 The real gap
```
1.24 GB/s available (measured, best case) vs 4.96 GB/s needed per core (calculated)
→ DDR3 can sustain at most ~25% of one core's peak DSP throughput,
even in the BEST case (zero row-switch overhead, one core, nothing else
sharing the bus).
Historical: 1.24 GB/s available vs 4.96 GB/s needed per core → ~25% sustainable
Current: 1.24 GB/s available vs 2.48 GB/s needed per core → ~50% sustainable
```
The physical channel ceiling (1.24 GB/s, §3.1) did **not** change — §5.1's
fix reduced waste against a fixed ceiling, it did not raise the ceiling
itself. Even at ~50%, DDR3 remains the binding constraint, not DSP count,
even in the BEST case (zero row-switch overhead, one core, nothing else
sharing the bus). Raising the *physical* ceiling requires a wider channel
(§5.5) or a second channel — both analyzed below.
This ceiling gets **worse**, not better, with:
- **Row switches**: real measured Activate→Read latency is 16.125 ns
(5 DDR3 clock cycles at 3.225 ns = real CAS-latency-5 timing, matches the
@@ -136,12 +164,16 @@ This ceiling gets **worse**, not better, with:
activation fetching, though this cost is shared across the M reuse
positions and becomes negligible for large M.
**Conclusion**: this is a genuine architectural ceiling, not a tuning
problem. The 2× byte-overhead from the current "1 tile = 1 full burst"
convention (chosen in EXP-0079 specifically to avoid a runtime-indexed
part-select, given the then-already-thin timing margin) is now confirmed,
with real numbers, to be the single most expensive design decision in the
current memory path. See §5.1 for the recommended fix.
**Conclusion**: this was a genuine architectural ceiling, not a tuning
problem, and half of it has now been recovered for free. The 2× byte-
overhead from the original "1 tile = 1 full burst" convention (chosen in
EXP-0079 specifically to avoid a runtime-indexed part-select, given the
then-already-thin timing margin) was confirmed, with real numbers, to be the
single most expensive design decision in the memory path — and has since
been fixed (§5.1, DONE, EXP-0081/0082) by registering the select bit at
request time instead of avoiding the select entirely. The remaining gap
(~50% sustainable, not 100%) is now a *physical channel width* problem, not
a packing-waste problem — see §5.5.
---
@@ -170,20 +202,24 @@ path currently lacks (§5.1's recommendation follows the SAME philosophy).
No real change recommended; this module is a good template for how the
activation path should evolve.
### 4.3 Activation-fetch path (`act_tile_fetch.v`, EXP-0079)
### 4.3 Activation-fetch path (`act_tile_fetch.v`, EXP-0079, updated EXP-0081/0082)
Real, correct (verified 3 levels deep, §2 of EXP-0079's own log entry), but
— per §3 above — the current real bottleneck. Two real design choices worth
re-examining now that real bandwidth numbers exist:
Real, correct (verified 3 levels deep, §2 of EXP-0079's own log entry; the
EXP-0081 packing change re-verified at all 3 levels again — `tb_act_tile_fetch.v`
8/8, `tb_packed_slot.v` 9/9, `tb_n2_system_ddr3.v` 8/8, plus real P&R). Two
real design choices identified in this analysis:
1. **1 tile = 1 full burst (2× byte overhead)**: chosen to avoid a
runtime-indexed part-select. §5.1 proposes a way to recover this
efficiency without reintroducing that risk.
2. **Lane A then lane B, sequential, per tile**: doubles the real number of
DDR3 transactions (and row-switch risk) versus a design that could fetch
both lanes in a single wider transaction when they happen to be adjacent
in memory. Not changed in this analysis pass — flagged for future work
if §5.1's fix doesn't fully close the gap.
1. **1 tile = 1 full burst (2× byte overhead)** — **FIXED (EXP-0081/0082,
DONE)**: now 2 tiles share 1 burst via a request-time-registered select
bit (`sel_lat`), avoiding the runtime-indexed-part-select Fmax risk while
still halving DDR3 waste. Real P&R confirms margin improved, not
degraded (+0.030ns → +0.068ns). See §5.1.
2. **Lane A then lane B, sequential, per tile**: still doubles the real
number of DDR3 transactions (and row-switch risk) versus a design that
could fetch both lanes in a single wider transaction when they happen to
be adjacent in memory. Not changed — flagged for future work; the
DDRManager (§5.2) and 32-bit channel widening (§5.5) are higher-leverage
and were prioritized first per the user's explicit direction.
### 4.4 Scheduling (`neural_director_packed.v`) and arbitration
(`sdram_arbiter_n.v`)
@@ -225,29 +261,37 @@ memory-bandwidth ceiling in §3. Recommended fix in §5.3.
## 5. Recommended interventions, ranked by real leverage
### 5.1 [Highest leverage] Denser activation packing — attack the real bandwidth ceiling directly
### 5.1 [Highest leverage] Denser activation packing — attack the real bandwidth ceiling directly — **DONE (EXP-0081/0082)**
**What**: pack 2 tiles (lane A + lane B, or two consecutive tiles of the
same lane) into one `BURST_LEN=8` burst instead of one tile per burst,
halving real DDR3 bytes-per-MAC from 2 to 1. This alone would raise the
real achievable fraction of one core's peak throughput from ~25% to ~50%
(§3.2's math, halved).
**What was built**: 2 tiles (even/odd tile index) now share one
`BURST_LEN=8` burst instead of one tile per burst, halving real DDR3
bytes-per-MAC from 2 to 1. Real, measured effect: the achievable fraction
of one core's peak DSP throughput rose from ~25% to ~50% (§3.2/§3.3,
current numbers).
**Why this was avoided in EXP-0079**: doing so naively requires a
runtime-indexed part-select (which half of the burst response to use,
selected by a runtime tile-index bit) — the same anti-pattern
`weight_tile_gather.v` (EXP-0061) already flagged as a real Fmax risk, and
the P&R margin was already thin (+0.013ns) when this design decision was
made.
the P&R margin was already thin (+0.013ns) when that original design
decision was made.
**A safer path to the same efficiency gain** (not yet built, this is a
recommendation): register the tile-index LSB **one cycle ahead of** the
burst response arriving (it's already known at request time, not something
that needs to race the read data) — using it to select a *pre-registered*
mux input rather than gating the read-data path itself keeps the selection
off the critical timing path. This needs a real prototype and a real P&R
check before being trusted — proposed as the next concrete engineering task,
not asserted as already safe.
**The safe path that was actually built**: the tile-index LSB is registered
into `sel_lat` at *request* time (`S_IDLE`, the same cycle `tcnt` is
latched) — many `ui_clk` cycles before the real DDR3 round-trip completes
and `ctrl_rdata` becomes valid. The eventual data-select mux therefore
selects on an already-long-stable registered bit, never one racing the read
data. **Real P&R confirms this is genuinely timing-safe, not just
functionally correct**: margin *improved* from +0.030ns to +0.068ns
(EXP-0082), despite the added mux logic (LUTs 5379→5437). Full detail:
`hardware/v3/rtl/act_tile_fetch.v` header, `docs/PHYSICAL_REALIZATION.md` §4,
`hardware/v2/logs/experiments.log` EXP-0081/EXP-0082.
**Verification**: `tb_act_tile_fetch.v` (8/8 PASS, covers even/odd-in-same-
burst, new-burst crossing, back-to-back alternation), `tb_packed_slot.v`
(9/9 PASS, bit-identical numeric results to the pre-change run),
`tb_n2_system_ddr3.v` (8/8 PASS, real xsim against real `ddr3_model.sv`,
JEDEC trace confirmed to show no more half-burst zero-padding).
### 5.2 [Complementary, addresses latency not bandwidth] DDRManager with orchestrator-driven prefetch (user's proposal)
@@ -309,29 +353,93 @@ never as N-scaled literal top-level pins again. Same architectural shape as
the weight-fetch path, in reverse (write instead of read) — a reasonable,
bounded scope, and a real prerequisite, not optional polish.
### 5.4 Scaling path recommendation (real numbers, not a guess)
### 5.4 [Decided] Widening the physical DDR3 channel: 32-bit single channel vs. a second independent 16-bit channel
**The real question**: §5.1 halved *waste* against a fixed 1.24 GB/s
physical ceiling; it did not raise the ceiling itself. Getting past ~50%
sustained DSP utilization requires more physical bytes/second, which means
either (a) widening the existing channel from 16-bit to 32-bit data width,
or (b) adding a second, independent 16-bit DDR3 channel. Both roughly
double the real 1.24 GB/s ceiling to ~2.48 GB/s. The user asked for an
honest comparison, not a diplomatically-balanced non-answer — here it is,
based on **real device data**, not guessed.
**Real device data** (queried directly from the actual Vivado part database
for this exact part/package, XC7A100T-**CSG324**): this package has only
**5 total I/O banks** — 14 (56 pins), 15 (56 pins), 16 (11 pins), 34 (56
pins), 35 (56 pins). All report `BANK_TYPE=BT_HIGH_RANGE` (Artix-7 has no
separate "HP" bank class the way some other families do). Banks **14 and
15** each expose **8 DQS-capable pin pairs** — the same memory-PHY
signature already used by the real, placed DDR3 controller on banks 34/35.
This means a second, independent DDR3 channel is *physically plausible* on
this package (the DQS-capable pins exist), but:
| Factor | 32-bit single channel | Second independent 16-bit channel |
|---|---|---|
| Real ceiling gain | ~2× (1.24 → ~2.48 GB/s) | ~2× (aggregate, same total) |
| Pin cost | Reuses/extends the existing MIG's own bank(s); no new bank claimed | Would claim banks 14 **and/or** 15 (the only banks with free DQS-capable pins) |
| **Conflict with already-placed I/O** | None | **Real, direct**: the management-SPI bus (bank 15: A15/B16/B17/A16) and the config-flash bus (bank 14: K17/K18/L13) are already placed in exactly the banks that would need to host a second channel. Only bank 16 (11 pins) would remain free — not enough margin for either SPI bus, let alone both. |
| Controller logic cost | One MIG instance, wider data path (mostly automatic — the MIG wizard regenerates CAS/CWL/MMCM ratios for the new width) | A full second MIG instance: second calibration sequence, second `ui_clk` domain, and critically the **DDRManager/arbiter would need to become channel-aware**, not just requester-aware — real added complexity on top of §5.2's own design, not a simplification |
| Real risk given current thin margin (+0.068ns) | Lower — one controller, one clock domain, incremental change to an already-proven design | Higher — two independent PHYs, two calibration state machines, cross-channel coordination logic, all new |
| PCB impact | None (same pins, same DDR3 part, different bus width usage — **the physical board the user is designing does not need to change** for this) | Would require re-routing/relocating whichever board-level bus (SPI mgmt or flash) currently occupies bank 14/15 pins — real PCB-level rework, not just RTL |
**Recommendation (honest, not deferential, as requested)**: **32-bit single-
channel widening**, not a second independent channel. The bandwidth gain is
identical, but the 32-bit path has zero pin conflicts with already-placed,
already-verified I/O (SPI management bus, config-flash bus), a much smaller
real risk profile against the current thin timing margin, doesn't require
a second full MIG/calibration instance, and — most importantly for the
user's own board — needs **no PCB changes**, since it reuses the DDR3 part's
own existing data pins at a wider access width rather than claiming new
banks. A second channel's only real advantage (aggregate bandwidth could in
principle scale further with a 3rd/4th channel later) doesn't apply here —
this package genuinely has no more free DQS-capable banks to grow into
after banks 14/15/34/35, so there's no future-proofing benefit being given
up. **User confirmed this recommendation and it is the decided path
forward.**
**What this requires (not yet done, real, disclosed)**: the real Xilinx MIG
"Customize IP" wizard must be re-run interactively (Data Width 16→32 AND
Input Clock Period both changed in the *same* wizard session, since both
require the wizard's own JEDEC/PLL calculator to recompute CAS Latency/CWL/
MMCM ratios correctly — this is **not** safe to hand-edit in `mig_a.prj` the
way the earlier `TargetFPGA` speed-grade field was, per this project's own
established discipline). This is a real, outstanding, user-gated
prerequisite before §5.2's DDRManager and any N>2 scaling test can use the
wider channel.
### 5.5 Scaling path recommendation (real numbers, not a guess) — updated per user's final directive
Given §3's real bandwidth ceiling: **scaling core count alone, without
§5.1/§5.2, provides no real additional throughput past whatever N already
saturates the 1.24 GB/s ceiling** — back-of-envelope, using §3.2's numbers,
that's already close to N=1 in the worst case, and at most N≈2 in the best
(zero-row-switch) case. **Building N=4/8/16 today, before §5.1, would very
likely show near-IDENTICAL real throughput to N=2 in a real P&R'd, real-
DDR3-simulated test** — a real, wasted engineering cycle the analysis
recommends avoiding.
§5.2/§5.4, provides no real additional throughput past whatever N already
saturates the (post-§5.1) ~2.48 GB/s-equivalent demand ceiling** —
back-of-envelope, using §3.2's current numbers (2.48 GB/s needed per core
at peak, 1.24 GB/s physically available today pre-widening), that's already
close to N≈1 in the worst case and at most N≈2 in the best (zero-row-switch)
case on the current 16-bit channel. **Building N=4/8/16 before §5.2/§5.4 are
in place would very likely show near-IDENTICAL real throughput to N=2** — a
real, wasted engineering cycle the analysis recommends avoiding.
**Recommended real order of work**:
1. §5.3 (result-writeback) — genuine blocker, bounded scope.
2. §5.1 (denser activation packing) — highest real leverage on the actual
ceiling, needs a real P&R check given the thin margin.
3. Re-measure real achievable throughput at N=2 with §5.1 in place (real
simulation, real cycle counts) — THIS number, not a projection, should
decide whether N=4 is worth building next.
4. §5.2 (DDRManager/prefetch) — real, valuable, but its benefit is easier to
quantify and justify once §5.1 has already raised the ceiling it's
filling.
5. Only then: N=4/8/16, each with its own real P&R signoff (the margin is
**Decided real order of work** (§5.1 already done; this reflects the user's
own explicit final direction — 32-bit widening, then a complete DDRManager,
then N=2/4/8/16 tests, real target N=8, N=16 built specifically to document
where/how it breaks rather than to succeed):
1. ~~§5.3 (result-writeback)~~ / ~~§5.1 (denser activation packing)~~ — §5.1
**DONE** (EXP-0081/0082). §5.3 remains a genuine blocker for N>2 and must
land before any scaling test that needs real result data out of more than
2 cores' worth of pins.
2. §5.4 (32-bit channel widening) — **decided**, user-gated on a real
interactive MIG wizard session (Data Width + Input Clock Period changed
together). Doubles the physical ceiling itself, which §5.1 alone could
not do.
3. §5.2 (DDRManager, "evoluto e completo" per the user's own spec) — build
against the widened channel so its benefit is measured against the real
final bandwidth budget, not the pre-widening one.
4. Real N=2/4/8/16 tests, each with its own real P&R signoff (margin is
thin, §2 — do not assume a prior N's timing closure predicts the next).
N=8 is the real target configuration; N=16 is expected to expose real
bus/arbitration/timing limits and is built specifically to document that
breakdown, not to be a viable production configuration.
---
@@ -339,11 +447,14 @@ recommends avoiding.
| Claim | Status |
|---|---|
| WNS/WHS/LUT/DSP numbers throughout | **Measured** (real Vivado P&R reports) |
| DDR3 back-to-back burst throughput (1.24 GB/s) | **Measured** (real `ddr3_model.sv` JEDEC trace) |
| WNS/WHS/LUT/DSP numbers throughout | **Measured** (real Vivado P&R reports, current: EXP-0082) |
| DDR3 back-to-back burst throughput (1.24 GB/s) | **Measured** (real `ddr3_model.sv` JEDEC trace) — physical channel limit, unchanged by §5.1's packing fix |
| Real Activate→Read latency (16.125 ns) | **Measured** (same trace) |
| Per-core compute throughput (2.48 GMAC/s) | **Calculated** from measured Fmax (155.039MHz) + known, fixed DSP-packing factor |
| Bandwidth needed per core (4.96 GB/s) | **Calculated** from the above + the real, as-built memory layout convention |
| "~25% of peak sustainable" | **Calculated** ratio of the two measured/calculated numbers above |
| Bandwidth needed per core, pre-packing (4.96 GB/s) | **Calculated**, historical (EXP-0079 layout) |
| Bandwidth needed per core, post-packing (2.48 GB/s) | **Calculated** from the real, as-built EXP-0081/0082 memory layout — current |
| "~25% of peak sustainable" (pre-packing) / "~50%" (post-packing, current) | **Calculated** ratios; post-packing figure re-verified against real P&R (EXP-0082) and real simulation (§5.1) |
| I/O bank/DQS pin counts for XC7A100T-CSG324 (banks 14/15/16/34/35) | **Measured** — queried directly from the real Vivado part database for this exact part/package, used in §5.4's dual-channel-vs-widening analysis |
| Row-switch penalty as a fraction of real workloads | **Not measured** — depends on host-chosen memory layout, flagged as an open question, not asserted |
| DDRManager's real stall-reduction benefit | **Not measured** — no prototype exists yet; §5.2 recommends building a minimal version specifically to get this real number |
| 32-bit widening's real post-change bandwidth/timing numbers | **Not measured** — requires the user's own interactive MIG wizard session (§5.4); this document's ~2.48 GB/s figure is a doubling projection, not yet re-verified by real P&R/simulation |
docs/PHYSICAL_REALIZATION.md
+43 -15
View File
@@ -118,7 +118,7 @@ these from an ESP32 doing real JTAG bit-banging (TAP state machine, IR/DR
shifting) rather than a bench programmer — that firmware is separate,
software-side work, not covered here.
## 3. Real timing signoff (EXP-0078, the current, trustworthy number)
## 3. Real timing signoff (EXP-0082, the current, trustworthy number)
Real in-context Vivado place-and-route (not out-of-context, not estimated):
@@ -126,14 +126,23 @@ Real in-context Vivado place-and-route (not out-of-context, not estimated):
|---|---|
| DDR3 PHY clock (sys_clk_i) | **310.078 MHz** (3.225ns period) |
| Compute domain clock (ui_clk, PLL-derived 2:1 from sys_clk_i) | **155.039 MHz** |
| WNS (setup slack) | **+0.013 ns** — real, but very thin. Re-verify with a fresh P&R after ANY further logic addition. |
| WHS (hold slack) | +0.032 ns |
| Failing endpoints | 0 / 17473 (setup), 0 / 17470 (hold) |
| LUTs used | 5213 / 63400 (8.22%) |
| DSP48E1 used | 16 / 240 (6.67%) — 8 per compute core × 2 cores |
| WNS (setup slack) | **+0.068 ns** — real, still thin but improved vs EXP-0078/0079. Re-verify with a fresh P&R after ANY further logic addition. |
| Failing endpoints | 0 (setup), 0 (hold) |
| LUTs used | 5437 / 63400 (8.58%) |
| DSP48E1 used | 16 / 240 (6.67%) — 8 per compute core × 2 cores, unchanged since EXP-0059 |
| Block RAM used | 0 |
| STARTUPE2 used | 1 / 1 (100%) — the config-flash bridge |
Signoff history (every real change, same target, in-context P&R):
| EXP | WNS (ns) | LUTs | DSP48E1 |
|---|---|---|---|
| 0074 (first real DDR3 + pins) | +0.040 | 5140 | 16 |
| 0076 (+regfile, +pins, +SPI fix) | +0.056 | 5173 | 16 |
| 0078 (+flash bridge, STARTUPE2) | +0.013 | 5213 | 16 |
| 0079 (+real activation engine) | +0.030 | 5379 | 16 |
| 0082 (+denser activation packing) | **+0.068** | 5437 | 16 |
## 4. Real DDR3 memory layout convention
Both weight data and activation data share the same DDR3 address space
@@ -143,17 +152,36 @@ Both weight data and activation data share the same DDR3 address space
WORDS_PER_LAYER` (`WORDS_PER_LAYER = LAYER_BYTES/2`). Densely packed —
`layer_prefetch_ctrl.v` reads full bursts sequentially into the on-chip
weight buffer once per job.
- **Activations** (real engine since EXP-0079, `act_tile_fetch.v`): **each
tile (P_IN=8 INT8 values) occupies its own full `BURST_LEN=8`-word
(128-bit) burst slot** — the 8 useful bytes sit in the low 64 bits, the
upper 64 bits are unused padding. This is deliberately 2× wasteful of DDR3
capacity, in exchange for needing zero runtime-indexed bit-selects in the
fetch logic (a real Fmax risk this project's thin P&R margin, §3, can't
currently afford). Tile `t`'s word address is `base + t*BURST_LEN`, always
burst-aligned by construction.
- **Activations** (real engine, `act_tile_fetch.v`; current layout is the
**v2 convention, EXP-0081/0082**): **two consecutive tiles (P_IN=8 INT8
values each) share ONE full `BURST_LEN=8`-word (128-bit) burst** — the
even-indexed tile occupies the low 64 bits, the odd-indexed tile occupies
the high 64 bits. Tile `t`'s burst address is `base + (t>>1)*BURST_LEN`
(integer division — two tiles per burst), always burst-aligned by
construction. This **halves** real DDR3 bytes-moved-per-useful-byte versus
the original EXP-0079 "1 tile = 1 burst" layout, which wasted the upper 64
bits of every burst as padding.
- **Why this is timing-safe despite selecting a sub-burst half at read
time**: the tile index's LSB (which half of the burst a given tile
lives in) is known at *request* time, not at response time. It's
latched into a register (`sel_lat`) the same cycle the request is
accepted — many `ui_clk` cycles before the real DDR3 round-trip
completes and `ctrl_rdata` becomes valid. The eventual data-select mux
therefore always selects on an already-long-stable registered bit,
never a bit racing live read data — this is NOT the runtime-indexed-
part-select-on-the-critical-path pattern flagged as a real Fmax risk in
EXP-0061 (that pattern is about a select signal arriving *late*,
simultaneously with the data it gates). Confirmed timing-safe by real
P&R (EXP-0082): margin *improved* from +0.030ns to +0.068ns, not
degraded.
- Real measured effect: back-to-back same-row DDR3 throughput is a fixed
1.24 GB/s (measured, EXP-0080) regardless of packing — this convention
doesn't change that ceiling, it changes how much of it is *wasted* on
padding, doubling the real achievable useful fraction (see
`docs/ARCHITECTURE_ANALYSIS.md` §3 and §5.1).
- `base` (a job's own `x_base_a`/`x_base_b`) is chosen freely by whoever
submits jobs (the SPI host) — just keep each position's own activation
array in its own non-overlapping `N_TILES * BURST_LEN`-word region.
array in its own non-overlapping `(N_TILES/2) * BURST_LEN`-word region.
## 5. FPGA configuration (boot) procedure