diff --git a/docs/ARCHITECTURE_ANALYSIS.md b/docs/ARCHITECTURE_ANALYSIS.md index 311d007..e87929b 100644 --- a/docs/ARCHITECTURE_ANALYSIS.md +++ b/docs/ARCHITECTURE_ANALYSIS.md @@ -1,11 +1,16 @@ # FPGA-Neural V3 — Architecture Analysis: Timing, Bottlenecks, and Recommended Interventions Scope: the current, real, P&R-verified V3 design (`hardware/v3/`, branch -`v3-artix7`), updated through EXP-0088 (result-writeback engine, real, -closed timing). The current real, trustworthy *timing* signoff is now -EXP-0088 (32-bit, 155.039MHz, WNS +0.100ns), replacing EXP-0086's own -pointer (which itself replaced EXP-0083's 16-bit-era one). Every number -in this document is either +`v3-artix7`), updated through EXP-0095/0096 (N=8 hybrid systolic, real, +closed timing — the current, definitive real deployment target, chosen +by the user after a real, measured N=4/8/16 timing curve). The current +real, trustworthy *timing* signoff is now EXP-0095/0096 +(`n8_system_ddr3_top.v`, 32-bit DDR3, 155.039MHz, WNS 0.000ns, 8 real +parallel PEs), replacing EXP-0088's own N=2 pointer (which itself +replaced EXP-0086's, which replaced EXP-0083's 16-bit-era one). See §5.6 +for the full real story, including N=16's own real RTL (functionally +verified but NOT timing-closed, kept as documented future work). Every +number in this document is either directly measured (real simulation trace, real P&R report) or a calculation built from directly-measured building blocks — the two are labeled explicitly throughout. Nothing here is guessed. @@ -659,7 +664,7 @@ specifically to document where/how it breaks rather than to succeed): --- -### 5.6 [N=8 REALLY CLOSES timing (0 failing endpoints); N=16 still -0.338ns — EXP-0089…0095] Hybrid systolic scaling: up to 4 groups × 4-PE weight-stationary chains +### 5.6 [N=8 IS THE REAL, DEFINITIVE DEPLOYMENT TARGET — `n8_system_ddr3_top.v`, EXP-0089…0096] Hybrid systolic scaling: up to 4 groups × 4-PE weight-stationary chains Captured from a 2026-09-20 brainstorming session as a purely exploratory idea; the same day, per the user's own explicit reprioritization, the real @@ -832,16 +837,36 @@ and clock domain, not a projection. N=4 is a hair's breadth away placer/router variance, not a contradiction). **The real failure point is specifically between N=8 (closed) and N=16 (EXP-0094: −0.338ns)** — N=12 (N_GROUPS=3) has not yet been measured and would narrow this -further. Real, disclosed caveat: N=4/N=8 have not yet had their own -dedicated functional xsim (only N=16's full-scale RTL is functionally -verified, EXP-0092/0094) — the correctness argument for N=4/N=8 rests -on the architecture's own embarrassingly-parallel-across-groups design, -not yet a directly measured result at those specific N. Real bug found -and fixed first: `neural_director_grouped.v`'s own bare -`$clog2(N_GROUPS)` was invalid (`[-1:0]`) for the never-before-tested -N_GROUPS=1 case — fixed with the same real `SELW`-style guard pattern -`sdram_arbiter_n.v` already established. See EXP-0095's own -`next_action`. +further. Real bug found and fixed first: `neural_director_grouped.v`'s +own bare `$clog2(N_GROUPS)` was invalid (`[-1:0]`) for the +never-before-tested N_GROUPS=1 case — fixed with the same real +`SELW`-style guard pattern `sdram_arbiter_n.v` already established. + +**N=8 promoted to the real, definitive deployment target (EXP-0096)** +— the user's own explicit decision ("creiamo una versione funzionante +completamente per N=8, la pushiamo come versione definitiva"). New +`hardware/v3/rtl/n8_system_ddr3_top.v` — a real, permanent, named +top-level (not a build-time override), byte-for-byte the same RTL as +`n16_system_ddr3_top.v` with `N_GROUPS` defaulting to 2. Real, full +P&R re-run under this file's own name gives the EXACT SAME numbers as +the generic-override result (WNS=0.000ns, WHS=+0.017ns, 0 failing +endpoints, 64 DSP48E1/26.7%, 12535 LUTs/19.77%) — confirms the file +split is correct. New `tb_n8_system_ddr3.v` (real DDR3-model +methodology, M=16 positions, every one of the 2 groups × 4 PEs × 2 +lanes exercised exactly once): **real functional xsim, 16/16 PASS, 0 +errors** — closes the real, disclosed functional-verification gap +this section previously flagged for N=8 specifically. + +**Current real status: N=8 (`n8_system_ddr3_top.v`) is BOTH +functionally verified AND timing-closed under its own permanent real +name — the definitive deployment target.** N=2 (`n2_system_ddr3_top.v`, +EXP-0088) remains documented as a real, valid, simpler fallback. N=16 +(`n16_system_ddr3_top.v`) remains real, functionally-verified RTL, not +abandoned, just not timing-closed and not the current target. Real, +disclosed caveat carried forward: N=8's WNS=0.000ns is an exact-zero +margin — any future RTL change touching this top-level or its +dependents needs a fresh real P&R (same `Explore`/`ExtraNetDelay_high`/ +`AggressiveExplore` directive stack) before trusting timing again. **The problem it targets**: plain N=16 independent cores (§5.5's own "documentary, expected to break" framing) means 16 independent DDR3 diff --git a/docs/PHYSICAL_REALIZATION.md b/docs/PHYSICAL_REALIZATION.md index 39d4b92..6c98638 100644 --- a/docs/PHYSICAL_REALIZATION.md +++ b/docs/PHYSICAL_REALIZATION.md @@ -165,38 +165,72 @@ these from an ESP32 doing real JTAG bit-banging (TAP state machine, IR/DR shifting) rather than a bench programmer — that firmware is separate, software-side work, not covered here. -## 3. Real timing signoff (EXP-0088, the current, trustworthy number — real 32-bit DDR3 channel + result-writeback engine, CLOSED) +## 3. Real timing signoff (EXP-0095/0096, the current, definitive target — N=8 hybrid systolic, `n8_system_ddr3_top.v`, CLOSED) -Real in-context Vivado place-and-route (not out-of-context, not estimated): +**This is the real, definitive deployment target as of 2026-09-21** +(the user's own explicit decision, after a real, measured N=4/8/16 +timing curve found N=8 to be the largest real configuration that +closes timing cleanly — see `docs/ARCHITECTURE_ANALYSIS.md` §5.6 for +the full real story). Top-level RTL: `hardware/v3/rtl/ +n8_system_ddr3_top.v` — 2 groups × 4 PEs, shared-weight-broadcast +systolic architecture (`systolic_group.v`/`packed_pe.v`, +`neural_director_grouped.v`, `sdram_arbiter_hier.v`'s own real +hierarchical 2-level arbiter), same real DDR3/SPI/flash bridge as +every other real V3 top-level. Real in-context Vivado place-and-route +(not out-of-context, not estimated): | Metric | Value | |---|---| | DDR3 PHY clock (sys_clk_p/n) | **310.078 MHz** (3.225ns period) | | Compute domain clock (ui_clk/clk_pll_i, PLL-derived 2:1 from sys_clk) | **155.039 MHz** | -| WNS (setup slack) | **+0.099962 ns** — real, closed. Re-verify with a fresh P&R after ANY further logic addition. | -| WHS (hold slack) | +0.036275 ns | -| Failing endpoints | 0 of 27868 (setup), 0 of 27865 (hold), 0 of 10529 (pulse width) | -| LUTs used | 6642 / 63400 (10.48%) | -| DSP48E1 used | 16 / 240 (6.67%) — 8 per compute core × 2 cores, unchanged since EXP-0059 | -| Block RAM used | 0 | -| STARTUPE2 used | 1 / 1 (100%) — the config-flash bridge | +| WNS (setup slack) | **0.000 ns** — real, closed, 0 failing setup endpoints (EXP-0095/0096). Re-verify with a fresh P&R after ANY further logic addition — this margin is exactly zero, no slack to spare. | +| WHS (hold slack) | +0.017 ns | +| Failing endpoints | 0 of 63212 (setup), 0 of 63209 (hold) | +| LUTs used | 12535 / 63400 (19.77%) | +| Registers used | 19902 / 126800 (15.70%) | +| DSP48E1 used | 64 / 240 (26.7%) — 8 per PE × 8 real parallel PEs | | DDR3 channel width | **32-bit real physical channel** (two MT41J128M16JT-125:K chips) — see §3.1 | -| Real physical bandwidth ceiling | **~2.48 GB/s** (2× EXP-0083's 16-bit ~1.24 GB/s, real, closed timing — not a projection) | -| Result writeback | **real, into DDR3** (EXP-0088, `result_writeback.v`) — no longer literal top-level pins, see §4.6/§5.3 of `docs/ARCHITECTURE_ANALYSIS.md` | +| Real physical bandwidth ceiling | **~2.48 GB/s** (unchanged from the N=2 baseline — same DDR3 channel, shared across 8 real parallel PEs instead of 2) | +| Real parallelism | **8 real parallel processing elements** (4× the original N=2 design) | +| Result writeback | **real, into DDR3** (`result_writeback.v`), see §4.6/§5.3 of `docs/ARCHITECTURE_ANALYSIS.md` | -Signoff history (every real change, same target, in-context P&R): +**Real, disclosed caveat**: this WNS=0.000ns margin is exactly zero — +real, closed, but with zero slack. Re-run a real P&R (same +`opt_design -directive Explore` / `place_design -directive +ExtraNetDelay_high` / `phys_opt_design -directive AggressiveExplore` / +`route_design -directive AggressiveExplore` directive stack used to +reach this result, EXP-0094/0095) after ANY further RTL change before +trusting timing again — do not assume it still holds. -| EXP | WNS (ns) | LUTs | DSP48E1 | -|---|---|---|---| -| 0074 (first real DDR3 + pins) | +0.040 | 5140 | 16 | -| 0076 (+regfile, +pins, +SPI fix) | +0.056 | 5173 | 16 | -| 0078 (+flash bridge, STARTUPE2) | +0.013 | 5213 | 16 | -| 0079 (+real activation engine) | +0.030 | 5379 | 16 | -| 0082 (+denser activation packing) | +0.068 | 5437 | 16 | -| 0083 (+DDRManager phase 1, `ddr_prefetch_mgr.v`, 16-bit) | +0.073 | 5644 | 16 | -| 0084 (32-bit DDR3 widening, 2900ps/172.414MHz) | -0.618 (FAILED) | 6418 | 16 | -| 0086 (32-bit DDR3 widening, clock reverted to 3225ps/155.039MHz) | +0.096 (CLOSED) | 6382 | 16 | -| 0088 (+result-writeback engine, `result_writeback.v`) | **+0.100 (CLOSED)** | 6642 | 16 | +**Superseded, real predecessor signoff (N=2, flat architecture, +`n2_system_ddr3_top.v`, EXP-0088)** — kept below for reference; still +a real, valid, simpler fallback target if N=8's zero-margin closure +ever proves too fragile in practice: + +| Metric | Value | +|---|---| +| WNS (setup slack) | +0.099962 ns | +| WHS (hold slack) | +0.036275 ns | +| LUTs used | 6642 / 63400 (10.48%) | +| DSP48E1 used | 16 / 240 (6.67%) — 8 per compute core × 2 cores | +| Real parallelism | 2 real parallel processing elements | + +Signoff history (every real change, in-context P&R): + +| EXP | Target | WNS (ns) | LUTs | DSP48E1 | +|---|---|---|---|---| +| 0074 (first real DDR3 + pins) | N=2 | +0.040 | 5140 | 16 | +| 0076 (+regfile, +pins, +SPI fix) | N=2 | +0.056 | 5173 | 16 | +| 0078 (+flash bridge, STARTUPE2) | N=2 | +0.013 | 5213 | 16 | +| 0079 (+real activation engine) | N=2 | +0.030 | 5379 | 16 | +| 0082 (+denser activation packing) | N=2 | +0.068 | 5437 | 16 | +| 0083 (+DDRManager phase 1, `ddr_prefetch_mgr.v`, 16-bit) | N=2 | +0.073 | 5644 | 16 | +| 0084 (32-bit DDR3 widening, 2900ps/172.414MHz) | N=2 | -0.618 (FAILED) | 6418 | 16 | +| 0086 (32-bit DDR3 widening, clock reverted to 3225ps/155.039MHz) | N=2 | +0.096 (CLOSED) | 6382 | 16 | +| 0088 (+result-writeback engine, `result_writeback.v`) | N=2 | +0.100 (CLOSED) | 6642 | 16 | +| 0093 (hybrid systolic, flat 21-way arbiter) | N=16 | -0.913 (FAILED) | 19751 | 128 | +| 0094 (+hierarchical 2-level arbiter, +P&R directive tuning) | N=16 | -0.338 (still FAILED) | 19936 | 128 | +| 0095/0096 (`n8_system_ddr3_top.v`, hybrid systolic) | **N=8** | **0.000 (CLOSED)** | 12535 | 64 | ### 3.1 Real 32-bit DDR3 widening — functionally verified AND timing CLOSED (EXP-0084 → EXP-0086) @@ -363,12 +397,17 @@ when it actually fires, instead of polling every loop iteration. ## 7. Known-open items (honestly disclosed, not hidden) -- Scaling past N=2 compute cores (silicon budget allows up to ~30 per the - DSP48E1 count) is not yet built or timing-verified. The real pin-budget - blocker that used to require this (`result_data_a/b` as literal - top-level pins) is now REMOVED (EXP-0088, `result_writeback.v` — see - `docs/ARCHITECTURE_ANALYSIS.md` §4.6/§5.3) — real N=2/4/8/16 P&R - scaling tests are the next real milestone, no longer blocked. +- Scaling past N=2 compute cores: **RESOLVED, real N=8 is now the + definitive deployment target** (§3, EXP-0095/0096, `n8_system_ddr3_ + top.v`, a real hybrid systolic shared-weight-broadcast architecture, + 2 groups × 4 PEs, WNS=0.000ns CLOSED). Real N=4/8/16 P&R curve + measured: N=4 −0.005ns (2 failing endpoints, essentially at the + edge), N=8 **0.000ns (CLOSED)**, N=16 −0.338ns (still failing even + after a real hierarchical-arbiter fix + P&R directive tuning, EXP- + 0093/0094) — see `docs/ARCHITECTURE_ANALYSIS.md` §5.6 for the full + real story. N=16 remains real, functionally verified RTL + (`n16_system_ddr3_top.v`), just not yet timing-closed; not the + current real deployment target. - `sys_rst` has only a *tentative* real pin (G13, bank 15, EXP-0084 — chosen just to unblock real P&R, not a final board decision) — assign its real, permanent location once the rest of the board layout (reset @@ -392,6 +431,6 @@ when it actually fires, instead of polling every loop iteration. checkpoint) — see §6. - `result_writeback.v` (EXP-0088) is real, built, and verified (9/9 PASS including a new real DDR3 read-after-write check, `tb_packed_slot.v`). - Real P&R re-verification for this specific addition is the next real - step, together with the N=2/4/8/16 core-count scaling tests it - unblocks — see `docs/ARCHITECTURE_ANALYSIS.md` §5.3/§5.5. + P&R re-verified with the result-writeback addition (EXP-0088) and, + since then, real N=4/8/16 scaling has been built and P&R-tested — see + §3 above and `docs/ARCHITECTURE_ANALYSIS.md` §5.3/§5.6. diff --git a/docs/PINOUT.md b/docs/PINOUT.md index 58f4deb..a4ffc6d 100644 --- a/docs/PINOUT.md +++ b/docs/PINOUT.md @@ -6,9 +6,13 @@ board). This is a flattened, board-layout-ready extract of narrated source (it explains *why* each pin was chosen and every real conflict/fix found along the way); this file is the quick reference for schematic capture. Every pin below comes from a real routed design -(`n2_system_ddr3_top_routed.dcp`, EXP-0086 signoff) or the MIG-generated -`mig_7series_0.xdc` — none of it is guessed. Re-generate this file if -`docs/PHYSICAL_REALIZATION.md` §2 ever changes. +(originally `n2_system_ddr3_top_routed.dcp`, EXP-0086; the same real pin +set — DDR3/SPI/flash/config are all package-level, unaffected by internal +core count — was re-confirmed real and unchanged in `n8_system_ddr3_top.v`'s +own real routed signoff, EXP-0095/0096, now the current definitive +deployment target, see `docs/PHYSICAL_REALIZATION.md` §3) or the +MIG-generated `mig_7series_0.xdc` — none of it is guessed. Re-generate +this file if `docs/PHYSICAL_REALIZATION.md` §2 ever changes. ## Bank summary diff --git a/hardware/v2/logs/experiments.log b/hardware/v2/logs/experiments.log index a0a1377..c6a0d2f 100644 --- a/hardware/v2/logs/experiments.log +++ b/hardware/v2/logs/experiments.log @@ -6720,3 +6720,67 @@ DIRECTLY measured at N=16 so far). (2) optionally, real P&R at N=12 (N_GROUPS=3) to pin down the exact real N where the WNS trend crosses zero. (3) optionally, a real re-attempt at N=4 (likely closes on retry, given how close -0.005ns already is). + +EXP-0096 -- N=8 promoted to the real, definitive deployment target: +dedicated top-level, real P&R re-confirmed under its own name, real +functional xsim 16/16 PASS (2026-09-21, user's own explicit direction: +"si creaiamo una versione funzionante completamente per n=8 la +pushiamo come versione definitiva") + +CONTEXT: EXP-0095's own real N=8 closure (WNS=0.000ns) was measured via +a build-time `synth_design -generic N_GROUPS=2` override against +n16_system_ddr3_top.v -- real and trustworthy, but not yet a real, +permanent, named top-level file (this project's own established +convention: one real file per real deployment configuration, e.g. +n2_system_ddr3_top.v), and not yet functionally verified at this +specific N (only N=16's full-scale RTL had a dedicated functional +xsim, EXP-0092/0094). + +METHOD: (1) new `hardware/v3/rtl/n8_system_ddr3_top.v` -- byte-for-byte +the same real RTL structure as n16_system_ddr3_top.v, N_GROUPS +defaulting to 2 instead of 4 (still a real, visible parameter, not +hardcoded away), real header documenting this as the definitive +2026-09-21 target. (2) new `hardware/v3/sim/tb_n8_system_ddr3.v`, +directly adapted from tb_n16_system_ddr3.v's own real DDR3-model +methodology (same real 2-chip 32-bit DDR3 model, same real +mig_7series_0_mig, real Vivado xsim), scaled to N_GROUPS=2/M=16 +positions (exercises every one of the 2 groups x 4 PEs x 2 lanes +exactly once). (3) real, full P&R re-run under n8_system_ddr3_top.v's +own real name (added to the real Vivado project's sources_1 fileset as +a direct reference, real stale-import check confirmed empty before +trusting the result) with the same real directive stack that closed +N=8 the first time (EXP-0094/0095: Explore/ExtraNetDelay_high/ +AggressiveExplore). + +REAL RESULT: P&R under the real n8_system_ddr3_top.v name gives the +EXACT SAME real numbers as EXP-0095's own generic-override result -- +**WNS=0.000ns, TNS=0.000ns, 0 failing setup endpoints, WHS=+0.017ns**, +12535 LUTs (19.77%), 19902 registers (15.70%), 64 DSP48E1/240 (26.7%) +-- confirms the real file split was done correctly, byte-for-byte +equivalent RTL. Real functional xsim (`tb_n8_system_ddr3.v`, real DDR3 +model, real Vivado xsim): **16/16 PASS, 0 errors**, `$finish` at +131145.8335ns -- closes the real, disclosed functional-verification +gap for this specific N. + +DECISION: N=8 (`n8_system_ddr3_top.v`) is now the real, definitive +deployment target -- BOTH functionally verified AND timing-closed +under its own permanent real name, not a build-time override. Real, +disclosed caveat carried forward: WNS=0.000ns is an exact-zero real +margin, not a comfortable one -- any future RTL change to this top- +level or its dependents needs a fresh real P&R (same directive stack) +before trusting timing again, per this project's own "re-verify after +any further logic addition" discipline. + +`docs/PHYSICAL_REALIZATION.md` §3 and §7, and +`docs/ARCHITECTURE_ANALYSIS.md`'s own top-of-document pointer, updated +to promote this as the current real signoff (replacing the EXP-0088 +N=2 pointer) -- N=2 (`n2_system_ddr3_top.v`) kept documented as a real, +valid, simpler fallback; N=16 (`n16_system_ddr3_top.v`) kept documented +as real, functionally-verified-but-not-timing-closed future work, not +abandoned. + +next_action: real board bring-up planning (schematic capture from +docs/PINOUT.md, BOM confirmation from docs/BOM.md -- both already real +and unaffected by the N_GROUPS choice, since DDR3/SPI/flash/config +pins are package-level, not internal-core-count-dependent) is the real +next milestone now that a real, closed, deployable RTL target exists. diff --git a/hardware/v3/rtl/n8_system_ddr3_top.v b/hardware/v3/rtl/n8_system_ddr3_top.v new file mode 100644 index 0000000..130bf7e --- /dev/null +++ b/hardware/v3/rtl/n8_system_ddr3_top.v @@ -0,0 +1,302 @@ +`timescale 1ns/1ps + +// ============================================================ +// V3 -- REAL synthesis/P&R top for the N=8 (2 groups x 4 PEs) hybrid +// systolic system, the real, definitive deployment target (EXP-0095/ +// 0096): byte-for-byte the SAME real RTL as n16_system_ddr3_top.v +// (EXP-0089..0094), just with N_GROUPS defaulting to 2 instead of 4 -- +// forked into its own real top-level file (not just a build-time +// `-generic` override) so the definitive N=8 hardware target has a +// permanent, unambiguous real source of truth, matching this project's +// own established one-file-per-real-configuration convention +// (n2_system_ddr3_top.v, n16_system_ddr3_top.v). +// +// REAL, MEASURED RESULT THIS FILE EXISTS TO CAPTURE (EXP-0095): real, +// full P&R (synth_design+opt_design+place_design+phys_opt_design+ +// route_design, same real XC7A100T-CSG324-2 part, same real 155.039MHz +// clk_pll_i domain N=2's own EXP-0088 signoff was measured on) gives +// **WNS=+0.000ns, TNS=0.000ns, 0 FAILING SETUP ENDPOINTS** -- a real, +// closed timing signoff, not a projection (measured via +// `-generic N_GROUPS=2` against n16_system_ddr3_top.v before this file +// existed; re-confirmed against this file's own real name/module in +// EXP-0096). 64 DSP48E1/240 (26.7%), matching the real 8 DSP/PE x 8 PE +// projection exactly. +// +// Everything else — MIG, mig_native_adapter.v, sdram_arbiter_hier.v +// (EXP-0094's own real hierarchical arbiter, already scales correctly +// to N_GROUPS=2's own real 3-way top level: 2 groups + 1 host), +// neural_director_grouped.v, spi_host_bridge_v3.v, flash_spi_master.v, +// host_mem_bridge.v — completely unmodified from n16_system_ddr3_top.v, +// same real zero-protocol-change finding (EXP-0090) still holds: +// N_SLOTS=8 passed to spi_host_bridge_v3.v purely informationally. +// +// ARBITER SLOT MAP (real, NUM_REQ=11): slots 0-1 = groups 0-1's own +// weight-fetch; slots 2-9 = the 8 PEs' own activation-fetch+writeback, +// 4 consecutive slots per group (group g's PEs at slots 2+4g..2+4g+3); +// slot 10 = host_mem_bridge.v. +// ============================================================ +module n8_system_ddr3_top #( + parameter DATA_WIDTH = 8, + parameter P_IN = 8, + parameter ACC_WIDTH = 32, + parameter BURST_LEN = 8, + parameter JOB_ADDR_WIDTH = 26, + parameter MEM_ADDR_WIDTH = 25, + parameter LAYER_BYTES = 128, + parameter N_GROUPS = 2, + parameter QUEUE_DEPTH = 16 +)( + input wire sys_clk_p, + input wire sys_clk_n, + input wire sys_rst, + input wire clk_ref_p, + input wire clk_ref_n, + + inout wire [31:0] ddr3_dq, + inout wire [3:0] ddr3_dqs_n, + inout wire [3:0] ddr3_dqs_p, + output wire [13:0] ddr3_addr, + output wire [2:0] ddr3_ba, + output wire ddr3_ras_n, + output wire ddr3_cas_n, + output wire ddr3_we_n, + output wire ddr3_reset_n, + output wire [0:0] ddr3_ck_p, + output wire [0:0] ddr3_ck_n, + output wire [0:0] ddr3_cke, + output wire [0:0] ddr3_cs_n, + output wire [3:0] ddr3_dm, + output wire [0:0] ddr3_odt, + + input wire sclk, + input wire mosi, + output wire miso, + input wire cs_n, + + output wire flash_cs_n, + output wire flash_mosi, + input wire flash_miso, + + output wire ui_clk_o, + output wire init_calib_complete, + output wire job_out_done, + + output wire data_ready_n +); + localparam N_PES = N_GROUPS * 4; // 8 + + wire [27:0] app_addr; + wire [2:0] app_cmd; + wire app_en, app_rdy; + wire [127:0] app_wdf_data; + wire app_wdf_end; + wire [15:0] app_wdf_mask; + wire app_wdf_wren, app_wdf_rdy; + wire [127:0] app_rd_data; + wire app_rd_data_end, app_rd_data_valid; + wire ui_clk, ui_clk_sync_rst; + + assign ui_clk_o = ui_clk; + + mig_7series_0 u_mig ( + .ddr3_dq(ddr3_dq), .ddr3_dqs_n(ddr3_dqs_n), .ddr3_dqs_p(ddr3_dqs_p), + .ddr3_addr(ddr3_addr), .ddr3_ba(ddr3_ba), + .ddr3_ras_n(ddr3_ras_n), .ddr3_cas_n(ddr3_cas_n), .ddr3_we_n(ddr3_we_n), + .ddr3_reset_n(ddr3_reset_n), + .ddr3_ck_p(ddr3_ck_p), .ddr3_ck_n(ddr3_ck_n), + .ddr3_cke(ddr3_cke), .ddr3_cs_n(ddr3_cs_n), + .ddr3_dm(ddr3_dm), .ddr3_odt(ddr3_odt), + .sys_clk_p(sys_clk_p), .sys_clk_n(sys_clk_n), .clk_ref_p(clk_ref_p), .clk_ref_n(clk_ref_n), + .app_addr(app_addr), .app_cmd(app_cmd), .app_en(app_en), + .app_wdf_data(app_wdf_data), .app_wdf_end(app_wdf_end), + .app_wdf_mask(app_wdf_mask), .app_wdf_wren(app_wdf_wren), + .app_rd_data(app_rd_data), .app_rd_data_end(app_rd_data_end), + .app_rd_data_valid(app_rd_data_valid), .app_rdy(app_rdy), .app_wdf_rdy(app_wdf_rdy), + .app_sr_req(1'b0), .app_ref_req(1'b0), .app_zq_req(1'b0), + .app_sr_active(), .app_ref_ack(), .app_zq_ack(), + .ui_clk(ui_clk), .ui_clk_sync_rst(ui_clk_sync_rst), + .init_calib_complete(init_calib_complete), + .device_temp(), + .sys_rst(sys_rst) + ); + + wire adp_req, adp_wr; + wire [MEM_ADDR_WIDTH-1:0] adp_addr; + wire [32*BURST_LEN-1:0] adp_wdata; + wire [4*BURST_LEN-1:0] adp_wmask; + wire [32*BURST_LEN-1:0] adp_rdata; + wire adp_ready, adp_busy; + + mig_native_adapter #(.BURST_LEN(BURST_LEN), .ADDR_WIDTH(MEM_ADDR_WIDTH)) u_adapter ( + .clk(ui_clk), .rst(ui_clk_sync_rst), + .req(adp_req), .wr(adp_wr), .addr(adp_addr), .wdata(adp_wdata), .wmask(adp_wmask), + .rdata(adp_rdata), .ready(adp_ready), .busy(adp_busy), + .app_addr(app_addr), .app_cmd(app_cmd), .app_en(app_en), .app_rdy(app_rdy), + .app_wdf_data(app_wdf_data), .app_wdf_end(app_wdf_end), .app_wdf_mask(app_wdf_mask), + .app_wdf_wren(app_wdf_wren), .app_wdf_rdy(app_wdf_rdy), + .app_rd_data(app_rd_data), .app_rd_data_end(app_rd_data_end), .app_rd_data_valid(app_rd_data_valid) + ); + + // ---- real arbiter: 2 group weight-fetch + 8 PE activation/ + // writeback + 1 host_mem_bridge = 11 ---- + localparam NUM_REQ = N_GROUPS + N_PES + 1; // 2 + 8 + 1 = 11 + localparam HOST_SLOT = NUM_REQ - 1; // 10 + + wire [NUM_REQ-1:0] req_active, req_grant, req_req, req_wr; + wire [NUM_REQ-1:0] req_ready, req_busy; + wire [NUM_REQ*MEM_ADDR_WIDTH-1:0] req_addr; + wire [NUM_REQ*32*BURST_LEN-1:0] req_wdata; + wire [NUM_REQ*4*BURST_LEN-1:0] req_wmask; + wire [NUM_REQ*32*BURST_LEN-1:0] req_rdata; + + // real, hierarchical 2-level arbiter (EXP-0094), same real module + // as n16_system_ddr3_top.v -- generalizes cleanly to N_GROUPS=2 + // (2 real leaf instances + a 3-way top instance: 2 groups + host). + sdram_arbiter_hier #( + .N_GROUPS(N_GROUPS), .PES_PER_GROUP(4), .ADDR_WIDTH(MEM_ADDR_WIDTH), .BURST_LEN(BURST_LEN) + ) u_arb ( + .clk(ui_clk), .rst(ui_clk_sync_rst), + .req_active(req_active), .req_grant(req_grant), + .req_req(req_req), .req_wr(req_wr), .req_addr(req_addr), + .req_wdata(req_wdata), .req_wmask(req_wmask), + .req_rdata(req_rdata), .req_ready(req_ready), .req_busy(req_busy), + .ctrl_req(adp_req), .ctrl_wr(adp_wr), .ctrl_addr(adp_addr), + .ctrl_wdata(adp_wdata), .ctrl_wmask(adp_wmask), + .ctrl_rdata(adp_rdata), .ctrl_ready(adp_ready), .ctrl_busy(adp_busy) + ); + + // ---- grouped Director ---- + wire job_in_valid, job_in_ready; + wire [JOB_ADDR_WIDTH-1:0] job_in_x_base, job_in_w_base, job_in_result_addr; + wire [15:0] job_in_n_tiles, job_in_node_id; + + wire [N_GROUPS-1:0] group_job_start; + wire [JOB_ADDR_WIDTH*N_GROUPS-1:0] group_w_base; + wire [16*N_GROUPS-1:0] group_n_tiles; + wire [4*JOB_ADDR_WIDTH*N_GROUPS-1:0] group_pe_x_base_a, group_pe_x_base_b; + wire [4*JOB_ADDR_WIDTH*N_GROUPS-1:0] group_pe_result_addr_a, group_pe_result_addr_b; + wire [4*16*N_GROUPS-1:0] group_pe_node_id_a, group_pe_node_id_b; + wire [N_GROUPS-1:0] group_job_done; + localparam GROUP_IDX_WIDTH = (N_GROUPS <= 1) ? 1 : $clog2(N_GROUPS); + wire [GROUP_IDX_WIDTH-1:0] job_out_group_w; + wire [3:0] dir_state; + wire dir_error; + wire queue_empty; + + neural_director_grouped #( + .ADDR_WIDTH(JOB_ADDR_WIDTH), .N_GROUPS(N_GROUPS), .QUEUE_DEPTH(QUEUE_DEPTH) + ) u_dir ( + .clk(ui_clk), .rst(ui_clk_sync_rst), + .job_in_valid(job_in_valid), .job_in_ready(job_in_ready), + .job_in_x_base(job_in_x_base), .job_in_w_base(job_in_w_base), + .job_in_n_tiles(job_in_n_tiles), .job_in_result_addr(job_in_result_addr), + .job_in_node_id(job_in_node_id), + .group_job_start(group_job_start), .group_w_base(group_w_base), .group_n_tiles(group_n_tiles), + .group_pe_x_base_a(group_pe_x_base_a), .group_pe_x_base_b(group_pe_x_base_b), + .group_pe_result_addr_a(group_pe_result_addr_a), .group_pe_result_addr_b(group_pe_result_addr_b), + .group_pe_node_id_a(group_pe_node_id_a), .group_pe_node_id_b(group_pe_node_id_b), + .group_job_done(group_job_done), + .job_out_done(job_out_done), .job_out_group(job_out_group_w), + .dir_state(dir_state), .dir_error(dir_error), .queue_empty(queue_empty) + ); + + // ---- physical SPI host interface: UNMODIFIED (EXP-0090's own real + // finding -- zero protocol changes needed). N_SLOTS=8 reported + // purely informationally (REG_READ 0x03), matches the real total + // parallel-PE count, never gates any control logic in this module. ---- + wire mem_req, mem_wr, mem_lb_n, mem_ub_n, mem_ready; + wire [MEM_ADDR_WIDTH-1:0] mem_addr; + wire [15:0] mem_wdata, mem_rdata; + wire soft_rst_pulse; + + wire flash_xfer_active, flash_byte_req, flash_byte_done; + wire [7:0] flash_byte_wdata, flash_byte_rdata; + + spi_host_bridge_v3 #( + .JOB_ADDR_WIDTH(JOB_ADDR_WIDTH), .MEM_ADDR_WIDTH(MEM_ADDR_WIDTH), .N_SLOTS(N_PES) + ) u_spi ( + .clk(ui_clk), .rst(ui_clk_sync_rst), + .sclk(sclk), .mosi(mosi), .miso(miso), .cs_n(cs_n), + .init_calib_complete(init_calib_complete), .dir_error(dir_error), + .job_out_done(job_out_done), .data_ready_n(data_ready_n), + .job_in_valid(job_in_valid), .job_in_ready(job_in_ready), + .job_in_x_base(job_in_x_base), .job_in_w_base(job_in_w_base), + .job_in_n_tiles(job_in_n_tiles), .job_in_result_addr(job_in_result_addr), + .job_in_node_id(job_in_node_id), + .mem_req(mem_req), .mem_wr(mem_wr), .mem_addr(mem_addr), + .mem_wdata(mem_wdata), .mem_lb_n(mem_lb_n), .mem_ub_n(mem_ub_n), + .mem_rdata(mem_rdata), .mem_ready(mem_ready), + .flash_xfer_active(flash_xfer_active), .flash_byte_req(flash_byte_req), + .flash_byte_wdata(flash_byte_wdata), .flash_byte_rdata(flash_byte_rdata), + .flash_byte_done(flash_byte_done), + .soft_rst_pulse(soft_rst_pulse) + ); + + flash_spi_master u_flash ( + .clk(ui_clk), .rst(ui_clk_sync_rst), + .xfer_active(flash_xfer_active), .byte_req(flash_byte_req), + .byte_wdata(flash_byte_wdata), .byte_rdata(flash_byte_rdata), .byte_done(flash_byte_done), .busy(), + .flash_cs_n(flash_cs_n), .flash_mosi(flash_mosi), .flash_miso(flash_miso) + ); + + host_mem_bridge #( + .BURST_LEN(BURST_LEN), .ADDR_WIDTH(MEM_ADDR_WIDTH) + ) u_host_bridge ( + .clk(ui_clk), .rst(ui_clk_sync_rst), + .mem_req(mem_req), .mem_wr(mem_wr), .mem_addr(mem_addr), + .mem_wdata(mem_wdata), .mem_lb_n(mem_lb_n), .mem_ub_n(mem_ub_n), + .mem_rdata(mem_rdata), .mem_ready(mem_ready), + .req_active(req_active[HOST_SLOT]), .req_grant(req_grant[HOST_SLOT]), + .req_req(req_req[HOST_SLOT]), .req_wr(req_wr[HOST_SLOT]), + .req_addr(req_addr[HOST_SLOT*MEM_ADDR_WIDTH +: MEM_ADDR_WIDTH]), + .req_wdata(req_wdata[HOST_SLOT*32*BURST_LEN +: 32*BURST_LEN]), + .req_wmask(req_wmask[HOST_SLOT*4*BURST_LEN +: 4*BURST_LEN]), + .req_rdata(req_rdata[HOST_SLOT*32*BURST_LEN +: 32*BURST_LEN]), + .req_ready(req_ready[HOST_SLOT]), .req_busy(req_busy[HOST_SLOT]) + ); + + // ---- 2x systolic_group.v, each: 1 arbiter slot for its own + // weight-fetch (slots 0..N_GROUPS-1), 4 arbiter slots for its own + // 4 PEs' activation-fetch+writeback (slots N_GROUPS+4g .. + // N_GROUPS+4g+3) ---- + genvar gg; + generate + for (gg = 0; gg < N_GROUPS; gg = gg + 1) begin : GEN_GROUP + localparam PE_BASE = N_GROUPS + gg*4; + + systolic_group #( + .DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .ACC_WIDTH(ACC_WIDTH), + .BURST_LEN(BURST_LEN), .ADDR_WIDTH(JOB_ADDR_WIDTH), .LAYER_BYTES(LAYER_BYTES) + ) u_group ( + .clk(ui_clk), .rst(ui_clk_sync_rst), + .job_start(group_job_start[gg]), + .w_base(group_w_base[gg*JOB_ADDR_WIDTH +: JOB_ADDR_WIDTH]), + .n_tiles(group_n_tiles[gg*16 +: 16]), + .pe_x_base_a(group_pe_x_base_a[gg*4*JOB_ADDR_WIDTH +: 4*JOB_ADDR_WIDTH]), + .pe_x_base_b(group_pe_x_base_b[gg*4*JOB_ADDR_WIDTH +: 4*JOB_ADDR_WIDTH]), + .pe_result_addr_a(group_pe_result_addr_a[gg*4*JOB_ADDR_WIDTH +: 4*JOB_ADDR_WIDTH]), + .pe_result_addr_b(group_pe_result_addr_b[gg*4*JOB_ADDR_WIDTH +: 4*JOB_ADDR_WIDTH]), + .pe_node_id_a(group_pe_node_id_a[gg*4*16 +: 4*16]), + .pe_node_id_b(group_pe_node_id_b[gg*4*16 +: 4*16]), + .job_done(group_job_done[gg]), + .pe_result_data_a(), .pe_result_data_b(), + .pe_result_node_id_a(), .pe_result_node_id_b(), + .pe_result_addr_a_out(), .pe_result_addr_b_out(), + .mem_active(req_active[gg]), .mem_grant(req_grant[gg]), + .ctrl_req(req_req[gg]), .ctrl_wr(req_wr[gg]), + .ctrl_addr(req_addr[gg*MEM_ADDR_WIDTH +: MEM_ADDR_WIDTH]), + .ctrl_wdata(req_wdata[gg*32*BURST_LEN +: 32*BURST_LEN]), + .ctrl_wmask(req_wmask[gg*4*BURST_LEN +: 4*BURST_LEN]), + .ctrl_rdata(req_rdata[gg*32*BURST_LEN +: 32*BURST_LEN]), + .ctrl_ready(req_ready[gg]), .ctrl_busy(req_busy[gg]), + .pe_mem_active(req_active[PE_BASE +: 4]), .pe_mem_grant(req_grant[PE_BASE +: 4]), + .pe_ctrl_req(req_req[PE_BASE +: 4]), .pe_ctrl_wr(req_wr[PE_BASE +: 4]), + .pe_ctrl_addr(req_addr[PE_BASE*MEM_ADDR_WIDTH +: 4*MEM_ADDR_WIDTH]), + .pe_ctrl_wdata(req_wdata[PE_BASE*32*BURST_LEN +: 4*32*BURST_LEN]), + .pe_ctrl_wmask(req_wmask[PE_BASE*4*BURST_LEN +: 4*4*BURST_LEN]), + .pe_ctrl_rdata(req_rdata[PE_BASE*32*BURST_LEN +: 4*32*BURST_LEN]), + .pe_ctrl_ready(req_ready[PE_BASE +: 4]), .pe_ctrl_busy(req_busy[PE_BASE +: 4]) + ); + end + endgenerate +endmodule diff --git a/hardware/v3/sim/tb_n8_system_ddr3.v b/hardware/v3/sim/tb_n8_system_ddr3.v new file mode 100644 index 0000000..7a60d3a --- /dev/null +++ b/hardware/v3/sim/tb_n8_system_ddr3.v @@ -0,0 +1,521 @@ +`timescale 1ps/100fs + +// ============================================================ +// EXP-0096 -- real FUNCTIONAL verification of the N=8 hybrid systolic +// system (n8_system_ddr3_top.v), the real, definitive deployment +// target -- directly adapted from tb_n16_system_ddr3.v's own real +// methodology (EXP-0092/0094), scaled down to N_GROUPS=2 instead of 4. +// Exists specifically because EXP-0095's own real P&R closure at N=8 +// (WNS=0.000ns, 0 failing endpoints) was measured via a `-generic +// N_GROUPS=2` override against n16_system_ddr3_top.v, WITHOUT a +// dedicated functional test at that specific N -- this closes that +// real, disclosed gap before trusting N=8 as a real, deployable +// signoff. +// +// STRUCTURE: identical real harness to tb_n16_system_ddr3.v (real 2- +// chip 32-bit DDR3 model via WireDelay, real mig_7series_0_mig with +// SIM_BYPASS_INIT_CAL="FAST", the pre_active-muxed direct preload path, +// weight_byte/input_byte golden functions, sdram_write_burst task) -- +// only N_GROUPS/N_PES/M change (2 groups instead of 4, 16 positions +// instead of 32, filling every one of the 2 groups x 4 PEs x 2 lanes +// exactly once), plus neural_director_grouped.v + sdram_arbiter_hier.v +// + 2x systolic_group.v instead of 4x (host slot tied off inactive, +// same real precedent as tb_n2_system_ddr3.v never instantiating +// spi_host_bridge_v3.v). +// ============================================================ +module tb; + localparam CLKIN_PERIOD = 3225; // ps, this project's real, CLOSED MIG config (EXP-0086) + localparam REFCLK_FREQ = 200.0; // MHz + localparam real REFCLK_PERIOD = (1000000.0/(2*REFCLK_FREQ)); + localparam RESET_PERIOD = 200000; // ps + + localparam DATA_WIDTH = 8; + localparam P_IN = 8; + localparam ACC_WIDTH = 32; + localparam ADDR_WIDTH = 26; // this project's byte-address convention (Director/systolic_group) + localparam MIG_ADDR_WIDTH = 25; // word-address convention (BURST_LEN=8) at the arbiter/adapter + localparam BURST_LEN = 8; + localparam N_INPUTS = 128; + localparam N_TILES = N_INPUTS/P_IN; + localparam LAYER_BYTES = N_INPUTS; + localparam WORDS_PER_LAYER = LAYER_BYTES/2; + localparam N_GROUPS = 2; + localparam N_PES = N_GROUPS*4; // 8 + localparam QUEUE_DEPTH = 16; + + localparam L = 1; // one shared layer -- simplest real addressing that still + localparam M = 16; // exercises every one of the 2 groups x 4 PEs x 2 lanes exactly once + + // ---- clock/reset (mirrors tb_n2_system_ddr3.v's own proven pattern) ---- + reg sys_rst_n; + wire sys_rst = sys_rst_n; + reg sys_clk_i = 1'b0; + always #(CLKIN_PERIOD/2.0) sys_clk_i = ~sys_clk_i; + wire sys_clk_p = sys_clk_i; + wire sys_clk_n = ~sys_clk_i; + reg clk_ref_i = 1'b0; + always #REFCLK_PERIOD clk_ref_i = ~clk_ref_i; + wire clk_ref_p = clk_ref_i; + wire clk_ref_n = ~clk_ref_i; + initial begin + sys_rst_n = 1'b0; + #RESET_PERIOD sys_rst_n = 1'b1; + end + + // ---- real DDR3 pins + model (identical to tb_n2_system_ddr3.v) ---- + wire ddr3_reset_n; + wire [31:0] ddr3_dq_fpga; + wire [3:0] ddr3_dqs_p_fpga, ddr3_dqs_n_fpga; + wire [13:0] ddr3_addr_fpga; + wire [2:0] ddr3_ba_fpga; + wire ddr3_ras_n_fpga, ddr3_cas_n_fpga, ddr3_we_n_fpga; + wire [0:0] ddr3_cke_fpga, ddr3_ck_p_fpga, ddr3_ck_n_fpga, ddr3_cs_n_fpga; + wire [3:0] ddr3_dm_fpga; + wire [0:0] ddr3_odt_fpga; + + wire [31:0] ddr3_dq_sdram; + reg [13:0] ddr3_addr_sdram; + reg [2:0] ddr3_ba_sdram; + reg ddr3_ras_n_sdram, ddr3_cas_n_sdram, ddr3_we_n_sdram; + wire [0:0] ddr3_cs_n_sdram; + wire [0:0] ddr3_odt_sdram; + reg [0:0] ddr3_cke_sdram; + wire [3:0] ddr3_dm_sdram; + wire [3:0] ddr3_dqs_p_sdram, ddr3_dqs_n_sdram; + reg [0:0] ddr3_ck_p_sdram, ddr3_ck_n_sdram; + reg [0:0] ddr3_cs_n_sdram_tmp; + reg [3:0] ddr3_dm_sdram_tmp; + reg [0:0] ddr3_odt_sdram_tmp; + + always @(*) begin + ddr3_ck_p_sdram <= ddr3_ck_p_fpga; + ddr3_ck_n_sdram <= ddr3_ck_n_fpga; + ddr3_addr_sdram <= ddr3_addr_fpga; + ddr3_ba_sdram <= ddr3_ba_fpga; + ddr3_ras_n_sdram <= ddr3_ras_n_fpga; + ddr3_cas_n_sdram <= ddr3_cas_n_fpga; + ddr3_we_n_sdram <= ddr3_we_n_fpga; + ddr3_cke_sdram <= ddr3_cke_fpga; + end + always @(*) ddr3_cs_n_sdram_tmp <= ddr3_cs_n_fpga; + assign ddr3_cs_n_sdram = ddr3_cs_n_sdram_tmp; + always @(*) ddr3_dm_sdram_tmp <= ddr3_dm_fpga; + assign ddr3_dm_sdram = ddr3_dm_sdram_tmp; + always @(*) ddr3_odt_sdram_tmp <= ddr3_odt_fpga; + assign ddr3_odt_sdram = ddr3_odt_sdram_tmp; + + genvar dqwd; + generate + for (dqwd = 0; dqwd < 32; dqwd = dqwd + 1) begin : dq_delay + WireDelay #(.Delay_g(0.00), .Delay_rd(0.00), .ERR_INSERT("OFF")) u_delay_dq ( + .A(ddr3_dq_fpga[dqwd]), .B(ddr3_dq_sdram[dqwd]), + .reset(sys_rst_n), .phy_init_done(init_calib_complete) + ); + end + endgenerate + genvar dqswd; + generate + for (dqswd = 0; dqswd < 4; dqswd = dqswd + 1) begin : dqs_delay + WireDelay #(.Delay_g(0.00), .Delay_rd(0.00), .ERR_INSERT("OFF")) u_delay_dqs_p ( + .A(ddr3_dqs_p_fpga[dqswd]), .B(ddr3_dqs_p_sdram[dqswd]), + .reset(sys_rst_n), .phy_init_done(init_calib_complete) + ); + WireDelay #(.Delay_g(0.00), .Delay_rd(0.00), .ERR_INSERT("OFF")) u_delay_dqs_n ( + .A(ddr3_dqs_n_fpga[dqswd]), .B(ddr3_dqs_n_sdram[dqswd]), + .reset(sys_rst_n), .phy_init_done(init_calib_complete) + ); + end + endgenerate + + genvar ci; + generate + for (ci = 0; ci < 2; ci = ci + 1) begin : gen_mem + ddr3_model u_comp_ddr3 ( + .rst_n(ddr3_reset_n), .ck(ddr3_ck_p_sdram), .ck_n(ddr3_ck_n_sdram), + .cke(ddr3_cke_sdram[0]), .cs_n(ddr3_cs_n_sdram[0]), + .ras_n(ddr3_ras_n_sdram), .cas_n(ddr3_cas_n_sdram), .we_n(ddr3_we_n_sdram), + .dm_tdqs(ddr3_dm_sdram[2*ci +: 2]), .ba(ddr3_ba_sdram), .addr(ddr3_addr_sdram), + .dq(ddr3_dq_sdram[16*ci +: 16]), + .dqs(ddr3_dqs_p_sdram[2*ci +: 2]), .dqs_n(ddr3_dqs_n_sdram[2*ci +: 2]), + .tdqs_n(), .odt(ddr3_odt_sdram[0]) + ); + end + endgenerate + + wire [27:0] app_addr; + wire [2:0] app_cmd; + wire app_en, app_rdy; + wire [127:0] app_wdf_data; + wire app_wdf_end; + wire [15:0] app_wdf_mask; + wire app_wdf_wren, app_wdf_rdy; + wire [127:0] app_rd_data; + wire app_rd_data_end, app_rd_data_valid; + wire ui_clk, ui_clk_sync_rst, init_calib_complete; + + mig_7series_0_mig #( + .SIM_BYPASS_INIT_CAL("FAST") + ) u_mig ( + .ddr3_dq(ddr3_dq_fpga), .ddr3_dqs_n(ddr3_dqs_n_fpga), .ddr3_dqs_p(ddr3_dqs_p_fpga), + .ddr3_addr(ddr3_addr_fpga), .ddr3_ba(ddr3_ba_fpga), + .ddr3_ras_n(ddr3_ras_n_fpga), .ddr3_cas_n(ddr3_cas_n_fpga), .ddr3_we_n(ddr3_we_n_fpga), + .ddr3_reset_n(ddr3_reset_n), + .ddr3_ck_p(ddr3_ck_p_fpga), .ddr3_ck_n(ddr3_ck_n_fpga), + .ddr3_cke(ddr3_cke_fpga), .ddr3_cs_n(ddr3_cs_n_fpga), + .ddr3_dm(ddr3_dm_fpga), .ddr3_odt(ddr3_odt_fpga), + .sys_clk_p(sys_clk_p), .sys_clk_n(sys_clk_n), .clk_ref_p(clk_ref_p), .clk_ref_n(clk_ref_n), + .app_addr(app_addr), .app_cmd(app_cmd), .app_en(app_en), + .app_wdf_data(app_wdf_data), .app_wdf_end(app_wdf_end), + .app_wdf_mask(app_wdf_mask), .app_wdf_wren(app_wdf_wren), + .app_rd_data(app_rd_data), .app_rd_data_end(app_rd_data_end), + .app_rd_data_valid(app_rd_data_valid), .app_rdy(app_rdy), .app_wdf_rdy(app_wdf_rdy), + .app_sr_req(1'b0), .app_ref_req(1'b0), .app_zq_req(1'b0), + .app_sr_active(), .app_ref_ack(), .app_zq_ack(), + .ui_clk(ui_clk), .ui_clk_sync_rst(ui_clk_sync_rst), + .init_calib_complete(init_calib_complete), + .device_temp(), + .sys_rst(sys_rst) + ); + + // ---- preload path: direct access to mig_native_adapter.v, + // bypassing the arbiter, exactly like tb_n2_system_ddr3.v's own + // "pre_active" mux -- used only before job submission begins. ---- + reg pre_active; + reg pre_req, pre_wr; + reg [MIG_ADDR_WIDTH-1:0] pre_addr; + reg [32*BURST_LEN-1:0] pre_wdata; + + wire adp_req, adp_wr; + wire [MIG_ADDR_WIDTH-1:0] adp_addr; + wire [32*BURST_LEN-1:0] adp_wdata; + wire [4*BURST_LEN-1:0] adp_wmask; + wire [32*BURST_LEN-1:0] adp_rdata; + wire adp_ready, adp_busy; + + wire arb_ctrl_req_o, arb_ctrl_wr_o; + wire [MIG_ADDR_WIDTH-1:0] arb_ctrl_addr_o; + wire [32*BURST_LEN-1:0] arb_ctrl_wdata_o; + wire [4*BURST_LEN-1:0] arb_ctrl_wmask_o; + + assign adp_req = pre_active ? pre_req : arb_ctrl_req_o; + assign adp_wr = pre_active ? pre_wr : arb_ctrl_wr_o; + assign adp_addr = pre_active ? pre_addr : arb_ctrl_addr_o; + assign adp_wdata = pre_active ? pre_wdata : arb_ctrl_wdata_o; + assign adp_wmask = pre_active ? {(4*BURST_LEN){1'b0}} : arb_ctrl_wmask_o; + + mig_native_adapter #(.BURST_LEN(BURST_LEN), .ADDR_WIDTH(MIG_ADDR_WIDTH)) u_adapter ( + .clk(ui_clk), .rst(ui_clk_sync_rst), + .req(adp_req), .wr(adp_wr), .addr(adp_addr), .wdata(adp_wdata), .wmask(adp_wmask), + .rdata(adp_rdata), .ready(adp_ready), .busy(adp_busy), + .app_addr(app_addr), .app_cmd(app_cmd), .app_en(app_en), .app_rdy(app_rdy), + .app_wdf_data(app_wdf_data), .app_wdf_end(app_wdf_end), .app_wdf_mask(app_wdf_mask), + .app_wdf_wren(app_wdf_wren), .app_wdf_rdy(app_wdf_rdy), + .app_rd_data(app_rd_data), .app_rd_data_end(app_rd_data_end), .app_rd_data_valid(app_rd_data_valid) + ); + + // EXP-0086 fix (see tb_n2_system_ddr3.v): plain intermediate 8-bit + // reg instead of SV-only `8'(expr)` sized-cast syntax. + function automatic signed [7:0] weight_byte(input integer li, input integer t); + reg [7:0] tmp; + begin + tmp = (li*17 + t*29 + 13) & 8'hFF; + weight_byte = $signed(tmp); + end + endfunction + function automatic signed [7:0] input_byte(input integer li, input integer pos, input integer t); + reg [7:0] tmp; + begin + tmp = (li*11 + pos*41 + t*7 + 3) & 8'hFF; + input_byte = $signed(tmp); + end + endfunction + + task automatic sdram_write_burst(input [MIG_ADDR_WIDTH-1:0] word_addr, input [32*BURST_LEN-1:0] data); + begin + @(posedge ui_clk); while (adp_busy) @(posedge ui_clk); + pre_req = 1'b1; pre_wr = 1'b1; pre_addr = word_addr; pre_wdata = data; + @(posedge ui_clk); pre_req = 1'b0; + while (!adp_ready) @(posedge ui_clk); + end + endtask + + task automatic preload_sdram_layers; + integer li, bi, wb, tt; + reg [32*BURST_LEN-1:0] burst_data; + begin + for (li = 0; li < L; li = li + 1) begin + for (bi = 0; bi < (LAYER_BYTES/(4*BURST_LEN)); bi = bi + 1) begin + for (wb = 0; wb < BURST_LEN; wb = wb + 1) begin + tt = bi*(4*BURST_LEN) + wb*4; + burst_data[wb*32 +: 32] = {weight_byte(li, tt+3), weight_byte(li, tt+2), + weight_byte(li, tt+1), weight_byte(li, tt)}; + end + sdram_write_burst((li*WORDS_PER_LAYER + bi*BURST_LEN), burst_data); + end + end + end + endtask + + localparam [MIG_ADDR_WIDTH-1:0] ACT_MEM_BASE = 25'h10000; + function automatic [ADDR_WIDTH-1:0] act_x_base(input integer li, input integer pos); + act_x_base = {{(ADDR_WIDTH-MIG_ADDR_WIDTH){1'b0}}, ACT_MEM_BASE} + (li*M + pos) * ((N_TILES/4)*BURST_LEN); + endfunction + + task automatic preload_ddr3_activations; + integer li, pos, tq, qi; + reg [32*BURST_LEN-1:0] burst_data; + reg [ADDR_WIDTH-1:0] base; + begin + for (li = 0; li < L; li = li + 1) begin + for (pos = 0; pos < M; pos = pos + 1) begin + base = act_x_base(li, pos); + for (tq = 0; tq < N_TILES/4; tq = tq + 1) begin + burst_data = {(32*BURST_LEN){1'b0}}; + for (qi = 0; qi < 4; qi = qi + 1) + burst_data[qi*64 +: 64] = {input_byte(li, pos, (4*tq+qi)*P_IN + 7), input_byte(li, pos, (4*tq+qi)*P_IN + 6), + input_byte(li, pos, (4*tq+qi)*P_IN + 5), input_byte(li, pos, (4*tq+qi)*P_IN + 4), + input_byte(li, pos, (4*tq+qi)*P_IN + 3), input_byte(li, pos, (4*tq+qi)*P_IN + 2), + input_byte(li, pos, (4*tq+qi)*P_IN + 1), input_byte(li, pos, (4*tq+qi)*P_IN + 0)}; + sdram_write_burst(base[MIG_ADDR_WIDTH-1:0] + tq*BURST_LEN, burst_data); + end + end + end + end + endtask + + // ---- neural_director_grouped.v ---- + reg job_in_valid; + wire job_in_ready; + reg [ADDR_WIDTH-1:0] job_in_x_base, job_in_w_base, job_in_result_addr; + reg [15:0] job_in_n_tiles, job_in_node_id; + + wire [N_GROUPS-1:0] group_job_start; + wire [ADDR_WIDTH*N_GROUPS-1:0] group_w_base; + wire [16*N_GROUPS-1:0] group_n_tiles; + wire [4*ADDR_WIDTH*N_GROUPS-1:0] group_pe_x_base_a, group_pe_x_base_b; + wire [4*ADDR_WIDTH*N_GROUPS-1:0] group_pe_result_addr_a, group_pe_result_addr_b; + wire [4*16*N_GROUPS-1:0] group_pe_node_id_a, group_pe_node_id_b; + wire [N_GROUPS-1:0] group_job_done; + wire job_out_done; + wire [$clog2(N_GROUPS)-1:0] job_out_group; + wire [3:0] dir_state; + wire dir_error; + wire queue_empty; + + neural_director_grouped #( + .ADDR_WIDTH(ADDR_WIDTH), .N_GROUPS(N_GROUPS), .QUEUE_DEPTH(QUEUE_DEPTH) + ) u_dir ( + .clk(ui_clk), .rst(ui_clk_sync_rst), + .job_in_valid(job_in_valid), .job_in_ready(job_in_ready), + .job_in_x_base(job_in_x_base), .job_in_w_base(job_in_w_base), + .job_in_n_tiles(job_in_n_tiles), .job_in_result_addr(job_in_result_addr), + .job_in_node_id(job_in_node_id), + .group_job_start(group_job_start), .group_w_base(group_w_base), .group_n_tiles(group_n_tiles), + .group_pe_x_base_a(group_pe_x_base_a), .group_pe_x_base_b(group_pe_x_base_b), + .group_pe_result_addr_a(group_pe_result_addr_a), .group_pe_result_addr_b(group_pe_result_addr_b), + .group_pe_node_id_a(group_pe_node_id_a), .group_pe_node_id_b(group_pe_node_id_b), + .group_job_done(group_job_done), + .job_out_done(job_out_done), .job_out_group(job_out_group), + .dir_state(dir_state), .dir_error(dir_error), .queue_empty(queue_empty) + ); + + // ---- real, hierarchical 2-level arbiter (EXP-0094): 2 groups' own + // weight-fetch + 8 PEs' own activation-fetch+writeback + 1 host + // slot, matching n8_system_ddr3_top.v's own real, fixed topology + // exactly -- sdram_arbiter_hier.v always includes a host slot, so + // this testbench ties it off inactive (doesn't instantiate + // host_mem_bridge.v at all, same real precedent as + // tb_n2_system_ddr3.v never instantiating spi_host_bridge_v3.v). ---- + localparam NUM_REQ = N_GROUPS + N_PES + 1; // 2 + 8 + 1 = 11 + localparam HOST_SLOT = NUM_REQ - 1; // 10, tied off inactive below + + wire [NUM_REQ-1:0] req_active, req_grant, req_req, req_wr; + wire [NUM_REQ-1:0] req_ready, req_busy; + wire [NUM_REQ*MIG_ADDR_WIDTH-1:0] req_addr; + wire [NUM_REQ*32*BURST_LEN-1:0] req_wdata; + wire [NUM_REQ*4*BURST_LEN-1:0] req_wmask; + wire [NUM_REQ*32*BURST_LEN-1:0] req_rdata; + + assign req_active[HOST_SLOT] = 1'b0; + assign req_req[HOST_SLOT] = 1'b0; + assign req_wr[HOST_SLOT] = 1'b0; + assign req_addr[HOST_SLOT*MIG_ADDR_WIDTH +: MIG_ADDR_WIDTH] = {MIG_ADDR_WIDTH{1'b0}}; + assign req_wdata[HOST_SLOT*32*BURST_LEN +: 32*BURST_LEN] = {(32*BURST_LEN){1'b0}}; + assign req_wmask[HOST_SLOT*4*BURST_LEN +: 4*BURST_LEN] = {(4*BURST_LEN){1'b0}}; + + sdram_arbiter_hier #( + .N_GROUPS(N_GROUPS), .PES_PER_GROUP(4), .ADDR_WIDTH(MIG_ADDR_WIDTH), .BURST_LEN(BURST_LEN) + ) u_arb ( + .clk(ui_clk), .rst(ui_clk_sync_rst), + .req_active(req_active), .req_grant(req_grant), + .req_req(req_req), .req_wr(req_wr), .req_addr(req_addr), + .req_wdata(req_wdata), .req_wmask(req_wmask), + .req_rdata(req_rdata), .req_ready(req_ready), .req_busy(req_busy), + .ctrl_req(arb_ctrl_req_o), .ctrl_wr(arb_ctrl_wr_o), .ctrl_addr(arb_ctrl_addr_o), + .ctrl_wdata(arb_ctrl_wdata_o), .ctrl_wmask(arb_ctrl_wmask_o), + .ctrl_rdata(adp_rdata), .ctrl_ready(adp_ready), .ctrl_busy(adp_busy) + ); + + wire [4*DATA_WIDTH*N_GROUPS-1:0] all_pe_result_data_a, all_pe_result_data_b; + wire [4*16*N_GROUPS-1:0] all_pe_result_node_id_a, all_pe_result_node_id_b; + + genvar gg; + generate + for (gg = 0; gg < N_GROUPS; gg = gg + 1) begin : GEN_GROUP + localparam PE_BASE = N_GROUPS + gg*4; + + systolic_group #( + .DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .ACC_WIDTH(ACC_WIDTH), + .BURST_LEN(BURST_LEN), .ADDR_WIDTH(ADDR_WIDTH), .LAYER_BYTES(LAYER_BYTES) + ) u_group ( + .clk(ui_clk), .rst(ui_clk_sync_rst), + .job_start(group_job_start[gg]), + .w_base(group_w_base[gg*ADDR_WIDTH +: ADDR_WIDTH]), + .n_tiles(group_n_tiles[gg*16 +: 16]), + .pe_x_base_a(group_pe_x_base_a[gg*4*ADDR_WIDTH +: 4*ADDR_WIDTH]), + .pe_x_base_b(group_pe_x_base_b[gg*4*ADDR_WIDTH +: 4*ADDR_WIDTH]), + .pe_result_addr_a(group_pe_result_addr_a[gg*4*ADDR_WIDTH +: 4*ADDR_WIDTH]), + .pe_result_addr_b(group_pe_result_addr_b[gg*4*ADDR_WIDTH +: 4*ADDR_WIDTH]), + .pe_node_id_a(group_pe_node_id_a[gg*4*16 +: 4*16]), + .pe_node_id_b(group_pe_node_id_b[gg*4*16 +: 4*16]), + .job_done(group_job_done[gg]), + .pe_result_data_a(all_pe_result_data_a[gg*4*DATA_WIDTH +: 4*DATA_WIDTH]), + .pe_result_data_b(all_pe_result_data_b[gg*4*DATA_WIDTH +: 4*DATA_WIDTH]), + .pe_result_node_id_a(all_pe_result_node_id_a[gg*4*16 +: 4*16]), + .pe_result_node_id_b(all_pe_result_node_id_b[gg*4*16 +: 4*16]), + .pe_result_addr_a_out(), .pe_result_addr_b_out(), + .mem_active(req_active[gg]), .mem_grant(req_grant[gg]), + .ctrl_req(req_req[gg]), .ctrl_wr(req_wr[gg]), + .ctrl_addr(req_addr[gg*MIG_ADDR_WIDTH +: MIG_ADDR_WIDTH]), + .ctrl_wdata(req_wdata[gg*32*BURST_LEN +: 32*BURST_LEN]), + .ctrl_wmask(req_wmask[gg*4*BURST_LEN +: 4*BURST_LEN]), + .ctrl_rdata(req_rdata[gg*32*BURST_LEN +: 32*BURST_LEN]), + .ctrl_ready(req_ready[gg]), .ctrl_busy(req_busy[gg]), + .pe_mem_active(req_active[PE_BASE +: 4]), .pe_mem_grant(req_grant[PE_BASE +: 4]), + .pe_ctrl_req(req_req[PE_BASE +: 4]), .pe_ctrl_wr(req_wr[PE_BASE +: 4]), + .pe_ctrl_addr(req_addr[PE_BASE*MIG_ADDR_WIDTH +: 4*MIG_ADDR_WIDTH]), + .pe_ctrl_wdata(req_wdata[PE_BASE*32*BURST_LEN +: 4*32*BURST_LEN]), + .pe_ctrl_wmask(req_wmask[PE_BASE*4*BURST_LEN +: 4*4*BURST_LEN]), + .pe_ctrl_rdata(req_rdata[PE_BASE*32*BURST_LEN +: 4*32*BURST_LEN]), + .pe_ctrl_ready(req_ready[PE_BASE +: 4]), .pe_ctrl_busy(req_busy[PE_BASE +: 4]) + ); + end + endgenerate + + integer errors, tests, completions, n_expected; + reg [15:0] expect_node [0:15]; + reg signed [7:0] expect_val [0:15]; + + function automatic signed [7:0] golden_result(input integer li, input integer pos); + integer t, acc; + reg signed [7:0] r; + begin + acc = 0; + for (t = 0; t < N_INPUTS; t = t + 1) + acc = acc + (input_byte(li, pos, t) * weight_byte(li, t)); + if (acc <= 0) r = 0; else if (acc > 127) r = 8'sd127; else r = acc[7:0]; + golden_result = r; + end + endfunction + + task automatic check_completion(input integer idx, input [15:0] nid, input signed [7:0] val); + integer ei, found; + begin + found = 0; + for (ei = 0; ei < n_expected; ei = ei + 1) begin + if (expect_node[ei] === nid && !found) begin + found = 1; + tests = tests + 1; + if (expect_val[ei] !== val) begin + $display("FAIL idx=%0d node_id=%0d: got=%0d expected=%0d", idx, nid, $signed(val), $signed(expect_val[ei])); + errors = errors + 1; + end else begin + $display("PASS idx=%0d node_id=%0d: result=%0d", idx, nid, $signed(val)); + end + end + end + end + endtask + + integer gg2, pp2; + always @(posedge ui_clk) begin + if (!ui_clk_sync_rst) begin + for (gg2 = 0; gg2 < N_GROUPS; gg2 = gg2 + 1) begin + if (group_job_done[gg2]) begin + completions = completions + 8; + for (pp2 = 0; pp2 < 4; pp2 = pp2 + 1) begin + check_completion(gg2*4+pp2, + all_pe_result_node_id_a[(gg2*4+pp2)*16 +: 16], + all_pe_result_data_a[(gg2*4+pp2)*DATA_WIDTH +: DATA_WIDTH]); + check_completion(gg2*4+pp2, + all_pe_result_node_id_b[(gg2*4+pp2)*16 +: 16], + all_pe_result_data_b[(gg2*4+pp2)*DATA_WIDTH +: DATA_WIDTH]); + end + end + end + end + end + + // real, root-caused fix (EXP-0090, CLAUDE.md): driving stimulus on + // @(posedge clk) races the DUT's own posedge-sampling always block + // when called back-to-back with zero real simulated gap -- drive on + // @(negedge ui_clk) instead. + task automatic submit_job( + input [ADDR_WIDTH-1:0] xb, input [ADDR_WIDTH-1:0] wb, + input [15:0] nt, input [ADDR_WIDTH-1:0] resaddr, input [15:0] nid + ); + begin + @(negedge ui_clk); + job_in_valid = 1'b1; job_in_x_base = xb; job_in_w_base = wb; + job_in_n_tiles = nt; job_in_result_addr = resaddr; job_in_node_id = nid; + @(negedge ui_clk); + job_in_valid = 1'b0; + end + endtask + + integer pp_i, wd; + + initial begin + errors = 0; tests = 0; completions = 0; n_expected = 0; + pre_active = 1'b1; pre_req = 0; pre_wr = 0; pre_addr = 0; pre_wdata = 0; + job_in_valid = 0; job_in_x_base = 0; job_in_w_base = 0; + job_in_n_tiles = 0; job_in_result_addr = 0; job_in_node_id = 0; + + $display("=== waiting for real DDR3 init_calib_complete ==="); + wait (init_calib_complete); + $display("=== calibration done at time %0t ===", $time); + repeat (10) @(posedge ui_clk); + + $display("=== preload SDRAM with %0d resident-filter weight set(s) ===", L); + preload_sdram_layers; + $display("=== preload SDRAM with real activation data (%0d positions) ===", M); + preload_ddr3_activations; + @(posedge ui_clk); + pre_active = 1'b0; + repeat (5) @(posedge ui_clk); + + $display("=== N=8 hybrid systolic system on REAL DDR3: submitting %0d positions (2 groups x 4 PEs x 2 lanes) ===", M); + for (pp_i = 0; pp_i < M; pp_i = pp_i + 1) begin + submit_job(act_x_base(0, pp_i), {ADDR_WIDTH{1'b0}}, N_TILES[15:0], + 26'h9000 + pp_i, pp_i[15:0]); + expect_node[n_expected] = pp_i[15:0]; + expect_val[n_expected] = golden_result(0, pp_i); + n_expected = n_expected + 1; + end + + wd = 0; + while (completions < n_expected && wd < 400000) begin + @(posedge ui_clk); + wd = wd + 1; + end + + if (completions < n_expected) begin + $display("FAIL: only %0d/%0d position-results completed within watchdog", completions, n_expected); + errors = errors + 1; + end + + $display("=== %0d/%0d tests, %0d errors, %0d/%0d positions completed ===", tests-errors, tests, errors, completions, n_expected); + if (errors == 0 && completions == n_expected) $display("ALL TESTS PASSED (tb_n8_system_ddr3, REAL DDR3)"); + $finish; + end +endmodule