michele/FPGA-Neural
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

feat: real hierarchical 2-level arbiter, real N=16 timing WNS -0.913ns -> -0.646ns (EXP-0094)

Browse Source 
sdram_arbiter_hier.v: fixes EXP-0093's own real, traced P&R timing
failure (flat 21-way req_wdata mux, route-delay-dominated). Reuses
sdram_arbiter_n.v unmodified, twice: 4 leaf instances (NUM_REQ=5, one
per group) + 1 top instance (NUM_REQ=5: 4 groups + host, host
bypassed/unpipelined), one real pipeline register stage between
levels. Isolated verification (tb_sdram_arbiter_hier.v): 23/23 PASS.

Two real bugs found and fixed via signal tracing: a testbench helper
not waiting for grant before firing req, and a genuine RTL lost-pulse
bug at the leaf-to-top boundary (a transient one-shot request could be
dropped if the top level was busy with a different group) -- fixed
with a sticky per-group pending_req_r latch.

Wired into n16_system_ddr3_top.v (drop-in). Real, full P&R re-run:
WNS improved -0.913ns -> -0.646ns, TNS -690ns -> -97.5ns, failing
endpoints 3021 -> 771 -- substantial, measured improvement, confirming
the arbiter was correctly root-caused (bottleneck moved elsewhere:
neural_processor_packed.v's own already-thin-margin MAC datapath,
eroded by N=16's higher overall congestion). Functional xsim still
32/32 PASS. Timing not yet fully closed -- real next steps documented,
not yet attempted without further direction.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUG92aM9m68TRc4rG55BcC
This commit is contained in:
michele
2026-09-21 02:25:44 +02:00
co-authored by Claude Sonnet 5
parent 50c940a41d
commit 9e1f16db47
7 changed files with 840 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hardware/v2/logs/experiments.log
+190
View File
@@ -6397,3 +6397,193 @@ N=16 variant specifically (a real, measured tradeoff against N=2's own
real throughput, not yet quantified). Real signal-tracing already done
above narrows the fix to the arbiter's own write-data mux specifically
-- future work should start there, not guess elsewhere in the design.
EXP-0094 -- real, hierarchical 2-level arbiter (sdram_arbiter_hier.v):
real fix for EXP-0093's own real timing failure, isolated verification
23/23 PASS (2026-09-21, user's own explicit direction: "sarei tentato
per un arbiter a due livelli... secondo te funziona meglio?", confirmed
"ok ora cerca di terminare il lavoro")
CONTEXT: EXP-0093's own real, traced root cause: sdram_arbiter_n.v's
own flat req_wdata/req_addr mux grew from 3-way (N=2) to real 20/21-way
(N=16) over a 256-bit bus, dominated by ROUTE delay (73% of the real
critical path) -- a real PHYSICAL fan-in/placement problem (20
separately-placed sources converging on one central mux), not primarily
a logic-depth one.
METHOD: `sdram_arbiter_hier.v` (NEW), reusing `sdram_arbiter_n.v`
UNMODIFIED, twice, hierarchically -- 4 real LEAF instances (NUM_REQ=5:
1 weight-fetch + 4 PEs per group, physically local to their own real
systolic_group.v) + 1 real TOP instance (NUM_REQ=5: 4 groups' own
pipelined output + 1 host_mem_bridge.v, deliberately UNPIPELINED/
bypassed since it was never the reported critical path -- see the
file's own header for the full real design rationale). ONE real
pipeline register stage, both directions, between the two levels --
the actual real fix for the route-delay-dominated critical path, not
just a logic restructuring. Fork-before-promote: `sdram_arbiter_n.v`
itself untouched, N=2's own real EXP-0088 signoff unaffected.
REAL CORRECTNESS ARGUMENT (derived, not asserted): every real requester
in this project (`act_tile_fetch.v`, `layer_prefetch_ctrl.v`,
`host_mem_bridge.v`) already keeps its own `mem_active` asserted for
the FULL duration of its own outstanding transaction -- the same real
invariant `sdram_arbiter_n.v`'s own `locked` state already relies on
for transactions spanning many real DDR3-latency cycles today. The
added real pipeline latency (GROUP-sourced traffic only; host bypasses
it entirely) is indistinguishable, from any requester's own point of
view, from "DDR3 was slightly slower this time" -- confirmed, not
assumed, via real isolated verification below. EXP-0066's own real
"own grant same cycle as own active" requirement is preserved EXACTLY
for all 21 real requesters (weight-fetch/PE via the leaf's own
unmodified combinational grant; host via the top's own direct
combinational grant) -- only the underlying ctrl_req/wdata/etc
reaching mig_native_adapter.v is pipelined, never the grant signal a
real requester actually polls.
REAL BUGS FOUND AND FIXED VIA SIGNAL TRACING BEFORE A TRUSTWORTHY
RESULT WAS POSSIBLE (per this project's own "root-cause every anomaly,
never guess" discipline):
1. New isolated testbench `tb_sdram_arbiter_hier.v` (21-way, N_GROUPS=4/
PES_PER_GROUP=4/+1 host, real `burst_mem_model32.v` mock controller,
same real methodology as `tb_sdram_arbiter_n.v`) had NO real
watchdog -- a real protocol bug spun the real Icarus sim forever
(99% CPU, zero output) instead of failing cleanly. Added a real
500000ns watchdog (a NEW, generalizable gap: every OTHER testbench
in this project already has a `wd`-counted watchdog inside its own
completion-wait loop; this one, copied from a simpler precedent,
didn't). Also hit, mid-debugging, a real false alarm from the
user's own accidental Ctrl-C of a *different*, harmless foreground
`vvp` run -- confirmed via `git status`/`ps -ef` that nothing was
actually lost before continuing.
2. Testbench bug (NOT RTL): `one_shot_txn`'s own original version,
copied verbatim from `tb_sdram_arbiter_n.v`, fired `req_req` the
SAME cycle as `req_active`, unconditionally, without first
checking `req_grant` -- real, traced failure mode: this module's
own real extra lock-release lag (1-2 cycles, after a PRIOR
transaction on a DIFFERENT slot completes) can race a same-cycle
blind fire, losing the pulse. Confirmed, via `act_tile_fetch.v`'s
own real S_MEMWAIT state ("ctrl_req is only issued after mem_grant
is observed, never blind"), that REAL requesters in this project
already wait for grant before firing req -- fixed the testbench
helper to match real behavior (and this file's own already-correct
`concurrent_contention` task), not the RTL.
3. REAL RTL BUG (the actual fix this experiment exists to deliver):
`leaf_ctrl_req` is a TRANSIENT one-shot pulse (mirrors a real
requester's own one-cycle `ctrl_req`) -- if the TOP level is still
busy servicing a DIFFERENT group at the exact cycle it fires, a
bare "register leaf_ctrl_req every cycle" pipeline (the first,
broken version of this file) lets the pulse revert to 0 and be
silently lost before TOP ever gets to it -- the EXACT EXP-0066
lost-pulse class, newly exposed at the leaf-to-top boundary
because a leaf's own LOCAL grant does not guarantee the top level
is free to act on it the same cycle (unlike the flat, single-level
arbiter, where the winning requester's own grant and the physical
controller's own readiness to capture it are the SAME decision).
Root-caused via real $monitor signal tracing of TEST 3 (cross-
group contention: all 4 groups' own weight-fetch simultaneously) --
traced `u_top.locked`/`grant_idx_r` staying locked onto group 0
for ~125000fs (~8 real cycles, DDR3-model latency) while groups
1-3's own already-captured `top_req_req_r` bits had already
reverted to 0. Fixed with a real, sticky per-group `pending_req_r`
latch: SET the cycle `leaf_ctrl_req` first pulses, CLEARED only
once `top_req_grant && top_req_req` both confirm real dispatch for
that group -- `(pending_req_r | leaf_ctrl_req)` feeds the outgoing
pipeline register instead of the bare transient signal. addr/wr/
wdata/wmask do NOT need the same treatment (the leaf stays locked
onto the SAME real requester for its whole transaction, so those
fields are already stable throughout).
REAL RESULT (isolated, `tb_sdram_arbiter_hier.v`, real Icarus xsim):
**23/23 PASS, 0 errors** -- sequential single-requester across weight-
fetch/PE/host slots in all 4 groups, WITHIN-group contention (leaf-
level, group 1's weight-fetch + all 4 PEs simultaneously), CROSS-group
contention (top-level, all 4 groups' own weight-fetch simultaneously),
and full worst-case mixed contention (one PE from each of the 4 groups
+ host, all simultaneously) -- every real response verified bit-exact
back to its own correct requester, no misrouting through either
arbiter level.
Wired into `n16_system_ddr3_top.v` (drop-in replacement, same real
external port shape -- only the module name/parameters at the
instantiation site changed) and into `tb_n16_system_ddr3.v` (host slot
tied off inactive, matching the real topology exactly).
DECISION: the real, isolated arbiter fix is verified correct. Re-
running EXP-0092's own real functional DDR3 xsim (whole-system,
`tb_n16_system_ddr3.v`) next, to confirm the fix holds wired into the
full system before re-attempting real P&R.
next_action: (1) real functional xsim of the whole N=16 system with
the new hierarchical arbiter (re-run of EXP-0092's own test). (2) if
that passes, real, full P&R (place_design+route_design) to check
whether WNS actually closes now.
EXP-0094 (continued) -- real functional re-verification + real, full
P&R with the hierarchical arbiter wired in: SUBSTANTIAL real
improvement, root cause demonstrably shifted, timing STILL not fully
closed (2026-09-21)
METHOD: (1) real functional xsim, `tb_n16_system_ddr3.v` updated to
instantiate `sdram_arbiter_hier.v` (host slot tied off inactive,
matching the real topology) instead of the flat `sdram_arbiter_n.v`,
same real DDR3-model methodology as EXP-0092. (2) real, full P&R
(synth_design+opt_design+place_design+route_design) against the real
XC7A100T-CSG324-2 part, `n16_system_ddr3_top.v` updated to instantiate
`sdram_arbiter_hier.v` in place of the flat arbiter (drop-in, same
external port shape). `sdram_arbiter_hier.v` added to the real Vivado
project's own `sources_1` fileset as a direct reference (confirmed via
a real stale-import check before trusting the result: empty, no
`*/imports/*` copies of anything in the real dependency chain).
REAL RESULT (1), functional: **32/32 PASS, 0 errors**, `$finish` at
197735.6335ns -- the hierarchical arbiter fix does not break real
functional correctness at full N=16 scale.
REAL RESULT (2), P&R: real post-route utilization essentially
unchanged from EXP-0093 (19902 LUTs/31.39%, 35395 regs/27.91%, 128
DSP48E1/53.33%). **Real timing: WNS=-0.646ns (up from EXP-0093's real
-0.913ns), TNS=-97.541ns (up from -690.085ns, ~7x fewer), 771 failing
setup endpoints (down from 3021, ~4x fewer)** -- a real, substantial,
measured improvement in the RIGHT direction, but timing constraints
are STILL NOT MET.
REAL, HONEST NEW FINDING (traced via the actual new worst violated
path, not guessed): the bottleneck has DEMONSTRABLY SHIFTED away from
the arbiter entirely. The new worst path runs from `neural_processor_
packed.v`'s own `GEN_MAC_PACKED[4].product` (a real DSP48E1 MAC output,
inside group 3 / PE 2's own compute core) to its own `prodb1_reg[4][13]`
-- a path dominated by LOGIC delay (79%, CARRY4=4/LUT2=1), NOT route,
unlike EXP-0093's own arbiter-driven, route-dominated (73%) violation.
This is a PRE-EXISTING module, byte-for-byte unchanged since N=2 (EXP-
0088), where it closed with an already razor-thin real margin
(WNS=+0.099962ns). Real, coherent interpretation (not asserted without
the evidence above): N=16's real overall die utilization (31% LUT vs
N=2's much lower level) increases general placement/routing congestion
enough, on its own, to erode that already-thin pre-existing margin --
a DIFFERENT, more diffuse, congestion-driven problem than the
arbiter's own single, structural fan-in bottleneck, without an
equally obvious single-point fix.
DECISION: the hierarchical arbiter (EXP-0094) is a real, verified,
substantial improvement, kept and committed regardless of whether full
N=16 timing closure is reached -- it fixed the exact problem it was
built for (confirmed by the bottleneck moving elsewhere entirely).
Full N=16 timing closure needs a SEPARATE, further real intervention
(likely touching `neural_processor_packed.v` itself, the compute core
shared by every real PE in this project, or accepting a lower target
clock for the N=16 variant specifically) -- a real engineering decision
better made with the user's own direction than guessed at further
unprompted, given `neural_processor_packed.v`'s own central, shared
role across the whole project.
next_action: real options for closing the REMAINING gap, not yet
attempted, to discuss before committing more RTL effort: (1) real
pipelining inside `neural_processor_packed.v`'s own MAC datapath at the
specific `GEN_MAC_PACKED`/`prodb1_reg` boundary (fork-before-promote,
same discipline as the arbiter fix -- N=2's own real signoff must stay
protected). (2) a real, measured lower target clock for the N=16
variant specifically (quantify the real throughput trade-off against
N=2, not yet done). (3) real Vivado placement/timing directives
(e.g. `-directive` on `place_design`/`route_design`, or a `PBLOCK`
constraint per group to reduce congestion) as a lower-RTL-risk first
attempt before touching `neural_processor_packed.v` itself.