FPGA-Neural/hardware/v2/reports/step17_n4_timing_throughput.md

STEP17 — SDRAM N=4 Timing Closure & Throughput Decomposition

Governing spec: the user's STEP17 message, in full. Builds directly on STEP16 (DEC-0031, ADOPT SDRAM), which remains the closed, unreopened V2 memory decision. All new evidence below is classified by type per the spec's own discipline (THEORETICAL / RTL SIMULATION / IDEAL MEMORY / POST-SYNTHESIS / POST-P&R / REAL SDRAM SIMULATION / INTEGRATED BENCHMARK / DERIVED). Raw data: step17_timing_seeds.csv, step17_cycle_decomposition.csv, step17_sdram_effectiveness.csv. Full narrative: experiments.log (EXP-0044/0045), errors.log (ERR-0021), decisions.log (DEC-0032).

Executive Summary

Part A (timing): the N=4 SDRAM design's 81.55MHz is NOT a defect specific to SDRAM integration. Re-synthesizing the dual-PSRAM baseline with the identical toolchain/flags reveals it ALSO only reaches 74.88-87.26MHz at N=4 (best 87.26MHz) — the originally-reported 110.28MHz does not reproduce under this toolchain/methodology. The real, apples-to-apples gap between the two architectures at N=4 is small (~7%, 81.55 vs 87.26MHz), not the ~26% the STEP16 report implied. Both architectures show a LARGE N=2→N=4 Fmax drop (SDRAM: 103.99→81.55, -21.6%; dual-PSRAM: 103.52→87.26, -15.7%), and the critical path traces to dependency_manager.v's own N_NODES-wide priority-encoder scan (first_ready_idx) — a module completely unchanged between the two architectures. Conclusion: the Fmax ceiling at N=4 is primarily an N-scaling effect of the shared architecture, with a smaller secondary amplification from SDRAM's added logic increasing placement congestion. A candidate minimal fix (pipelining first_ready_idx) was implemented, found to introduce a genuine bit-exact correctness regression (a double-dispatch race), and was reverted rather than risk shipping unproven behavior in a heavily-reused module — per the spec's own "preserve the working architecture" rule. N=4 already meets the hard minimum (Fmax ≥ 80MHz, best-of-3-seeds = 81.55MHz PASS) without any RTL change.

Part B/C (decomposition): of 49,430 D-Stress cycles at N=4, real processors are actively delivering a tile only ~2.06% of the slot-cycle budget; ~90.91% is spent specifically blocked waiting for weight data. The real SDRAM controller is busy 99.92% of the run — essentially saturated — with per-transaction latency (10.06 cycles avg) sitting right at its fixed isolated minimum (10 cycles). The system is MEMORY-BANDWIDTH-BOUND, not latency-bound and not primarily arbitration-bound.

1. Exact critical path (Part A)

Real post-P&R critical path (nextpnr-ecp5 0.11.1, N=4 SDRAM, seed 3, 81.55MHz, POST-P&R):

Field	Value
Source register	u_dataflow_core.u_dep_mgr.node_state[15] (TRELLIS_FF Q)
Destination register	u_dataflow_core.u_dep_mgr.ready_result_addr (via node_result_addr[0].1$DPRAM_COMB0)
RTL source location	hardware/v2/rtl/dependency_manager.v:100-112 (first_ready_idx priority-encoder scan) and :183-187 (dispatch capture, array-indexed reads)
Logic elements	1 clk-to-q + ~9 cascaded LUT4/PFUMX stages (priority-encoder chain) + 1 DPRAM_COMB read
Logic delay	3.01 ns (25%)
Routing delay	9.26 ns (75%), including one 3.10ns single-hop jump (tile (41,20)→(8,9))
Total path delay	12.26 ns
Estimated Fmax (this path alone)	1/12.26ns = 81.6 MHz (matches reported 81.55MHz)
Classification	primarily ROUTING, arbitration-adjacent (priority-encoder), memory-controller NOT involved

The scan's own output (first_ready_idx) fans out into FOUR wide array reads (node_x_base/node_w_base/node_n_tiles/ node_result_addr) in the same cycle it is computed, forcing the placer to keep the whole scan-to-array-read chain physically close — this is why routing (not logic) dominates.

2. N2 vs N4 comparison (Part A.2)

POST-P&R, identical toolchain, --lpf-allow-unconstrained, 3 seeds each (step17_timing_seeds.csv):

Config	Seed1	Seed2	Seed3	Best
SDRAM N=2	102.29	94.32	103.99	103.99 MHz
SDRAM N=4	80.15	74.88 (FAIL)	81.55	81.55 MHz
Dual-PSRAM N=2	100.48	103.52	101.37	103.52 MHz
Dual-PSRAM N=4	84.68	77.72 (FAIL)	87.26	87.26 MHz

Both architectures drop sharply from N=2 to N=4 (SDRAM: -21.6%, dual-PSRAM: -15.7%). Resource utilization (Yosys, N=4 vs N=2, SDRAM): TRELLIS_FF 6215 vs 3724, TRELLIS_COMB 5516 vs 3783 — roughly doubles, as expected (4 vs 2 memory-manager/neural-processor instances). This confirms the Fmax ceiling is dominated by N scaling of shared logic fan-out (more GEN_SLOT instances → longer nets to the shared dependency_manager/arbiter), not by anything unique to N=4 SDRAM.

3. SDRAM vs dual-PSRAM critical path comparison (Part A.3)

The two architectures' #1 reported critical paths are in DIFFERENT (but both pre-existing, unchanged) modules:

	SDRAM N=4 (seed3, 81.55MHz)	Dual-PSRAM N=4 (seed3, 87.26MHz)
Source	dependency_manager.u_dep_mgr.node_state[15]	nms_memory_manager_stream_wide.GEN_SLOT[1].u_mm.x_base_reg
Destination	dependency_manager.ready_result_addr	nms_memory_manager_stream_wide.GEN_SLOT[1].u_mm.rd_ptr (CE)
Path type	first_ready_idx priority encoder + array read	buf_valid/issue_rd_now read-issue combinational chain
Logic delay	3.01 ns	3.41 ns
Routing delay	9.26 ns	8.05 ns
Total	12.26 ns (81.55MHz)	11.46 ns (87.26MHz)

Both critical paths sit in modules completely unchanged between the two architectures (dependency_manager.v is byte-for-byte identical; nms_memory_manager_stream_wide.v differs only in its MEM_DATA_WIDTH parameter value, not its control logic). Concrete explanation for 110.28MHz → 81.55MHz: the 110.28MHz figure does not reproduce with this toolchain/seed/methodology for the SAME (unmodified) dual-PSRAM RTL at N=4 — the best this round's own re-synthesis achieves for dual-PSRAM N=4 is 87.26MHz. The originally reported 110.28MHz was very likely obtained under different tool version, seed, or placement-effort conditions not reproduced here; this is disclosed honestly rather than papered over. The REAL, consistent-methodology gap between the two architectures at N=4 is ~7% (81.55 vs 87.26MHz), and both share the SAME root driver (N=4 fan-out into shared control logic), with SDRAM's own added logic providing a smaller secondary congestion effect on top.

4. Timing root cause

Primary driver: N=4 scaling of shared, pre-existing control logic (dependency_manager.v's priority encoder and/or nms_memory_manager_stream_wide.v's read-issue chain), present in BOTH architectures, unrelated to SDRAM. Secondary, smaller driver: SDRAM's added arbiter/controller logic increases overall die utilization, modestly worsening placement congestion for the SHARED bottleneck (a well-known FPGA phenomenon — unrelated logic elsewhere can lengthen an unrelated critical path purely through reduced placement freedom). The SDRAM controller/model themselves are NOT on the critical path in any measured run — the fixed-latency, always-precharge design (STEP16) is timing-friendly by construction.

5. Minimum fix experiment — attempted, reverted

Fix attempted: pipeline dependency_manager.v's first_ready_idx/ any_ready scan output by one register stage, consumed by the dispatch branch one cycle later (control-plane latency only, no change to tile delivery, no NP serialization, no NP-SDRAM connection).

Result: FAILED functional regression. A standalone unit test (16 independent nodes, back-to-back registration, ready_ready held high) showed node 0 dispatched twice — a real double-dispatch race: the register could capture a "node still READY" snapshot on the exact cycle that node's own dispatch was also committing, and use the stale snapshot to re-dispatch it one cycle later. A first attempted repair (gating the register update by !ready_valid) did not fix it — traced further and found the gate blocks the wrong cycle window, leaving the register still one full cycle stale relative to the commit. Given the real complexity of correctly reconciling snapshot timing with commit timing in this shared, heavily-reused module, and given N=4 already meets the hard minimum without any change, the edit was reverted in full (git checkout --, confirmed clean). ERR-0021 documents the root cause for any future attempt.

Conclusion for Part A: no RTL change is recommended or shipped this round. N=4 reliably exceeds 80MHz (81.55MHz, best-of-3-seeds, POST-P&R) on the unmodified, STEP16-validated RTL. The "preferred" (≥90MHz) and "excellent" (≥100MHz) tiers are not reached at N=4 by either architecture with this toolchain — this is a pre-existing, shared-architecture characteristic, not a regression introduced by SDRAM adoption.

6. Cycle decomposition (Part B)

Real, testbench-only instrumentation (no RTL touched) on the STEP16-validated D-Stress benchmark (256 neurons, 4096 tiles), INTEGRATED BENCHMARK classification unless noted. Full data: step17_cycle_decomposition.csv.

Category	N=2 (slot-cycles)	N=2 (%)	N=4 (slot-cycles)	N=4 (%)
slot-cycle budget (N_SLOTS×total_cycles)	104,322	100%	197,720	100%
useful MAC/tile-delivery cycles	4,080	3.91%	4,078	2.06%
weight-wait cycles (measured, STEP11 instrumentation)	86,464	82.88%	179,756	90.91%
per-slot idle cycles (sum)	1,072	1.03%	1,772	0.90%
unaccounted residual	12,706	12.18%	12,114	6.13%

The residual is not invented as a specific category: it plausibly combines activation-wait cycles, pipeline/tile-boundary bubbles, and job-dispatch overhead, none of which were separately isolated this round (would require additional instrumentation of the skid-buffer and activation-residency signals specifically — flagged as follow-up work, not fabricated here). A small (~16 slot-cycle) discrepancy between two independently-computed tile-delivery counters (the pre-existing slot_tiles_delivered sum vs this round's new useful_mac_cycles) is also folded into the residual and disclosed rather than hidden. The accounting closes to within 100% by construction (every slot-cycle is either useful, weight-wait, idle, or residual) — nothing is double-counted or dropped.

Startup: 56 cycles before the first tile is delivered anywhere. Drain: 32 cycles after the last tile delivery until job completion. Both are negligible relative to the 49,430/52,161-cycle totals (<0.2%).

7. Processor utilization (Part B.2)

Metric	N=2	N=4	Definition
Theoretical peak MAC/cycle	16	32	N_SLOTS × P_IN, THEORETICAL
Sustained MAC/cycle	0.6282	0.6629	tiles×8/total_cycles, INTEGRATED BENCHMARK
Processor utilization	3.93%	2.07%	sustained/theoretical, DERIVED
Cycles with 0 active slots	0.03%	0.03%	active=mm.state≠IDLE
Cycles with 1 active	2.00%	0.03%
Cycles with 2 active	97.97%	0.66%
Cycles with 3 active	N/A	2.07%
Cycles with 4 active	N/A	97.22%

Nearly all cycles show ALL slots simultaneously "active" (mm.state≠ IDLE) — but "active" (FSM not idle) is very different from "useful" (delivering a tile): the FSM sits in a busy-but-stalled state for the large majority of that time (per Part B's own weight-wait figure, ~91% at N=4). Peak instantaneous MAC/cycle was not separately measured this round (would require per-cycle tile-delivery-count sampling at finer granularity than the histogram already captures — the existing histogram already IS the per-cycle active-slot count, but distinguishing "active" from "instantaneously delivering" requires the useful_mac accounting above, not a separate peak metric). Startup vs steady-state vs drain utilization was not separately broken out beyond the startup/drain cycle counts in Part B — both are a negligible fraction (<0.2%) of total runtime, so a separate utilization split for them would not be numerically meaningful at this scale.

8. SDRAM effectiveness (Part C)

Real signals traced directly on u_sdram_backend.u_sdram_ctrl (the single physical controller servicing all weight-fetch traffic). Full data: step17_sdram_effectiveness.csv.

Metric	N=2	N=4
Read transactions	4,096	4,096
Write transactions	0	0 (weight fetch is read-only, as designed)
Bytes transferred	32,768	32,768
Controller busy	94.71%	99.92%
Refresh events (real AUTO REFRESH issued)	42	40
Request latency (req→ready)	min 10, max 16, avg 10.06 cyc	min 10, max 16, avg 10.06 cyc
Isolated single-transaction cost (Phase 4/6 baseline)	10 cyc	10 cyc
N-way arbitration overhead	+2.73 cyc/tile	+2.07 cyc/tile
Sustained bandwidth	50.26 MB/s	53.04 MB/s
Nominal (2B×16bit×80MHz)	160.0 MB/s	160.0 MB/s
Bandwidth utilization	31.4%	33.2%

Reads/writes do not alternate inefficiently — there are zero writes; every transaction is a read, and the controller's own always-auto-precharge design means every transaction pays the same fixed row-open/row-close cost regardless of address pattern (bank/row locality is irrelevant to this controller's timing by construction — it never keeps a row open across transactions, so there is no page-hit/page-miss distinction to analyze here). Average request latency (10.06 cycles) sits almost exactly at the fixed isolated minimum (10 cycles) — confirming requests are essentially NEVER queued waiting for arbitration; the controller is simply always busy with back-to-back transactions.

Classification: MEMORY-BANDWIDTH-BOUND, not latency-bound (avg latency ≈ minimum possible latency) and not primarily controller/arbitration-bound (busy 99.92% of the time means the controller is essentially never idle waiting for the next request — the arbiter is not introducing meaningful queuing delay on top of the controller's own fixed service time). The modest N-way arbitration overhead (+2.07 to +2.73 cycles/tile above the isolated 10-cycle minimum) reflects real, small round-robin-turn-taking cost among N_SLOTS competing requesters, not a queuing pathology.

9. Roofline update (Part D)

Ceiling	N=2	N=4	Classification
Compute ceiling (N×P_IN)	16 MAC/cycle	32 MAC/cycle	THEORETICAL
Internal memory ceiling (ideal processor-side operand delivery)	not measured this round	not measured this round	requires an ideal-memory-model variant (STEP11/14's own ideal_memory_model.v family); flagged as N/A rather than invented
External SDRAM ceiling (measured sustained)	50.26 MB/s	53.04 MB/s	INTEGRATED BENCHMARK (real controller behavior, not nominal bus width)
System ceiling (actual observed)	0.6282 MAC/cycle	0.6629 MAC/cycle	INTEGRATED BENCHMARK

The dominant ceiling is the external SDRAM ceiling. The system's actual sustained throughput (0.66 MAC/cycle at N=4) tracks the real SDRAM controller's own fixed 10-cycles/tile transaction cost (plus a small arbitration overhead) far more closely than it tracks the 32 MAC/cycle compute ceiling — confirmed directly by the controller sitting busy 99.92% of the runtime (Part C). Nominal SDRAM bandwidth (160MB/s THEORETICAL) is NOT used as a stand-in for real behavior anywhere in this roofline — the external ceiling entry above is the REAL, measured, integrated-benchmark bandwidth.

10. Resource comparison (Part F)

No RTL change was kept this round (the Part A fix was reverted), so resources are unchanged from the STEP16 baseline: TRELLIS_IO 194/245, TRELLIS_FF 6215 (N=4)/3724 (N=2), TRELLIS_COMB 5516/3783, MULT18X18D 32/16, DP16KD 0/0. No tradeoff to report — Fmax, resources, and cycle counts are all identical to STEP16's own already-reported figures.

11. Before/after measurements (Part E)

No timing fix was shipped, so there is no "after" configuration to validate — the STEP16 baseline stands as both "before" and current:

	Fmax (N=4, best-of-3)	Cycles (N=4)	Bit-exact
BEFORE (STEP16)	81.55 MHz	49,430	PASS
AFTER (attempted fix)	not measured (reverted before synthesis, due to functional regression)	49,503 (regressed, wrong results)	FAIL
Current (STEP17, unchanged RTL)	81.55 MHz	49,430	PASS (re-confirmed)

The attempted fix is reported honestly as a failed experiment, not retried under time pressure — matching the spec's own "a timing improvement that reduces throughput [or breaks correctness] is NOT automatically an improvement" rule.

12. N2/N4 scaling check (Part G)

	N=2	N=4	N=8
Cycles	52,161	49,430	not run (optional, exploratory only per spec — skipped this round to keep effort on N=4)
Cycles/tile	12.73	12.07	—
Sustained MAC/cycle	0.6282	0.6629	—
Bit-exact	PASS	PASS	—
Fmax (best-of-3, POST-P&R)	103.99 MHz	81.55 MHz	—

N=4 is modestly FASTER in total cycles than N=2 (5.5% fewer cycles) despite doubling theoretical compute — expected and consistent with Part D's own finding that the system is memory-bandwidth-bound in both configurations (SDRAM busy 94.71% at N=2, 99.92% at N=4): extra compute slots cannot be exploited once the shared SDRAM port is already the limiting resource. The architecture scales cleanly in the sense that N=4 introduces no new correctness issues, no new deadlocks, and no throughput regression versus N=2 — it simply cannot yet convert the extra compute capacity into proportionally more throughput, because memory bandwidth (not compute or Fmax) is the active constraint.

13. Limitations

• The originally-reported dual-PSRAM 110.28MHz figure could not be reproduced with this round's own toolchain/seed/methodology (best achieved: 87.26MHz) — flagged as an open discrepancy in the ORIGINAL STEP15/16 comparison basis, not resolved here.
• Internal (ideal) memory ceiling was not separately measured this round (would require reviving an ideal_memory_model.v-style variant) — marked N/A, not estimated.
• The ~6-12% "unaccounted residual" in the cycle decomposition was not further subdivided into activation-wait vs pipeline-bubble vs dispatch-overhead components this round.
• N=8 was not run (explicitly optional/exploratory per the governing spec).
• Peak instantaneous MAC/cycle and a separate startup/steady-state/ drain utilization split were not measured (both flagged as N/A rather than guessed).
• The attempted timing fix's failure mode (dispatch race) was root-caused only to the point of confirming it is unsafe as implemented — a fully correct pipelined version was not derived this round, given the risk of a heavily-reused shared module.

14. Final recommendation

Keep the STEP16 SDRAM architecture and RTL exactly as validated, with no changes. N=4 already meets the hard minimum Fmax requirement (81.55MHz ≥ 80MHz, POST-P&R, best-of-3-seeds) without any modification. The Fmax ceiling at N=4 is a real, now-understood, shared-architecture characteristic (present in the dual-PSRAM baseline too, once measured with a consistent methodology) rather than an SDRAM-specific defect, so there is no urgent architectural pressure to change course. The system's real bottleneck for further throughput gains is now clearly identified as external SDRAM bandwidth (controller busy 99.92% at N=4, near its own fixed-latency minimum) — the next optimization opportunity, if pursued in a future step, is the SDRAM controller's own per-transaction efficiency (e.g. page-mode/ keep-row-open optimization, explicitly a LARGER change than this round's "smallest possible" mandate allows), not further parallelism (N=8) and not a dependency-manager timing patch. N=4 is a solid, validated V2 baseline, and the path toward N=8 should be understood as primarily a memory-bandwidth question, not a timing-closure or compute-scaling question.