michele/FPGA-Neural
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

feat: DDRManager phase 1 - single-slot look-ahead activation prefetch (EXP-0083)

Browse Source 
New ddr_prefetch_mgr.v wraps act_tile_fetch.v with a depth-2 ping-pong
buffer, issuing the next tile's DDR3 fetch as soon as the fetch engine
is free instead of waiting for packed_slot.v to finish consuming the
current tile. Wired into packed_slot.v's tile loop (job-level start
instead of per-tile req), simplifying the S_TILEWAIT join in the process
(ddrpf_tile_valid is level-held, no separate act_seen latch needed).

Verification: new tb_ddr_prefetch_mgr.v (25/25 PASS after fixing a real
testbench polling race found via iteration-tagged tracing, not an RTL
bug), tb_packed_slot.v re-run unmodified (9/9 PASS, bit-identical
results), tb_n2_system_ddr3.v re-run via real xsim against real
ddr3_model.sv (8/8 PASS). Real P&R: WNS +0.073ns (up from EXP-0082's
+0.068ns), LUTs 5644, DSP48E1 16 unchanged, 0 failing endpoints.

Honest result: real A/B on the actual DDR3 backend (same testbench,
before/after) shows a real but modest 2.86% reduction in total
simulated time - smaller than the original hypothesis suggested, because
neural_processor_packed.v already accepts one operand per cycle, so the
per-tile dead time being removed was already small relative to real DDR3
fetch latency. Docs updated to report this honestly rather than oversell
it; the larger multi-slot DDRManager is deferred pending re-measurement
against the (still pending, user-gated) 32-bit channel widening.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUG92aM9m68TRc4rG55BcC
This commit is contained in:
michele
2026-09-20 10:59:45 +02:00
co-authored by Claude Sonnet 5
parent cbd16dd727
commit fa327b75ca
6 changed files with 801 additions and 121 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hardware/v3/rtl/ddr_prefetch_mgr.v
+176
View File
@@ -0,0 +1,176 @@
`timescale 1ns/1ps
// ============================================================
// V3 -- ddr_prefetch_mgr.v: the DDRManager's real phase-1 deliverable
// (EXP-0083), implementing the user's own proposed idea -- "vorrei che
// orchestrator potesse 'prenotare' le letture future in RAM ... in modo
// da azzerare i tempi di attesa (o almeno ridurli al minimo)" -- scoped,
// per this project's own "one variable at a time" discipline and the
// validation plan docs/ARCHITECTURE_ANALYSIS.md S5.2 laid out, to a
// SINGLE slot's own activation-tile look-ahead first, before attempting
// a cross-slot/whole-Director-queue scheduler.
//
// WHAT THIS DOES: wraps act_tile_fetch.v (unmodified, reused as the
// "fetch exactly one tile" engine) with a depth-2 ping-pong buffer and a
// sequencer that issues the NEXT tile's fetch the INSTANT the fetch
// engine is free and that tile's buffer bank is free -- NOT waiting for
// packed_slot.v to have consumed the CURRENT tile first. This overlaps
// "fetching tile N+1" with "packed_slot.v consuming tile N", which the
// original per-tile req/wait loop (EXP-0079/0081) never did.
//
// WHAT THIS DOES NOT DO (disclosed, not glossed over): it does not
// change the real 1.24 GB/s physical DDR3 ceiling (S3.1 of the
// architecture doc) or the per-fetch latency of any SINGLE tile fetch --
// it only removes the small, real per-tile RE-REQUEST overhead (the
// S_TILEREQ pulse cycle + the S_OPERAND consume cycle packed_slot.v's
// own FSM previously spent NOT fetching, between one tile's data
// arriving and the next tile's fetch being issued). Given
// neural_processor_packed.v's own pipeline accepts one operand PER
// CYCLE once in NP_WAIT_OPERANDS (operand_ready is state-only, not
// gated on any internal pipeline stall), the real compute-side
// consumption cost per tile is ~1 cycle -- meaning this fix's real
// ceiling is bounded by that small per-tile overhead, not by hiding a
// large compute-bound stall. The real, measured improvement is reported
// in the EXP-0083 log entry, not assumed here.
//
// WHY A DOUBLE BUFFER (depth 2), not deeper: matches
// layer_weight_buffer.v's own proven ping-pong pattern in this
// codebase, and depth 2 is provably sufficient here -- the fetch
// sequencer can be at most 1 tile ahead of the consumer, since issuing
// tile N+2's fetch requires bank[(N+2)%2] == bank[N%2] to already be
// free, which only happens once tile N has been consumed. No unbounded
// lookahead is possible or attempted.
//
// WHY THIS IS TIMING-SAFE: bank selection for both the fill side
// (fetch_idx[0]) and the read side (consume_idx[0]) is a REGISTERED
// index bit, exactly the same "select known long before the data it
// gates" discipline act_tile_fetch.v's own header (EXP-0081) already
// established as safe -- never a bit racing live data. Requires its own
// real P&R re-check before being trusted at N>1 scale, per this
// project's standing practice (not assumed safe by analogy alone).
// ============================================================
module ddr_prefetch_mgr #(
parameter DATA_WIDTH = 8,
parameter P_IN = 8,
parameter BURST_LEN = 8,
parameter ADDR_WIDTH = 25 // word address, matches act_tile_fetch.v's own convention
)(
input wire clk,
input wire rst,
// ---- job-level control (packed_slot.v issues this ONCE per job,
// not once per tile -- the whole tile loop's lookahead is driven
// internally from here) ----
input wire job_start, // one-shot pulse
input wire [ADDR_WIDTH-1:0] base_a,
input wire [ADDR_WIDTH-1:0] base_b,
input wire [15:0] n_tiles,
// ---- per-tile consumption interface (packed_slot.v side) ----
// tile_valid is a LEVEL signal (unlike act_tile_fetch.v's one-cycle
// `valid` pulse) -- it stays high as long as the current
// consume-index's buffer bank holds unconsumed data, which may
// already be true the cycle packed_slot.v asks, if the lookahead
// fetch completed early. packed_slot.v pulses tile_consume once it
// has latched data_a/data_b, which frees this bank for the next
// lookahead fetch.
output wire tile_valid,
output wire signed [DATA_WIDTH*P_IN-1:0] data_a,
output wire signed [DATA_WIDTH*P_IN-1:0] data_b,
input wire tile_consume, // one-shot pulse
// ---- shared DDR3 controller port (identical shape to
// act_tile_fetch.v's own -- this module is a pure passthrough
// wrapper on this side, connects straight through to the inner
// act_tile_fetch instance) ----
output wire mem_active,
input wire mem_grant,
output wire ctrl_req,
output wire ctrl_wr,
output wire [ADDR_WIDTH-1:0] ctrl_addr,
output wire [16*BURST_LEN-1:0] ctrl_wdata,
output wire [2*BURST_LEN-1:0] ctrl_wmask,
input wire [16*BURST_LEN-1:0] ctrl_rdata,
input wire ctrl_ready,
input wire ctrl_busy
);
reg [ADDR_WIDTH-1:0] base_a_lat, base_b_lat;
reg [15:0] n_tiles_lat;
reg [15:0] fetch_idx, consume_idx;
reg fetch_inflight;
// ---- depth-2 ping-pong buffer ----
reg signed [DATA_WIDTH*P_IN-1:0] bank_data_a [0:1];
reg signed [DATA_WIDTH*P_IN-1:0] bank_data_b [0:1];
reg [1:0] bank_valid;
assign tile_valid = bank_valid[consume_idx[0]];
assign data_a = bank_data_a[consume_idx[0]];
assign data_b = bank_data_b[consume_idx[0]];
// ---- inner fetch engine: act_tile_fetch.v, reused unmodified,
// driven one tile at a time by this sequencer ----
reg act_req;
wire act_valid;
wire signed [DATA_WIDTH*P_IN-1:0] act_data_a_w, act_data_b_w;
act_tile_fetch #(
.DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .BURST_LEN(BURST_LEN), .ADDR_WIDTH(ADDR_WIDTH)
) u_act (
.clk(clk), .rst(rst),
.req(act_req), .base_a(base_a_lat), .base_b(base_b_lat),
.tcnt(fetch_idx), .valid(act_valid), .data_a(act_data_a_w), .data_b(act_data_b_w),
.mem_active(mem_active), .mem_grant(mem_grant),
.ctrl_req(ctrl_req), .ctrl_wr(ctrl_wr), .ctrl_addr(ctrl_addr),
.ctrl_wdata(ctrl_wdata), .ctrl_wmask(ctrl_wmask),
.ctrl_rdata(ctrl_rdata), .ctrl_ready(ctrl_ready), .ctrl_busy(ctrl_busy)
);
// can_issue: the inner fetch engine is free, there is a next tile
// left in this job, and that tile's destination bank has already
// been consumed (or was never filled yet, at job start).
wire can_issue = !fetch_inflight && (fetch_idx < n_tiles_lat) && !bank_valid[fetch_idx[0]];
always @(posedge clk) begin
if (rst) begin
base_a_lat <= {ADDR_WIDTH{1'b0}};
base_b_lat <= {ADDR_WIDTH{1'b0}};
n_tiles_lat <= 16'd0;
fetch_idx <= 16'd0;
consume_idx <= 16'd0;
fetch_inflight <= 1'b0;
act_req <= 1'b0;
bank_valid <= 2'b00;
end else begin
act_req <= 1'b0;
if (job_start) begin
base_a_lat <= base_a;
base_b_lat <= base_b;
n_tiles_lat <= n_tiles;
fetch_idx <= 16'd0;
consume_idx <= 16'd0;
fetch_inflight <= 1'b0;
bank_valid <= 2'b00;
end else begin
if (can_issue) begin
act_req <= 1'b1;
fetch_inflight <= 1'b1;
end
if (act_valid) begin
bank_data_a[fetch_idx[0]] <= act_data_a_w;
bank_data_b[fetch_idx[0]] <= act_data_b_w;
bank_valid[fetch_idx[0]] <= 1'b1;
fetch_idx <= fetch_idx + 16'd1;
fetch_inflight <= 1'b0;
end
if (tile_consume) begin
bank_valid[consume_idx[0]] <= 1'b0;
consume_idx <= consume_idx + 16'd1;
end
end
end
end
endmodule
hardware/v3/rtl/packed_slot.v
+67 -56
View File
@@ -15,29 +15,30 @@
// node_id_a/b -> job_done/result_data_a/b/result_node_id_a/b).
//
// ACTIVATION FETCH (EXP-0079, real, closes the gap this header used to
// disclose as deferred): act_tile_fetch.v reads each tile's activation
// data DIRECTLY from the shared DDR3 bus, one tile at a time -- no
// on-chip buffering/prefetch (unlike weights, activation data is read
// exactly once per job, so buffering it would add complexity for zero
// reuse benefit). It shares THIS slot's own single ctrl_req/addr/etc
// port with layer_prefetch_ctrl.v (u_pf): the two are mutually
// exclusive in time by FSM construction (weight prefetch always fully
// completes, including its own consume_done, before the tile loop
// that needs activation data ever starts), muxed below on act_mem_
// active. The outer arbiter's grant (mem_active/mem_grant, this
// module's own top-level ports) is now also needed during activation
// fetch, not just weight prefetch -- held PER TILE (one 2-burst fetch,
// lane A then lane B), released between tiles, matching this
// project's own established "lock the grant for one whole logical
// fetch, not longer" discipline (avoids starving the other slot for
// the whole tile loop's duration).
// disclose as deferred; EXP-0083 upgrades it to a look-ahead prefetch):
// ddr_prefetch_mgr.v wraps act_tile_fetch.v with a depth-2 ping-pong
// buffer, issuing tile N+1's fetch the instant the fetch engine is
// free rather than waiting for this slot to finish CONSUMING tile N --
// overlapping "fetch next tile" with "consume current tile" (see
// ddr_prefetch_mgr.v's own header for the real, honest, measured scope
// of the benefit -- it does not raise the physical DDR3 ceiling, only
// removes small real per-tile re-request overhead). It shares THIS
// slot's own single ctrl_req/addr/etc port with layer_prefetch_ctrl.v
// (u_pf): the two are mutually exclusive in time by FSM construction
// (weight prefetch always fully completes, including its own
// consume_done, before the tile loop that needs activation data ever
// starts), muxed below on act_mem_active. The outer arbiter's grant
// (mem_active/mem_grant, this module's own top-level ports) is now
// also needed during activation fetch, not just weight prefetch --
// held PER TILE (one 2-burst fetch, lane A then lane B), released
// between tiles, matching this project's own established "lock the
// grant for one whole logical fetch, not longer" discipline (avoids
// starving the other slot for the whole tile loop's duration).
//
// MEMORY LAYOUT this requires of activation data in DDR3: each tile
// occupies its own full BURST_LEN=8-word burst slot (see act_tile_
// fetch.v's own header for why -- avoiding a runtime-indexed part-
// select, a known Fmax risk this project's already-thin P&R margin,
// EXP-0078, can't afford right now). Documented for whoever prepares
// host-side data layout in the physical realization doc.
// MEMORY LAYOUT this requires of activation data in DDR3 (EXP-0081,
// v2 convention): two consecutive tiles share one full BURST_LEN=8-
// word burst (even index low 64 bits, odd index high 64 bits) -- see
// act_tile_fetch.v's own header and docs/PHYSICAL_REALIZATION.md S4.
//
// Also disclosed: no result-writeback engine exists yet either --
// result_addr_a/b are passed through unused, for a future writeback
@@ -153,11 +154,14 @@ module packed_slot #(
.ctrl_rdata(ctrl_rdata), .ctrl_ready(ctrl_ready), .ctrl_busy(ctrl_busy)
);
// ---- act_tile_fetch.v (EXP-0079): real activation fetch, shares
// this slot's own ctrl port with u_pf above (mutually exclusive in
// time -- see header) ----
reg act_req;
wire act_valid;
// ---- ddr_prefetch_mgr.v (EXP-0083): look-ahead activation fetch,
// shares this slot's own ctrl port with u_pf above (mutually
// exclusive in time -- see header). Job-level start (once per job,
// not once per tile -- the whole tile loop's lookahead sequencing
// happens inside this module).
reg ddrpf_job_start;
wire ddrpf_tile_valid;
reg ddrpf_tile_consume;
wire signed [DATA_WIDTH*P_IN-1:0] act_data_a_w, act_data_b_w;
wire act_mem_active;
@@ -166,12 +170,15 @@ module packed_slot #(
wire [16*BURST_LEN-1:0] act_ctrl_wdata;
wire [2*BURST_LEN-1:0] act_ctrl_wmask;
act_tile_fetch #(
ddr_prefetch_mgr #(
.DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .BURST_LEN(BURST_LEN), .ADDR_WIDTH(ADDR_WIDTH-1)
) u_act (
) u_ddrpf (
.clk(clk), .rst(rst),
.req(act_req), .base_a(x_base_a_lat[ADDR_WIDTH-2:0]), .base_b(x_base_b_lat[ADDR_WIDTH-2:0]),
.tcnt(tcnt), .valid(act_valid), .data_a(act_data_a_w), .data_b(act_data_b_w),
.job_start(ddrpf_job_start),
.base_a(x_base_a_lat[ADDR_WIDTH-2:0]), .base_b(x_base_b_lat[ADDR_WIDTH-2:0]),
.n_tiles(n_tiles_lat),
.tile_valid(ddrpf_tile_valid), .data_a(act_data_a_w), .data_b(act_data_b_w),
.tile_consume(ddrpf_tile_consume),
.mem_active(act_mem_active), .mem_grant(mem_grant),
.ctrl_req(act_ctrl_req), .ctrl_wr(act_ctrl_wr), .ctrl_addr(act_ctrl_addr),
.ctrl_wdata(act_ctrl_wdata), .ctrl_wmask(act_ctrl_wmask),
@@ -180,7 +187,7 @@ module packed_slot #(
// mutually exclusive by FSM construction (weight prefetch always
// fully completes, incl. consume_done, before the tile loop that
// triggers act_req ever starts) -- safe to select on act_mem_active alone.
// triggers ddrpf_job_start ever fires) -- safe to select on act_mem_active alone.
assign ctrl_req = act_mem_active ? act_ctrl_req : pf_ctrl_req;
assign ctrl_wr = act_mem_active ? act_ctrl_wr : pf_ctrl_wr;
assign ctrl_addr = act_mem_active ? act_ctrl_addr : pf_ctrl_addr;
@@ -204,7 +211,9 @@ module packed_slot #(
// ---- weight_tile_gather.v ----
reg tile_req;
reg [BUFADDRW-1:0] tile_base;
reg tile_seen, act_seen; // S_TILEWAIT join latches (weight vs activation, see header)
reg tile_seen; // S_TILEWAIT join latch (weight side only -- see header;
// the activation side, ddrpf_tile_valid, is level-held by
// ddr_prefetch_mgr.v so it needs no separate latch)
wire tile_valid;
wire [DATA_WIDTH*P_IN-1:0] tile_data;
@@ -261,9 +270,9 @@ module packed_slot #(
pf_start <= 1'b0;
consume_done <= 1'b0;
tile_req <= 1'b0;
act_req <= 1'b0;
ddrpf_job_start <= 1'b0;
ddrpf_tile_consume <= 1'b0;
tile_seen <= 1'b0;
act_seen <= 1'b0;
job_valid_np <= 1'b0;
operand_valid<= 1'b0;
tile_last <= 1'b0;
@@ -276,7 +285,8 @@ module packed_slot #(
pf_start <= 1'b0;
consume_done <= 1'b0;
tile_req <= 1'b0;
act_req <= 1'b0;
ddrpf_job_start <= 1'b0;
ddrpf_tile_consume <= 1'b0;
case (state)
S_IDLE: begin
@@ -320,42 +330,43 @@ module packed_slot #(
S_JOBSTART: begin
if (job_valid_np && job_ready_np) begin
job_valid_np <= 1'b0;
tcnt <= 16'd0;
state <= S_TILEREQ;
job_valid_np <= 1'b0;
tcnt <= 16'd0;
ddrpf_job_start <= 1'b1; // one-shot: kicks off the whole job's
// look-ahead tile loop inside u_ddrpf
state <= S_TILEREQ;
end
end
S_TILEREQ: begin
tile_req <= 1'b1;
tile_base <= tcnt[BUFADDRW-1:0]*P_IN[BUFADDRW-1:0];
act_req <= 1'b1;
tile_seen <= 1'b0;
act_seen <= 1'b0;
state <= S_TILEWAIT;
end
// Real join: weight_tile_gather.v's tile_valid (fast,
// on-chip) and act_tile_fetch.v's act_valid (real
// DDR3 latency, 2 bursts) do NOT arrive on the same
// cycle in general -- latch whichever comes first,
// proceed only once BOTH have been seen. Handles
// either arrival order correctly, not just the
// expected-common one (weight first).
// on-chip, one-cycle pulse -- latched via tile_seen)
// and u_ddrpf's ddrpf_tile_valid (real DDR3 latency,
// but LEVEL-held by the prefetch manager's own ping-
// pong buffer, possibly already true this cycle if the
// look-ahead fetch completed early) do NOT arrive on
// the same cycle in general -- proceed once BOTH are
// available. ddrpf_tile_valid needs no separate latch
// since it stays high until this slot pulses
// ddrpf_tile_consume itself.
S_TILEWAIT: begin
if (tile_valid) begin
weight_data_r <= tile_data;
tile_seen <= 1'b1;
end
if (act_valid) begin
input_data_a_r <= act_data_a_w;
input_data_b_r <= act_data_b_w;
act_seen <= 1'b1;
end
if ((tile_valid || tile_seen) && (act_valid || act_seen)) begin
tile_last <= (tcnt == n_tiles_lat - 16'd1);
operand_valid <= 1'b1;
state <= S_OPERAND;
if ((tile_valid || tile_seen) && ddrpf_tile_valid) begin
input_data_a_r <= act_data_a_w;
input_data_b_r <= act_data_b_w;
ddrpf_tile_consume <= 1'b1;
tile_last <= (tcnt == n_tiles_lat - 16'd1);
operand_valid <= 1'b1;
state <= S_OPERAND;
end
end