michele/FPGA-Neural
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
Files
8ad04987dea6f7eea96c1cf213c21336b4b0a348
FPGA-Neural/hardware/v3/rtl/packed_slot.v
T
micheleandClaude Sonnet 5 43a12379a5 feat: MILESTONE - real activation-fetch engine, full N=2 system verified on real DDR3 (EXP-0079) 
Closes the last major disclosed functional gap: packed_slot.v's
activation data was read through a combinational stand-in since
EXP-0062. New act_tile_fetch.v reads activation tiles directly from
DDR3 (no on-chip buffering needed, unlike weights -- activation data
has no reuse), sharing each slot's existing ctrl port with its own
weight-prefetch engine. Real memory layout: one full BURST_LEN=8-word
burst per tile, deliberately avoiding any runtime-indexed part-select
given this project's thin P&R timing margin (EXP-0078).

Verified at three levels: act_tile_fetch.v alone (6/6), packed_slot.v
with real preloaded activation data (9/9), and the full N=2 system
against real DDR3 via xsim (8/8, 0 errors) -- the first time this
project's compute path has been verified end-to-end with real DDR3
for both weights and activations.

Retired hardware/v3/rtl/n2_system_top.v and its testbench (pre-DDR3
SDR-placeholder era, fully superseded by n2_system_ddr3_top.v).

Also: docs/PHYSICAL_REALIZATION.md (real pinout/parts/timing/protocol
reference for the physical board) and CLAUDE.md (persistent project
instructions for future Claude Code sessions).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUG92aM9m68TRc4rG55BcC
2026-09-20 08:13:15 +02:00

395 lines
17 KiB
Verilog
Raw Blame History

`timescale 1ns/1ps
// ============================================================
// V3 -- packed_slot.v: real synthesizable per-slot sequencer, the
// piece that promotes EXP-0062's own PROCEDURAL testbench sequence
// (prefetch -> swap -> job dispatch -> tile-by-tile operand feed ->
// result capture) into real RTL, exactly the same class of promotion
// weight_tile_gather.v already did for the byte-gather step
// (EXP-0061).
//
// Wraps: layer_prefetch_ctrl.v -> layer_weight_buffer.v ->
// weight_tile_gather.v -> neural_processor_packed.v, driven by a new
// sequencing FSM, presenting the external contract neural_director_
// packed.v already expects (job_start/x_base_a/b/w_base/n_tiles/
// node_id_a/b -> job_done/result_data_a/b/result_node_id_a/b).
//
// ACTIVATION FETCH (EXP-0079, real, closes the gap this header used to
// disclose as deferred): act_tile_fetch.v reads each tile's activation
// data DIRECTLY from the shared DDR3 bus, one tile at a time -- no
// on-chip buffering/prefetch (unlike weights, activation data is read
// exactly once per job, so buffering it would add complexity for zero
// reuse benefit). It shares THIS slot's own single ctrl_req/addr/etc
// port with layer_prefetch_ctrl.v (u_pf): the two are mutually
// exclusive in time by FSM construction (weight prefetch always fully
// completes, including its own consume_done, before the tile loop
// that needs activation data ever starts), muxed below on act_mem_
// active. The outer arbiter's grant (mem_active/mem_grant, this
// module's own top-level ports) is now also needed during activation
// fetch, not just weight prefetch -- held PER TILE (one 2-burst fetch,
// lane A then lane B), released between tiles, matching this
// project's own established "lock the grant for one whole logical
// fetch, not longer" discipline (avoids starving the other slot for
// the whole tile loop's duration).
//
// MEMORY LAYOUT this requires of activation data in DDR3: each tile
// occupies its own full BURST_LEN=8-word burst slot (see act_tile_
// fetch.v's own header for why -- avoiding a runtime-indexed part-
// select, a known Fmax risk this project's already-thin P&R margin,
// EXP-0078, can't afford right now). Documented for whoever prepares
// host-side data layout in the physical realization doc.
//
// Also disclosed: no result-writeback engine exists yet either --
// result_addr_a/b are passed through unused, for a future writeback
// stage to consume.
//
// EVERY job re-fetches its layer from SDRAM (no resident-weight-skip
// optimization) -- correctness first; EXP-0057's own measured
// prefetch/reuse PERFORMANCE benefit is a property of the buffer
// being read MANY times per fetch (many reuse positions per Director-
// dispatched pair's own tile loop is NOT what's being reused here --
// see note in the FSM below), not of skipping fetches across
// DIFFERENT Director dispatches; adding that optimization is future
// work, not a correctness requirement.
// ============================================================
module packed_slot #(
parameter DATA_WIDTH = 8,
parameter P_IN = 8,
parameter ACC_WIDTH = 32,
parameter BURST_LEN = 8,
parameter ADDR_WIDTH = 26,
parameter LAYER_BYTES = 128,
parameter BUFADDRW = $clog2(LAYER_BYTES)
)(
input wire clk,
input wire rst,
// ---- Director interface (matches neural_director_packed.v's own
// per-slot output ports exactly) ----
input wire job_start,
input wire [ADDR_WIDTH-1:0] x_base_a,
input wire [ADDR_WIDTH-1:0] x_base_b,
input wire [ADDR_WIDTH-1:0] w_base,
input wire [15:0] n_tiles,
input wire [ADDR_WIDTH-1:0] result_addr_a,
input wire [ADDR_WIDTH-1:0] result_addr_b,
input wire [15:0] node_id_a,
input wire [15:0] node_id_b,
output reg job_done, // one-cycle pulse
output reg signed [DATA_WIDTH-1:0] result_data_a,
output reg signed [DATA_WIDTH-1:0] result_data_b,
output reg [15:0] result_node_id_a,
output reg [15:0] result_node_id_b,
output reg [ADDR_WIDTH-1:0] result_addr_a_out,
output reg [ADDR_WIDTH-1:0] result_addr_b_out,
// high exactly while this slot needs exclusive access to the
// shared SDRAM controller (its own weight-fetch OR activation-
// fetch phase) -- a shared-controller arbiter uses this to lock a
// grant for the whole multi-burst fetch, not just one transaction.
output wire mem_active,
// grant from a shared-controller arbiter (see mem_active's own
// comment): must be asserted before this slot may pulse its own
// layer_prefetch_ctrl.v start, since that module's ctrl_req is a
// one-shot pulse with no retry -- issuing it before the arbiter
// has actually granted this slot the bus loses it permanently
// (found empirically integrating N=2 slots behind sdram_slot_
// arbiter2.v: a slot could hang forever in S_WAIT with ctrl_req
// already dropped and ctrl_ready never coming). Tie high for a
// single-slot (N=1, no arbiter) system.
input wire mem_grant,
// ---- SDRAM controller port (connects directly, or through a
// shared arbiter for N>1 slots) ----
output wire ctrl_req,
output wire ctrl_wr,
output wire [ADDR_WIDTH-2:0] ctrl_addr,
output wire [16*BURST_LEN-1:0] ctrl_wdata,
output wire [2*BURST_LEN-1:0] ctrl_wmask,
input wire [16*BURST_LEN-1:0] ctrl_rdata,
input wire ctrl_ready,
input wire ctrl_busy
);
localparam S_IDLE = 4'd0,
S_MEMWAIT = 4'd1,
S_PREFETCH = 4'd2,
S_SWAP = 4'd3,
S_JOBSTART = 4'd4,
S_TILEREQ = 4'd5,
S_TILEWAIT = 4'd6,
S_OPERAND = 4'd7,
S_RESULT = 4'd8,
S_DONE = 4'd9;
reg [3:0] state;
reg [ADDR_WIDTH-1:0] w_base_lat, x_base_a_lat, x_base_b_lat;
reg [15:0] n_tiles_lat;
reg [ADDR_WIDTH-1:0] result_addr_a_lat, result_addr_b_lat;
reg [15:0] node_id_a_lat, node_id_b_lat;
reg [15:0] tcnt;
// ---- layer_prefetch_ctrl.v ----
reg pf_start;
wire pf_busy, pf_done;
wire pf_fill_we;
wire [BUFADDRW-1:0] pf_fill_addr;
wire [DATA_WIDTH-1:0] pf_fill_data;
wire pf_ctrl_req, pf_ctrl_wr;
wire [ADDR_WIDTH-2:0] pf_ctrl_addr;
wire [16*BURST_LEN-1:0] pf_ctrl_wdata;
wire [2*BURST_LEN-1:0] pf_ctrl_wmask;
layer_prefetch_ctrl #(
.DATA_WIDTH(DATA_WIDTH), .LAYER_BYTES(LAYER_BYTES), .BURST_LEN(BURST_LEN), .ADDR_WIDTH(ADDR_WIDTH-1)
) u_pf (
.clk(clk), .rst(rst),
.start(pf_start), .layer_base(w_base_lat[ADDR_WIDTH-2:0]), .busy(pf_busy), .done(pf_done),
.fill_we(pf_fill_we), .fill_addr(pf_fill_addr), .fill_data(pf_fill_data),
.ctrl_req(pf_ctrl_req), .ctrl_wr(pf_ctrl_wr), .ctrl_addr(pf_ctrl_addr),
.ctrl_wdata(pf_ctrl_wdata), .ctrl_wmask(pf_ctrl_wmask),
.ctrl_rdata(ctrl_rdata), .ctrl_ready(ctrl_ready), .ctrl_busy(ctrl_busy)
);
// ---- act_tile_fetch.v (EXP-0079): real activation fetch, shares
// this slot's own ctrl port with u_pf above (mutually exclusive in
// time -- see header) ----
reg act_req;
wire act_valid;
wire signed [DATA_WIDTH*P_IN-1:0] act_data_a_w, act_data_b_w;
wire act_mem_active;
wire act_ctrl_req, act_ctrl_wr;
wire [ADDR_WIDTH-2:0] act_ctrl_addr;
wire [16*BURST_LEN-1:0] act_ctrl_wdata;
wire [2*BURST_LEN-1:0] act_ctrl_wmask;
act_tile_fetch #(
.DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .BURST_LEN(BURST_LEN), .ADDR_WIDTH(ADDR_WIDTH-1)
) u_act (
.clk(clk), .rst(rst),
.req(act_req), .base_a(x_base_a_lat[ADDR_WIDTH-2:0]), .base_b(x_base_b_lat[ADDR_WIDTH-2:0]),
.tcnt(tcnt), .valid(act_valid), .data_a(act_data_a_w), .data_b(act_data_b_w),
.mem_active(act_mem_active), .mem_grant(mem_grant),
.ctrl_req(act_ctrl_req), .ctrl_wr(act_ctrl_wr), .ctrl_addr(act_ctrl_addr),
.ctrl_wdata(act_ctrl_wdata), .ctrl_wmask(act_ctrl_wmask),
.ctrl_rdata(ctrl_rdata), .ctrl_ready(ctrl_ready), .ctrl_busy(ctrl_busy)
);
// mutually exclusive by FSM construction (weight prefetch always
// fully completes, incl. consume_done, before the tile loop that
// triggers act_req ever starts) -- safe to select on act_mem_active alone.
assign ctrl_req = act_mem_active ? act_ctrl_req : pf_ctrl_req;
assign ctrl_wr = act_mem_active ? act_ctrl_wr : pf_ctrl_wr;
assign ctrl_addr = act_mem_active ? act_ctrl_addr : pf_ctrl_addr;
assign ctrl_wdata = act_mem_active ? act_ctrl_wdata : pf_ctrl_wdata;
assign ctrl_wmask = act_mem_active ? act_ctrl_wmask : pf_ctrl_wmask;
assign mem_active = (state == S_MEMWAIT) || (state == S_PREFETCH) || act_mem_active;
// ---- layer_weight_buffer.v ----
wire [BUFADDRW-1:0] lwb_rd_addr;
wire [DATA_WIDTH-1:0] lwb_rd_data;
reg consume_done;
layer_weight_buffer #(.DATA_WIDTH(DATA_WIDTH), .LAYER_DEPTH(LAYER_BYTES)) u_lwb (
.clk(clk), .rst(rst),
.fill_we(pf_fill_we), .fill_addr(pf_fill_addr), .fill_data(pf_fill_data), .fill_done(pf_done),
.rd_addr(lwb_rd_addr), .rd_data(lwb_rd_data), .consume_done(consume_done),
.active_sel(), .swapped()
);
// ---- weight_tile_gather.v ----
reg tile_req;
reg [BUFADDRW-1:0] tile_base;
reg tile_seen, act_seen; // S_TILEWAIT join latches (weight vs activation, see header)
wire tile_valid;
wire [DATA_WIDTH*P_IN-1:0] tile_data;
weight_tile_gather #(
.DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .BUFADDRW(BUFADDRW)
) u_gather (
.clk(clk), .rst(rst),
.tile_req(tile_req), .tile_base(tile_base),
.tile_valid(tile_valid), .tile_data(tile_data),
.rd_addr(lwb_rd_addr), .rd_data(lwb_rd_data)
);
// ---- neural_processor_packed.v ----
reg job_valid_np;
wire job_ready_np;
reg [1:0] job_activation;
reg signed [DATA_WIDTH-1:0] job_bias;
reg operand_valid;
wire operand_ready;
reg signed [DATA_WIDTH*P_IN-1:0] input_data_a_r, input_data_b_r;
reg [DATA_WIDTH*P_IN-1:0] weight_data_r;
reg tile_last;
wire result_valid_np;
reg result_ready;
wire signed [DATA_WIDTH-1:0] result_data_a_np, result_data_b_np;
wire [15:0] result_node_id_a_np, result_node_id_b_np;
wire [3:0] np_state;
wire np_error;
neural_processor_packed #(
.DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .ACC_WIDTH(ACC_WIDTH)
) u_np (
.clk(clk), .rst(rst),
.job_valid(job_valid_np), .job_ready(job_ready_np),
.job_node_id_a(node_id_a_lat), .job_node_id_b(node_id_b_lat),
.job_bias(job_bias), .job_activation(job_activation),
.operand_valid(operand_valid), .operand_ready(operand_ready),
.input_data_a(input_data_a_r), .input_data_b(input_data_b_r),
.weight_data(weight_data_r), .tile_last(tile_last),
.result_valid(result_valid_np), .result_ready(result_ready),
.result_data_a(result_data_a_np), .result_data_b(result_data_b_np),
.result_node_id_a(result_node_id_a_np), .result_node_id_b(result_node_id_b_np),
.np_state(np_state), .np_error(np_error)
);
localparam ACT_RELU = 2'd1;
always @(posedge clk) begin
if (rst) begin
state <= S_IDLE;
job_done <= 1'b0;
pf_start <= 1'b0;
consume_done <= 1'b0;
tile_req <= 1'b0;
act_req <= 1'b0;
tile_seen <= 1'b0;
act_seen <= 1'b0;
job_valid_np <= 1'b0;
operand_valid<= 1'b0;
tile_last <= 1'b0;
result_ready <= 1'b0;
job_bias <= {DATA_WIDTH{1'b0}};
job_activation <= ACT_RELU;
tcnt <= 16'd0;
end else begin
job_done <= 1'b0;
pf_start <= 1'b0;
consume_done <= 1'b0;
tile_req <= 1'b0;
act_req <= 1'b0;
case (state)
S_IDLE: begin
if (job_start) begin
w_base_lat <= w_base;
x_base_a_lat <= x_base_a;
x_base_b_lat <= x_base_b;
n_tiles_lat <= n_tiles;
result_addr_a_lat <= result_addr_a;
result_addr_b_lat <= result_addr_b;
node_id_a_lat <= node_id_a;
node_id_b_lat <= node_id_b;
job_bias <= {DATA_WIDTH{1'b0}};
job_activation <= ACT_RELU;
state <= S_MEMWAIT;
end
end
S_MEMWAIT: begin
if (mem_grant) begin
pf_start <= 1'b1;
state <= S_PREFETCH;
end
end
S_PREFETCH: begin
if (pf_done) begin
consume_done <= 1'b1;
state <= S_SWAP;
end
end
S_SWAP: begin
// one settle cycle for layer_weight_buffer.v's own
// do_swap (fill_done_latched already set from
// pf_done above; consume_done pulsed this cycle) --
// matches EXP-0058/0062's own tested sequencing.
job_valid_np <= 1'b1;
state <= S_JOBSTART;
end
S_JOBSTART: begin
if (job_valid_np && job_ready_np) begin
job_valid_np <= 1'b0;
tcnt <= 16'd0;
state <= S_TILEREQ;
end
end
S_TILEREQ: begin
tile_req <= 1'b1;
tile_base <= tcnt[BUFADDRW-1:0]*P_IN[BUFADDRW-1:0];
act_req <= 1'b1;
tile_seen <= 1'b0;
act_seen <= 1'b0;
state <= S_TILEWAIT;
end
// Real join: weight_tile_gather.v's tile_valid (fast,
// on-chip) and act_tile_fetch.v's act_valid (real
// DDR3 latency, 2 bursts) do NOT arrive on the same
// cycle in general -- latch whichever comes first,
// proceed only once BOTH have been seen. Handles
// either arrival order correctly, not just the
// expected-common one (weight first).
S_TILEWAIT: begin
if (tile_valid) begin
weight_data_r <= tile_data;
tile_seen <= 1'b1;
end
if (act_valid) begin
input_data_a_r <= act_data_a_w;
input_data_b_r <= act_data_b_w;
act_seen <= 1'b1;
end
if ((tile_valid || tile_seen) && (act_valid || act_seen)) begin
tile_last <= (tcnt == n_tiles_lat - 16'd1);
operand_valid <= 1'b1;
state <= S_OPERAND;
end
end
S_OPERAND: begin
if (operand_valid && operand_ready) begin
operand_valid <= 1'b0;
tile_last <= 1'b0;
if (tcnt == n_tiles_lat - 16'd1) begin
result_ready <= 1'b1;
state <= S_RESULT;
end else begin
tcnt <= tcnt + 16'd1;
state <= S_TILEREQ;
end
end
end
S_RESULT: begin
if (result_valid_np) begin
result_data_a <= result_data_a_np;
result_data_b <= result_data_b_np;
result_node_id_a <= result_node_id_a_np;
result_node_id_b <= result_node_id_b_np;
result_addr_a_out <= result_addr_a_lat;
result_addr_b_out <= result_addr_b_lat;
result_ready <= 1'b0;
job_done <= 1'b1;
state <= S_IDLE;
end
end
default: state <= S_IDLE;
endcase
end
end
endmodule
Reference in New Issue View Git Blame Copy Permalink