feat: real first step of 4x4 hybrid systolic architecture (EXP-0089)
Adds packed_pe.v (packed_slot.v's compute+activation-fetch+writeback subsystem, reusing ddr_prefetch_mgr.v/neural_processor_packed.v/ result_writeback.v completely unmodified, with its own private weight-fetch removed) and systolic_group.v (one real layer_prefetch_ ctrl.v+layer_weight_buffer.v+weight_tile_gather.v shared by 4x packed_pe.v via a real, barrier-synchronized broadcast bus). Real design choice confirmed with the user before writing any RTL (AskUserQuestion, concrete topology preview): shared-weight broadcast, not a literal PE-to-PE systolic shift register -- achieves the real, quantified rationale (4x reduction in redundant weight-fetch DDR3 traffic per group of 4 PEs) with much lower real risk than genuine inter-PE pipeline fill/drain. The real new design is the barrier: each PE's own tcnt is the join key against the group's broadcast tcnt, self-synchronizing regardless of which PE is momentarily ahead/behind (e.g. a real DDR3 row-switch stall on one PE's own activation fetch). Found and fixed a real bug during verification (not by inspection): the first full test run reported every result as undefined despite every control-flow signal tracing correctly -- root-caused via real signal tracing down to a 5-way test arbiter bus mis-sliced at the wrong slot offset (single-bit handshake buses happened to use a correct range and masked it from the control-flow trace; only the wide, byte-offset buses were wrong). Verified in isolation (tb_systolic_group.v, real Icarus xsim, real sdram_arbiter_n.v generalized to NUM_REQ=5 with zero changes): 8/8 PASS across 2 consecutive group jobs (exercising the barrier's own per-job reset path, not just cold start). Deliberately scoped to the isolated mechanism only, per this project's "one variable at a time" discipline -- Director/SPI job dispatch for group jobs, a real N=16 top-level, and real P&R are real, disclosed next steps, not done here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUG92aM9m68TRc4rG55BcC
This commit is contained in:
Claude Sonnet 5
@@ -5948,3 +5948,134 @@ design confirmed by the user -- NOT a literal PE-to-PE systolic shift
|
||||
register -- verified in isolation before any Director/SPI-protocol
|
||||
integration, matching this project's own "one variable at a time"
|
||||
discipline).
|
||||
|
||||
EXP-0089 -- real, isolated first step of the 4x4 hybrid systolic
|
||||
architecture: shared-weight-broadcast group, built and verified
|
||||
(2026-09-20, user's own explicit reprioritization: "Fai la parte
|
||||
realmente mancante prima, il RESULT-WRITEBACK e poi implementa la 4x4
|
||||
sistolica"; real A/B design decision confirmed by the user before any
|
||||
RTL was written: shared-weight BROADCAST, not a literal PE-to-PE
|
||||
systolic shift register)
|
||||
|
||||
CONTEXT: docs/ARCHITECTURE_ANALYSIS.md S5.6 captured this direction as
|
||||
purely exploratory ("nothing in this subsection is implemented"),
|
||||
explicitly flagging "intra-chain dataflow RTL... is a real, new design,
|
||||
not a trivial extension" as an open question. Before writing any RTL,
|
||||
the user was asked (AskUserQuestion, with a concrete side-by-side
|
||||
preview of both real candidate topologies) to resolve that open
|
||||
question directly: (A) one shared weight fetch per group of 4 PEs,
|
||||
broadcast to all 4, each PE computing its own independent activation
|
||||
positions in parallel -- achieves the doc's own quantified rationale
|
||||
(4x reduction in redundant weight-fetch DDR3 traffic per group) with
|
||||
much lower real risk; or (B) a literal PE-to-PE systolic shift
|
||||
register (weight physically translating PE by PE, real pipeline
|
||||
fill/drain at chain boundaries). User confirmed (A), explicitly noting
|
||||
(B) can be revisited later if real data justifies it.
|
||||
|
||||
DESIGN: two new modules.
|
||||
- `packed_pe.v`: packed_slot.v's own compute+activation-fetch+
|
||||
writeback subsystem (ddr_prefetch_mgr.v, neural_processor_packed.v,
|
||||
result_writeback.v -- all THREE reused completely unmodified),
|
||||
with packed_slot.v's own private layer_prefetch_ctrl.v/layer_
|
||||
weight_buffer.v/weight_tile_gather.v REMOVED -- weight tile data
|
||||
arrives via a real, group-broadcast interface instead
|
||||
(group_tcnt/group_tile_data/group_tile_valid).
|
||||
- `systolic_group.v`: ONE real layer_prefetch_ctrl.v + layer_weight_
|
||||
buffer.v + weight_tile_gather.v (group-level, shared by reference,
|
||||
not duplicated -- identical instances to what packed_slot.v already
|
||||
owned per-slot), driving 4x packed_pe.v.
|
||||
|
||||
REAL SYNCHRONIZATION (the actual new design, not a trivial extension --
|
||||
confirmed by building and testing it, not asserted): each packed_pe.v's
|
||||
own `tcnt` IS the join key -- a PE in S_TILEWAIT waits for
|
||||
`group_tile_valid && (group_tcnt == tcnt)` before consuming, a real,
|
||||
self-synchronizing comparison immune to which PE happens to be
|
||||
momentarily ahead or behind (e.g. one PE's own activation fetch hit a
|
||||
real DDR3 row switch the others didn't). systolic_group.v's own barrier
|
||||
extends this project's established 2-source "S_TILEWAIT join"
|
||||
discipline to 4 independent sources: it only advances to the next
|
||||
tile's weight fetch once ALL 4 PEs have ack'd the current one
|
||||
(`pe_acked`, individually latched, using a `pe_acked_next` combinational
|
||||
fold-in to detect same-cycle acks without an extra latency cycle). A
|
||||
similar accumulator (`pe_done_latch`) tracks all 4 PEs' own job_done
|
||||
pulses, deliberately made UNCONDITIONAL (not state-gated) to correctly
|
||||
catch a real race: a fast PE's own job_done can pulse the SAME cycle
|
||||
the group's own barrier clears the LAST tile (i.e. the same cycle the
|
||||
group transitions toward waiting for completion) -- a state-gated
|
||||
accumulator would have silently dropped that pulse.
|
||||
|
||||
REAL BUGS FOUND AND FIXED DURING DESIGN (before compiling, via re-
|
||||
deriving against already-proven code, not guessed) AND DURING
|
||||
VERIFICATION (via real signal tracing, not by inspection):
|
||||
1. First draft of packed_pe.v omitted the neural_processor_packed.v
|
||||
job_valid/job_ready handshake's own real gating (packed_slot.v's
|
||||
own proven S_JOBSTART state) -- would have raced operand_valid
|
||||
against job acceptance. Caught by re-deriving against packed_
|
||||
slot.v's own real sequencing before ever compiling, not by
|
||||
simulation.
|
||||
2. systolic_group.v's first draft indexed an expression directly
|
||||
(`(tcnt + 16'd1)[BUFADDRW-1:0]`) -- not valid plain Verilog syntax
|
||||
(only nets/regs, not arbitrary expressions, can be part-selected).
|
||||
Fixed with an intermediate `tcnt_next` wire.
|
||||
3. A duplicate `pe_acked_pulse` wire declaration (real copy-paste
|
||||
leftover from restructuring the generate block).
|
||||
4. THE real bug, found via real hierarchical signal tracing after the
|
||||
first full test run reported EVERY result as `x` (undefined) even
|
||||
though `job_done` fired and every control-flow signal traced
|
||||
correctly (barrier, tcnt, state transitions all real and correct)
|
||||
-- proving the bug was in DATA, not control flow. Traced down
|
||||
through ddr_prefetch_mgr.v's own real, already-proven internals
|
||||
(confirmed innocent) to the actual root cause: the ISOLATED
|
||||
testbench's own 5-way arbiter wiring (1 group weight-fetch slot +
|
||||
4 PE slots) sliced the flattened `pe_ctrl_addr/wdata/wmask/rdata`
|
||||
buses starting at the WRONG offset (`4*WIDTH`, i.e. slot 4, instead
|
||||
of `1*WIDTH`, slot 1 -- slot 0 is the group's own weight-fetch).
|
||||
The single-bit handshake buses (req/grant/ready/busy) happened to
|
||||
use a correct `[4:1]` bit range and so masked the bug from the
|
||||
control-flow trace entirely -- only the wide, byte-offset-computed
|
||||
buses were wrong, which is exactly why data silently corrupted
|
||||
while every handshake still looked correct. A real, honest example
|
||||
of why "the control flow works" is not suffient evidence that "the
|
||||
data path works" -- this project's own standing discipline (root-
|
||||
cause via real signal tracing, verify don't assume) is what caught
|
||||
it, not luck.
|
||||
|
||||
VERIFICATION: new `tb_systolic_group.v`, real Icarus xsim, real
|
||||
`burst_mem_model32.v` backend shared via a real `sdram_arbiter_n.v`
|
||||
(NUM_REQ=5, confirmed its own NUM_REQ parameter already generalizes to
|
||||
this shape with zero changes, per this project's own prior note).
|
||||
Preloads 2 real layers' worth of weights + 16 real activation
|
||||
positions, runs TWO consecutive group jobs (deliberately, to exercise
|
||||
the barrier's own per-job reset path, not just a single cold-start).
|
||||
**8/8 PASS (2 jobs x 4 PEs), 0 errors**, real, independently-reproduced
|
||||
golden model (same formula convention as every other v3 testbench).
|
||||
This verifies the shared-weight-broadcast mechanism itself is
|
||||
functionally correct -- it does NOT re-verify result_writeback.v's own
|
||||
real DDR3 read-after-write correctness, since packed_pe.v reuses that
|
||||
module completely unmodified and it already has its own real
|
||||
read-after-write verification (EXP-0088, tb_packed_slot.v).
|
||||
|
||||
DECISION: real, verified first step. This is deliberately SCOPED to the
|
||||
isolated group mechanism only, per this project's own "one variable at
|
||||
a time" discipline (same precedent as act_tile_fetch.v EXP-0079 or
|
||||
ddr_prefetch_mgr.v EXP-0083, both built and verified standalone well
|
||||
before system-level integration). NOT yet integrated: Director-level
|
||||
job dispatch (neural_director_packed.v only knows how to dispatch
|
||||
2-position jobs to flat slots, not 4-position jobs to a group),
|
||||
SPI/WRITE_JOB protocol support for group-level job submission, a real
|
||||
N=16 (4 groups x 4 PEs) top-level module, and real P&R signoff for any
|
||||
of this. These are real, disclosed, deliberately deferred next steps,
|
||||
not overlooked.
|
||||
|
||||
next_action: (1) real P&R for systolic_group.v in isolation (out-of-
|
||||
context first, matching this project's own precedent for brand-new
|
||||
modules before real system integration) to get a first real sense of
|
||||
its area/timing cost. (2) Design the real Director/SPI protocol
|
||||
extension needed for group-level job dispatch (a real, disclosed,
|
||||
non-trivial addition -- WRITE_JOB's current payload only carries one
|
||||
job's worth of x_base/w_base/result_addr/node_id, not 4). (3) Build a
|
||||
real N=16 top-level (4x systolic_group.v) once (1) and (2) are real and
|
||||
verified, with its own real P&R signoff. (4) Revisit the flat N=2/4/8/16
|
||||
core-count scaling tests (deferred by the user's own explicit
|
||||
reprioritization this session) once there's a real basis for comparing
|
||||
flat vs. grouped scaling with real numbers from both.
|
||||
|
||||
@@ -0,0 +1,322 @@
|
||||
`timescale 1ns/1ps
|
||||
|
||||
// ============================================================
|
||||
// V3 -- packed_pe.v: real weight-sharing PE for a systolic_group.v
|
||||
// (docs/ARCHITECTURE_ANALYSIS.md S5.6, EXP-0089), derived directly from
|
||||
// packed_slot.v -- SAME activation-fetch (ddr_prefetch_mgr.v), compute
|
||||
// (neural_processor_packed.v), and result-writeback (result_writeback.v)
|
||||
// subsystems, unmodified. The ONLY real difference: this module does
|
||||
// NOT own a layer_prefetch_ctrl.v/layer_weight_buffer.v/weight_tile_
|
||||
// gather.v of its own -- weight tile data is received via a real,
|
||||
// broadcast, GROUP-shared interface instead (group_tcnt/group_tile_
|
||||
// data/group_tile_valid), since the whole point of grouping 4 PEs is
|
||||
// ONE real weight fetch shared by all 4 (docs' own quantified
|
||||
// rationale: DDR3 requester count reduction), not 4 independent ones.
|
||||
//
|
||||
// REAL SYNCHRONIZATION (the part that needed real design, not a
|
||||
// trivial extension -- see docs S5.6's own "not a trivial extension"
|
||||
// disclosure): this PE's own `tcnt` IS the join key. It waits in
|
||||
// S_TILEWAIT for `group_tile_valid && (group_tcnt == tcnt)` before
|
||||
// consuming -- a real, self-synchronizing comparison, not a bare
|
||||
// pulse/level race. A PE that's briefly slower than its groupmates
|
||||
// (e.g. its own activation fetch hit a real DDR3 row switch the others
|
||||
// didn't) simply keeps waiting; when it finally reaches S_TILEWAIT for
|
||||
// its own `tcnt`, the comparison is either already true (if the group
|
||||
// had to wait for THIS PE, i.e. this PE IS the slow one) or becomes
|
||||
// true the moment the group's own barrier (systolic_group.v, all 4
|
||||
// pe_tile_ack seen) lets it advance -- correct regardless of which PE
|
||||
// is momentarily ahead or behind, no risk of double-consuming or
|
||||
// skipping a tile.
|
||||
//
|
||||
// group_n_tiles (broadcast, held stable for the whole job -- all 4 PEs
|
||||
// in a group process the SAME layer, same real weight-stationary
|
||||
// premise this module's own name comes from) replaces packed_slot.v's
|
||||
// own per-instance n_tiles input.
|
||||
// ============================================================
|
||||
module packed_pe #(
|
||||
parameter DATA_WIDTH = 8,
|
||||
parameter P_IN = 8,
|
||||
parameter ACC_WIDTH = 32,
|
||||
parameter BURST_LEN = 8,
|
||||
parameter ADDR_WIDTH = 26
|
||||
)(
|
||||
input wire clk,
|
||||
input wire rst,
|
||||
|
||||
// ---- own per-PE job trigger: x_base_a/b (this PE's own two
|
||||
// activation positions), result_addr_a/b, node_id_a/b -- w_base and
|
||||
// n_tiles are NOT here, they're group-broadcast (see group_n_tiles
|
||||
// below; w_base never reaches this module at all, only the group
|
||||
// controller needs it). ----
|
||||
input wire job_start,
|
||||
input wire [ADDR_WIDTH-1:0] x_base_a,
|
||||
input wire [ADDR_WIDTH-1:0] x_base_b,
|
||||
input wire [ADDR_WIDTH-1:0] result_addr_a,
|
||||
input wire [ADDR_WIDTH-1:0] result_addr_b,
|
||||
input wire [15:0] node_id_a,
|
||||
input wire [15:0] node_id_b,
|
||||
output reg job_done, // one-cycle pulse
|
||||
|
||||
output reg signed [DATA_WIDTH-1:0] result_data_a,
|
||||
output reg signed [DATA_WIDTH-1:0] result_data_b,
|
||||
output reg [15:0] result_node_id_a,
|
||||
output reg [15:0] result_node_id_b,
|
||||
output reg [ADDR_WIDTH-1:0] result_addr_a_out,
|
||||
output reg [ADDR_WIDTH-1:0] result_addr_b_out,
|
||||
|
||||
// ---- group-broadcast weight interface (real, level-held-until-
|
||||
// consumed, same discipline ddr_prefetch_mgr.v's own tile_valid
|
||||
// already established) ----
|
||||
input wire [15:0] group_n_tiles,
|
||||
input wire [15:0] group_tcnt,
|
||||
input wire [DATA_WIDTH*P_IN-1:0] group_tile_data,
|
||||
input wire group_tile_valid,
|
||||
output reg pe_tile_ack, // one-shot pulse
|
||||
|
||||
// ---- own activation-fetch + result-writeback arbiter port (still
|
||||
// one per PE -- activation data is NOT shared across PEs, each PE
|
||||
// computes different positions) ----
|
||||
output wire mem_active,
|
||||
input wire mem_grant,
|
||||
|
||||
output wire ctrl_req,
|
||||
output wire ctrl_wr,
|
||||
output wire [ADDR_WIDTH-2:0] ctrl_addr,
|
||||
output wire [32*BURST_LEN-1:0] ctrl_wdata,
|
||||
output wire [4*BURST_LEN-1:0] ctrl_wmask,
|
||||
input wire [32*BURST_LEN-1:0] ctrl_rdata,
|
||||
input wire ctrl_ready,
|
||||
input wire ctrl_busy
|
||||
);
|
||||
localparam S_IDLE = 3'd0,
|
||||
S_JOBSTART = 3'd1,
|
||||
S_TILEWAIT = 3'd2,
|
||||
S_OPERAND = 3'd3,
|
||||
S_RESULT = 3'd4,
|
||||
S_WRITEBACK = 3'd5;
|
||||
|
||||
reg [2:0] state;
|
||||
reg [ADDR_WIDTH-1:0] x_base_a_lat, x_base_b_lat;
|
||||
reg [15:0] n_tiles_lat;
|
||||
reg [ADDR_WIDTH-1:0] result_addr_a_lat, result_addr_b_lat;
|
||||
reg [15:0] node_id_a_lat, node_id_b_lat;
|
||||
reg [15:0] tcnt;
|
||||
|
||||
// ---- ddr_prefetch_mgr.v: own activation-tile look-ahead, exactly
|
||||
// as packed_slot.v already uses it ----
|
||||
reg ddrpf_job_start;
|
||||
wire ddrpf_tile_valid;
|
||||
reg ddrpf_tile_consume;
|
||||
wire signed [DATA_WIDTH*P_IN-1:0] act_data_a_w, act_data_b_w;
|
||||
wire act_mem_active;
|
||||
|
||||
wire act_ctrl_req, act_ctrl_wr;
|
||||
wire [ADDR_WIDTH-2:0] act_ctrl_addr;
|
||||
wire [32*BURST_LEN-1:0] act_ctrl_wdata;
|
||||
wire [4*BURST_LEN-1:0] act_ctrl_wmask;
|
||||
|
||||
ddr_prefetch_mgr #(
|
||||
.DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .BURST_LEN(BURST_LEN), .ADDR_WIDTH(ADDR_WIDTH-1)
|
||||
) u_ddrpf (
|
||||
.clk(clk), .rst(rst),
|
||||
.job_start(ddrpf_job_start),
|
||||
.base_a(x_base_a_lat[ADDR_WIDTH-2:0]), .base_b(x_base_b_lat[ADDR_WIDTH-2:0]),
|
||||
.n_tiles(n_tiles_lat),
|
||||
.tile_valid(ddrpf_tile_valid), .data_a(act_data_a_w), .data_b(act_data_b_w),
|
||||
.tile_consume(ddrpf_tile_consume),
|
||||
.mem_active(act_mem_active), .mem_grant(mem_grant),
|
||||
.ctrl_req(act_ctrl_req), .ctrl_wr(act_ctrl_wr), .ctrl_addr(act_ctrl_addr),
|
||||
.ctrl_wdata(act_ctrl_wdata), .ctrl_wmask(act_ctrl_wmask),
|
||||
.ctrl_rdata(ctrl_rdata), .ctrl_ready(ctrl_ready), .ctrl_busy(ctrl_busy)
|
||||
);
|
||||
|
||||
// ---- result_writeback.v: own result, own DDR3 write, exactly as
|
||||
// packed_slot.v already uses it ----
|
||||
reg wb_start;
|
||||
wire wb_busy, wb_done;
|
||||
wire wb_mem_active;
|
||||
|
||||
wire wb_ctrl_req, wb_ctrl_wr;
|
||||
wire [ADDR_WIDTH-2:0] wb_ctrl_addr;
|
||||
wire [32*BURST_LEN-1:0] wb_ctrl_wdata;
|
||||
wire [4*BURST_LEN-1:0] wb_ctrl_wmask;
|
||||
|
||||
result_writeback #(
|
||||
.BURST_LEN(BURST_LEN), .DATA_WIDTH(DATA_WIDTH),
|
||||
.JOB_ADDR_WIDTH(ADDR_WIDTH), .ADDR_WIDTH(ADDR_WIDTH-1)
|
||||
) u_wb (
|
||||
.clk(clk), .rst(rst),
|
||||
.start(wb_start),
|
||||
.result_addr_a(result_addr_a_lat), .result_addr_b(result_addr_b_lat),
|
||||
.result_data_a(result_data_a), .result_data_b(result_data_b),
|
||||
.result_node_id_a(result_node_id_a), .result_node_id_b(result_node_id_b),
|
||||
.busy(wb_busy), .done(wb_done),
|
||||
.mem_active(wb_mem_active), .mem_grant(mem_grant),
|
||||
.ctrl_req(wb_ctrl_req), .ctrl_wr(wb_ctrl_wr), .ctrl_addr(wb_ctrl_addr),
|
||||
.ctrl_wdata(wb_ctrl_wdata), .ctrl_wmask(wb_ctrl_wmask),
|
||||
.ctrl_rdata(ctrl_rdata), .ctrl_ready(ctrl_ready), .ctrl_busy(ctrl_busy)
|
||||
);
|
||||
|
||||
// mutually exclusive by FSM construction (activation fetch always
|
||||
// finishes each tile's own consume before writeback ever starts,
|
||||
// and writeback only starts once the whole tile loop is done) --
|
||||
// 2-way mux, one fewer branch than packed_slot.v's own 3-way (no
|
||||
// local weight-fetch mux here, weight is group-broadcast).
|
||||
assign ctrl_req = act_mem_active ? act_ctrl_req : wb_ctrl_req;
|
||||
assign ctrl_wr = act_mem_active ? act_ctrl_wr : wb_ctrl_wr;
|
||||
assign ctrl_addr = act_mem_active ? act_ctrl_addr : wb_ctrl_addr;
|
||||
assign ctrl_wdata = act_mem_active ? act_ctrl_wdata : wb_ctrl_wdata;
|
||||
assign ctrl_wmask = act_mem_active ? act_ctrl_wmask : wb_ctrl_wmask;
|
||||
|
||||
assign mem_active = act_mem_active || wb_mem_active;
|
||||
|
||||
// ---- neural_processor_packed.v (unmodified, same as packed_slot.v) ----
|
||||
reg job_valid_np;
|
||||
wire job_ready_np;
|
||||
reg [1:0] job_activation;
|
||||
reg signed [DATA_WIDTH-1:0] job_bias;
|
||||
|
||||
reg operand_valid;
|
||||
wire operand_ready;
|
||||
reg signed [DATA_WIDTH*P_IN-1:0] input_data_a_r, input_data_b_r;
|
||||
reg [DATA_WIDTH*P_IN-1:0] weight_data_r;
|
||||
reg tile_last;
|
||||
|
||||
wire result_valid_np;
|
||||
reg result_ready;
|
||||
wire signed [DATA_WIDTH-1:0] result_data_a_np, result_data_b_np;
|
||||
wire [15:0] result_node_id_a_np, result_node_id_b_np;
|
||||
wire [3:0] np_state;
|
||||
wire np_error;
|
||||
|
||||
neural_processor_packed #(
|
||||
.DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .ACC_WIDTH(ACC_WIDTH)
|
||||
) u_np (
|
||||
.clk(clk), .rst(rst),
|
||||
.job_valid(job_valid_np), .job_ready(job_ready_np),
|
||||
.job_node_id_a(node_id_a_lat), .job_node_id_b(node_id_b_lat),
|
||||
.job_bias(job_bias), .job_activation(job_activation),
|
||||
.operand_valid(operand_valid), .operand_ready(operand_ready),
|
||||
.input_data_a(input_data_a_r), .input_data_b(input_data_b_r),
|
||||
.weight_data(weight_data_r), .tile_last(tile_last),
|
||||
.result_valid(result_valid_np), .result_ready(result_ready),
|
||||
.result_data_a(result_data_a_np), .result_data_b(result_data_b_np),
|
||||
.result_node_id_a(result_node_id_a_np), .result_node_id_b(result_node_id_b_np),
|
||||
.np_state(np_state), .np_error(np_error)
|
||||
);
|
||||
|
||||
localparam ACT_RELU = 2'd1;
|
||||
|
||||
always @(posedge clk) begin
|
||||
if (rst) begin
|
||||
state <= S_IDLE;
|
||||
job_done <= 1'b0;
|
||||
ddrpf_job_start <= 1'b0;
|
||||
ddrpf_tile_consume <= 1'b0;
|
||||
job_valid_np <= 1'b0;
|
||||
operand_valid<= 1'b0;
|
||||
tile_last <= 1'b0;
|
||||
result_ready <= 1'b0;
|
||||
wb_start <= 1'b0;
|
||||
pe_tile_ack <= 1'b0;
|
||||
job_bias <= {DATA_WIDTH{1'b0}};
|
||||
job_activation <= ACT_RELU;
|
||||
tcnt <= 16'd0;
|
||||
end else begin
|
||||
job_done <= 1'b0;
|
||||
ddrpf_job_start <= 1'b0;
|
||||
ddrpf_tile_consume <= 1'b0;
|
||||
wb_start <= 1'b0;
|
||||
pe_tile_ack <= 1'b0;
|
||||
|
||||
case (state)
|
||||
S_IDLE: begin
|
||||
if (job_start) begin
|
||||
x_base_a_lat <= x_base_a;
|
||||
x_base_b_lat <= x_base_b;
|
||||
n_tiles_lat <= group_n_tiles;
|
||||
result_addr_a_lat <= result_addr_a;
|
||||
result_addr_b_lat <= result_addr_b;
|
||||
node_id_a_lat <= node_id_a;
|
||||
node_id_b_lat <= node_id_b;
|
||||
job_bias <= {DATA_WIDTH{1'b0}};
|
||||
job_activation <= ACT_RELU;
|
||||
tcnt <= 16'd0;
|
||||
job_valid_np <= 1'b1;
|
||||
state <= S_JOBSTART;
|
||||
end
|
||||
end
|
||||
|
||||
// real, proven sequencing (matches packed_slot.v's own
|
||||
// S_JOBSTART exactly): only kick off the activation
|
||||
// look-ahead loop and enter the tile-consumption join
|
||||
// AFTER neural_processor_packed.v has actually accepted
|
||||
// the job -- issuing operand_valid before job_ready_np
|
||||
// would race its own internal job-acceptance state.
|
||||
S_JOBSTART: begin
|
||||
if (job_valid_np && job_ready_np) begin
|
||||
job_valid_np <= 1'b0;
|
||||
ddrpf_job_start <= 1'b1; // one-shot: kicks off this PE's
|
||||
// own activation look-ahead loop
|
||||
state <= S_TILEWAIT;
|
||||
end
|
||||
end
|
||||
|
||||
// real join: group's broadcast weight tile (matched by
|
||||
// tcnt, see header) AND this PE's own activation fetch
|
||||
// -- same 2-source join shape packed_slot.v's own
|
||||
// S_TILEWAIT already uses, just with group_tile_valid+
|
||||
// tcnt-match replacing the local tile_valid.
|
||||
S_TILEWAIT: begin
|
||||
if (group_tile_valid && (group_tcnt == tcnt) && ddrpf_tile_valid) begin
|
||||
weight_data_r <= group_tile_data;
|
||||
input_data_a_r <= act_data_a_w;
|
||||
input_data_b_r <= act_data_b_w;
|
||||
pe_tile_ack <= 1'b1;
|
||||
ddrpf_tile_consume <= 1'b1;
|
||||
tile_last <= (tcnt == n_tiles_lat - 16'd1);
|
||||
operand_valid <= 1'b1;
|
||||
state <= S_OPERAND;
|
||||
end
|
||||
end
|
||||
|
||||
S_OPERAND: begin
|
||||
if (operand_valid && operand_ready) begin
|
||||
operand_valid <= 1'b0;
|
||||
tile_last <= 1'b0;
|
||||
if (tcnt == n_tiles_lat - 16'd1) begin
|
||||
result_ready <= 1'b1;
|
||||
state <= S_RESULT;
|
||||
end else begin
|
||||
tcnt <= tcnt + 16'd1;
|
||||
state <= S_TILEWAIT;
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
S_RESULT: begin
|
||||
if (result_valid_np) begin
|
||||
result_data_a <= result_data_a_np;
|
||||
result_data_b <= result_data_b_np;
|
||||
result_node_id_a <= result_node_id_a_np;
|
||||
result_node_id_b <= result_node_id_b_np;
|
||||
result_addr_a_out <= result_addr_a_lat;
|
||||
result_addr_b_out <= result_addr_b_lat;
|
||||
result_ready <= 1'b0;
|
||||
wb_start <= 1'b1;
|
||||
state <= S_WRITEBACK;
|
||||
end
|
||||
end
|
||||
|
||||
S_WRITEBACK: begin
|
||||
if (wb_done) begin
|
||||
job_done <= 1'b1;
|
||||
state <= S_IDLE;
|
||||
end
|
||||
end
|
||||
|
||||
default: state <= S_IDLE;
|
||||
endcase
|
||||
end
|
||||
end
|
||||
endmodule
|
||||
@@ -0,0 +1,323 @@
|
||||
`timescale 1ns/1ps
|
||||
|
||||
// ============================================================
|
||||
// V3 -- systolic_group.v: real group controller for the "4 groups x
|
||||
// 4-PE weight-stationary chains" hybrid scaling direction
|
||||
// (docs/ARCHITECTURE_ANALYSIS.md S5.6, EXP-0089). User-confirmed real
|
||||
// design choice (2026-09-20, explicit A/B decision): SHARED-WEIGHT
|
||||
// BROADCAST, not a literal PE-to-PE systolic shift register -- one
|
||||
// real weight fetch per group of 4 PEs (instead of 4 independent
|
||||
// fetches), each PE then computes its own, independent activation
|
||||
// positions in parallel. This achieves the doc's own real, quantified
|
||||
// rationale (reduce redundant weight-fetch DDR3 traffic 4x per group)
|
||||
// with far less real risk/complexity than a literal systolic pipeline
|
||||
// -- no inter-PE result propagation, no pipeline fill/drain at chain
|
||||
// boundaries, both of which the doc's own S5.6 explicitly flagged as
|
||||
// "a real, new design, not a trivial extension".
|
||||
//
|
||||
// STRUCTURE: ONE real layer_prefetch_ctrl.v + layer_weight_buffer.v +
|
||||
// weight_tile_gather.v (unmodified, identical instances to what
|
||||
// packed_slot.v already owns per-slot -- just now group-level, shared
|
||||
// by reference not duplicated), driving 4x packed_pe.v (packed_slot.v
|
||||
// minus its own weight-fetch, see packed_pe.v's own header) via a
|
||||
// real, level-held, tcnt-tagged broadcast bus.
|
||||
//
|
||||
// REAL BARRIER SYNCHRONIZATION (the actual new design, not asserted
|
||||
// correct without real verification -- see tb_systolic_group.v): the
|
||||
// group only advances to tile N+1's weight fetch once ALL 4 PEs have
|
||||
// ack'd tile N (`pe_tile_ack[i]`, individually latched since PEs may
|
||||
// consume at different real cycles -- e.g. one PE's own activation
|
||||
// fetch hit a real DDR3 row switch the others didn't). This is a real,
|
||||
// necessary extension of the "S_TILEWAIT join" discipline every other
|
||||
// module in this project already uses for 2-source joins (weight +
|
||||
// activation) -- here it's a 4-way join (one group weight source, 4
|
||||
// independent PE acks), same underlying principle: never advance past
|
||||
// a shared resource until every real consumer has confirmed it read
|
||||
// what it needed.
|
||||
// ============================================================
|
||||
module systolic_group #(
|
||||
parameter DATA_WIDTH = 8,
|
||||
parameter P_IN = 8,
|
||||
parameter ACC_WIDTH = 32,
|
||||
parameter BURST_LEN = 8,
|
||||
parameter ADDR_WIDTH = 26,
|
||||
parameter LAYER_BYTES = 128,
|
||||
parameter BUFADDRW = $clog2(LAYER_BYTES)
|
||||
)(
|
||||
input wire clk,
|
||||
input wire rst,
|
||||
|
||||
// ---- group-level job dispatch: ONE shared w_base/n_tiles (all 4
|
||||
// PEs process the SAME layer, weight-stationary), 4x independent
|
||||
// per-PE x_base_a/b/result_addr_a/b/node_id_a/b ----
|
||||
input wire job_start,
|
||||
input wire [ADDR_WIDTH-1:0] w_base,
|
||||
input wire [15:0] n_tiles,
|
||||
input wire [4*ADDR_WIDTH-1:0] pe_x_base_a, pe_x_base_b,
|
||||
input wire [4*ADDR_WIDTH-1:0] pe_result_addr_a, pe_result_addr_b,
|
||||
input wire [4*16-1:0] pe_node_id_a, pe_node_id_b,
|
||||
output reg job_done, // one-cycle pulse, ALL 4 PEs done
|
||||
|
||||
output wire [4*DATA_WIDTH-1:0] pe_result_data_a, pe_result_data_b,
|
||||
output wire [4*16-1:0] pe_result_node_id_a, pe_result_node_id_b,
|
||||
output wire [4*ADDR_WIDTH-1:0] pe_result_addr_a_out, pe_result_addr_b_out,
|
||||
|
||||
// ---- group's own single arbiter port, for the shared weight
|
||||
// fetch only -- each of the 4 PEs still owns ITS OWN separate
|
||||
// arbiter port for activation-fetch+writeback (see top-level
|
||||
// integration; not this module's own concern) ----
|
||||
output wire mem_active,
|
||||
input wire mem_grant,
|
||||
output wire ctrl_req,
|
||||
output wire ctrl_wr,
|
||||
output wire [ADDR_WIDTH-2:0] ctrl_addr,
|
||||
output wire [32*BURST_LEN-1:0] ctrl_wdata,
|
||||
output wire [4*BURST_LEN-1:0] ctrl_wmask,
|
||||
input wire [32*BURST_LEN-1:0] ctrl_rdata,
|
||||
input wire ctrl_ready,
|
||||
input wire ctrl_busy,
|
||||
|
||||
// ---- 4 independent per-PE arbiter ports (activation-fetch +
|
||||
// result-writeback, NOT shared -- flattened NUM_PE*width buses) ----
|
||||
output wire [3:0] pe_mem_active,
|
||||
input wire [3:0] pe_mem_grant,
|
||||
output wire [3:0] pe_ctrl_req,
|
||||
output wire [3:0] pe_ctrl_wr,
|
||||
output wire [4*(ADDR_WIDTH-1)-1:0] pe_ctrl_addr,
|
||||
output wire [4*32*BURST_LEN-1:0] pe_ctrl_wdata,
|
||||
output wire [4*4*BURST_LEN-1:0] pe_ctrl_wmask,
|
||||
input wire [4*32*BURST_LEN-1:0] pe_ctrl_rdata,
|
||||
input wire [3:0] pe_ctrl_ready,
|
||||
input wire [3:0] pe_ctrl_busy
|
||||
);
|
||||
localparam S_IDLE = 4'd0,
|
||||
S_MEMWAIT = 4'd1,
|
||||
S_PREFETCH = 4'd2,
|
||||
S_SWAP = 4'd3,
|
||||
S_PEJOBSTART = 4'd4,
|
||||
S_TILELOOP = 4'd5,
|
||||
S_WAITDONE = 4'd6;
|
||||
|
||||
reg [3:0] state;
|
||||
reg [ADDR_WIDTH-1:0] w_base_lat;
|
||||
reg [15:0] n_tiles_lat;
|
||||
reg [15:0] tcnt;
|
||||
|
||||
// ---- shared weight fetch: layer_prefetch_ctrl.v -> layer_weight_
|
||||
// buffer.v -> weight_tile_gather.v, IDENTICAL instances to what
|
||||
// packed_slot.v already owns per-slot, just group-level now ----
|
||||
reg pf_start;
|
||||
wire pf_busy, pf_done;
|
||||
wire pf_fill_we;
|
||||
wire [BUFADDRW-1:0] pf_fill_addr;
|
||||
wire [DATA_WIDTH-1:0] pf_fill_data;
|
||||
reg consume_done;
|
||||
|
||||
layer_prefetch_ctrl #(
|
||||
.DATA_WIDTH(DATA_WIDTH), .LAYER_BYTES(LAYER_BYTES), .BURST_LEN(BURST_LEN), .ADDR_WIDTH(ADDR_WIDTH-1)
|
||||
) u_pf (
|
||||
.clk(clk), .rst(rst),
|
||||
.start(pf_start), .layer_base(w_base_lat[ADDR_WIDTH-2:0]), .busy(pf_busy), .done(pf_done),
|
||||
.fill_we(pf_fill_we), .fill_addr(pf_fill_addr), .fill_data(pf_fill_data),
|
||||
.ctrl_req(ctrl_req), .ctrl_wr(ctrl_wr), .ctrl_addr(ctrl_addr),
|
||||
.ctrl_wdata(ctrl_wdata), .ctrl_wmask(ctrl_wmask),
|
||||
.ctrl_rdata(ctrl_rdata), .ctrl_ready(ctrl_ready), .ctrl_busy(ctrl_busy)
|
||||
);
|
||||
|
||||
assign mem_active = (state == S_MEMWAIT) || (state == S_PREFETCH);
|
||||
|
||||
wire [BUFADDRW-1:0] lwb_rd_addr;
|
||||
wire [DATA_WIDTH-1:0] lwb_rd_data;
|
||||
|
||||
layer_weight_buffer #(.DATA_WIDTH(DATA_WIDTH), .LAYER_DEPTH(LAYER_BYTES)) u_lwb (
|
||||
.clk(clk), .rst(rst),
|
||||
.fill_we(pf_fill_we), .fill_addr(pf_fill_addr), .fill_data(pf_fill_data), .fill_done(pf_done),
|
||||
.rd_addr(lwb_rd_addr), .rd_data(lwb_rd_data), .consume_done(consume_done),
|
||||
.active_sel(), .swapped()
|
||||
);
|
||||
|
||||
reg tile_req;
|
||||
reg [BUFADDRW-1:0] tile_base;
|
||||
wire tile_valid;
|
||||
wire [DATA_WIDTH*P_IN-1:0] tile_data;
|
||||
wire [15:0] tcnt_next = tcnt + 16'd1;
|
||||
|
||||
weight_tile_gather #(
|
||||
.DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .BUFADDRW(BUFADDRW)
|
||||
) u_gather (
|
||||
.clk(clk), .rst(rst),
|
||||
.tile_req(tile_req), .tile_base(tile_base),
|
||||
.tile_valid(tile_valid), .tile_data(tile_data),
|
||||
.rd_addr(lwb_rd_addr), .rd_data(lwb_rd_data)
|
||||
);
|
||||
|
||||
// ---- real broadcast bus to all 4 PEs (level-held, tcnt-tagged --
|
||||
// see packed_pe.v's own header for the real join discipline) ----
|
||||
reg group_tile_valid;
|
||||
reg [DATA_WIDTH*P_IN-1:0] group_tile_data_r;
|
||||
reg [3:0] pe_acked; // per-PE ack latch, cleared each tile
|
||||
|
||||
wire [3:0] pe_job_start_w = {4{(state == S_PEJOBSTART)}};
|
||||
wire [3:0] pe_job_done_w;
|
||||
wire [3:0] pe_acked_pulse;
|
||||
|
||||
genvar gi;
|
||||
generate
|
||||
for (gi = 0; gi < 4; gi = gi + 1) begin : GEN_PE
|
||||
packed_pe #(
|
||||
.DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .ACC_WIDTH(ACC_WIDTH),
|
||||
.BURST_LEN(BURST_LEN), .ADDR_WIDTH(ADDR_WIDTH)
|
||||
) u_pe (
|
||||
.clk(clk), .rst(rst),
|
||||
.job_start(pe_job_start_w[gi]),
|
||||
.x_base_a(pe_x_base_a[gi*ADDR_WIDTH +: ADDR_WIDTH]),
|
||||
.x_base_b(pe_x_base_b[gi*ADDR_WIDTH +: ADDR_WIDTH]),
|
||||
.result_addr_a(pe_result_addr_a[gi*ADDR_WIDTH +: ADDR_WIDTH]),
|
||||
.result_addr_b(pe_result_addr_b[gi*ADDR_WIDTH +: ADDR_WIDTH]),
|
||||
.node_id_a(pe_node_id_a[gi*16 +: 16]), .node_id_b(pe_node_id_b[gi*16 +: 16]),
|
||||
.job_done(pe_job_done_w[gi]),
|
||||
.result_data_a(pe_result_data_a[gi*DATA_WIDTH +: DATA_WIDTH]),
|
||||
.result_data_b(pe_result_data_b[gi*DATA_WIDTH +: DATA_WIDTH]),
|
||||
.result_node_id_a(pe_result_node_id_a[gi*16 +: 16]),
|
||||
.result_node_id_b(pe_result_node_id_b[gi*16 +: 16]),
|
||||
.result_addr_a_out(pe_result_addr_a_out[gi*ADDR_WIDTH +: ADDR_WIDTH]),
|
||||
.result_addr_b_out(pe_result_addr_b_out[gi*ADDR_WIDTH +: ADDR_WIDTH]),
|
||||
.group_n_tiles(n_tiles_lat), .group_tcnt(tcnt),
|
||||
.group_tile_data(group_tile_data_r), .group_tile_valid(group_tile_valid),
|
||||
.pe_tile_ack(pe_acked_pulse[gi]),
|
||||
.mem_active(pe_mem_active[gi]), .mem_grant(pe_mem_grant[gi]),
|
||||
.ctrl_req(pe_ctrl_req[gi]), .ctrl_wr(pe_ctrl_wr[gi]),
|
||||
.ctrl_addr(pe_ctrl_addr[gi*(ADDR_WIDTH-1) +: (ADDR_WIDTH-1)]),
|
||||
.ctrl_wdata(pe_ctrl_wdata[gi*32*BURST_LEN +: 32*BURST_LEN]),
|
||||
.ctrl_wmask(pe_ctrl_wmask[gi*4*BURST_LEN +: 4*BURST_LEN]),
|
||||
.ctrl_rdata(pe_ctrl_rdata[gi*32*BURST_LEN +: 32*BURST_LEN]),
|
||||
.ctrl_ready(pe_ctrl_ready[gi]), .ctrl_busy(pe_ctrl_busy[gi])
|
||||
);
|
||||
end
|
||||
endgenerate
|
||||
|
||||
reg [3:0] pe_done_latch;
|
||||
// combinational "what pe_acked/pe_done_latch would be if we also
|
||||
// fold in THIS cycle's own pulses" -- used both to DECIDE the
|
||||
// barrier this cycle (no extra latency) and, explicitly, as what
|
||||
// gets written back when the barrier hasn't cleared yet. Kept as
|
||||
// named wires (not relying on nonblocking-assignment-order
|
||||
// last-write-wins semantics) so the real intent is unambiguous to
|
||||
// a future reader, not just technically correct.
|
||||
wire [3:0] pe_acked_next = pe_acked | pe_acked_pulse;
|
||||
wire [3:0] pe_done_latch_next = pe_done_latch | pe_job_done_w;
|
||||
|
||||
always @(posedge clk) begin
|
||||
if (rst) begin
|
||||
state <= S_IDLE;
|
||||
job_done <= 1'b0;
|
||||
pf_start <= 1'b0;
|
||||
consume_done <= 1'b0;
|
||||
tile_req <= 1'b0;
|
||||
group_tile_valid <= 1'b0;
|
||||
pe_acked <= 4'b0;
|
||||
pe_done_latch <= 4'b0;
|
||||
tcnt <= 16'd0;
|
||||
end else begin
|
||||
job_done <= 1'b0;
|
||||
pf_start <= 1'b0;
|
||||
consume_done <= 1'b0;
|
||||
tile_req <= 1'b0;
|
||||
|
||||
// real race, deliberately handled: a PE's own job_done can
|
||||
// pulse the SAME cycle S_TILELOOP's barrier clears for the
|
||||
// LAST tile (i.e. the same cycle the group transitions to
|
||||
// S_WAITDONE) -- its own downstream writeback can complete
|
||||
// before the group has even formally entered S_WAITDONE.
|
||||
// Accumulate unconditionally, every cycle, so no early
|
||||
// pe_job_done_w pulse is ever missed; S_WAITDONE's own
|
||||
// success branch below explicitly overrides this back to 0
|
||||
// (Verilog's own last-nonblocking-write-wins rule within
|
||||
// one always block -- intentional here, unlike pe_acked's
|
||||
// window which is fully contained inside S_TILELOOP and so
|
||||
// uses the more explicit if/else form instead).
|
||||
pe_done_latch <= pe_done_latch_next;
|
||||
|
||||
case (state)
|
||||
S_IDLE: begin
|
||||
if (job_start) begin
|
||||
w_base_lat <= w_base;
|
||||
n_tiles_lat <= n_tiles;
|
||||
tcnt <= 16'd0;
|
||||
state <= S_MEMWAIT;
|
||||
end
|
||||
end
|
||||
|
||||
S_MEMWAIT: begin
|
||||
if (mem_grant) begin
|
||||
pf_start <= 1'b1;
|
||||
state <= S_PREFETCH;
|
||||
end
|
||||
end
|
||||
|
||||
S_PREFETCH: begin
|
||||
if (pf_done) begin
|
||||
consume_done <= 1'b1;
|
||||
state <= S_SWAP;
|
||||
end
|
||||
end
|
||||
|
||||
S_SWAP: begin
|
||||
// one settle cycle, same real reason packed_slot.v's
|
||||
// own S_SWAP exists (layer_weight_buffer.v's do_swap).
|
||||
state <= S_PEJOBSTART;
|
||||
end
|
||||
|
||||
S_PEJOBSTART: begin
|
||||
// pe_job_start_w is combinational on (state ==
|
||||
// S_PEJOBSTART), so all 4 PEs see job_start the
|
||||
// SAME cycle -- real, established one-shot-pulse
|
||||
// discipline (EXP-0066), now applied 4-way.
|
||||
tile_req <= 1'b1;
|
||||
tile_base <= tcnt[BUFADDRW-1:0]*P_IN[BUFADDRW-1:0];
|
||||
state <= S_TILELOOP;
|
||||
end
|
||||
|
||||
// real barrier: hold this tile's data broadcast until
|
||||
// ALL 4 PEs have ack'd it (pe_acked_next all-ones), THEN
|
||||
// advance tcnt and issue the next weight_tile_gather
|
||||
// request -- see this module's own header for why a
|
||||
// bare pulse/level race would be unsafe here. pe_acked
|
||||
// is explicitly written on EVERY path through this
|
||||
// state (either reset to 0 when the barrier clears, or
|
||||
// folded forward via pe_acked_next otherwise) -- never
|
||||
// relies on assignment order elsewhere in the block.
|
||||
S_TILELOOP: begin
|
||||
if (tile_valid && !group_tile_valid) begin
|
||||
group_tile_data_r <= tile_data;
|
||||
group_tile_valid <= 1'b1;
|
||||
end
|
||||
if (group_tile_valid && (&pe_acked_next)) begin
|
||||
group_tile_valid <= 1'b0;
|
||||
pe_acked <= 4'b0;
|
||||
if (tcnt == n_tiles_lat - 16'd1) begin
|
||||
state <= S_WAITDONE;
|
||||
end else begin
|
||||
tcnt <= tcnt_next;
|
||||
tile_req <= 1'b1;
|
||||
tile_base <= tcnt_next[BUFADDRW-1:0]*P_IN[BUFADDRW-1:0];
|
||||
state <= S_TILELOOP;
|
||||
end
|
||||
end else begin
|
||||
pe_acked <= pe_acked_next;
|
||||
end
|
||||
end
|
||||
|
||||
S_WAITDONE: begin
|
||||
if (&pe_done_latch_next) begin
|
||||
pe_done_latch <= 4'b0;
|
||||
job_done <= 1'b1;
|
||||
state <= S_IDLE;
|
||||
end
|
||||
end
|
||||
|
||||
default: state <= S_IDLE;
|
||||
endcase
|
||||
end
|
||||
end
|
||||
endmodule
|
||||
@@ -0,0 +1,292 @@
|
||||
`timescale 1ns/1ps
|
||||
|
||||
// ============================================================
|
||||
// EXP-0089 -- isolated correctness test for systolic_group.v: one
|
||||
// group of 4 packed_pe.v instances sharing ONE real weight fetch
|
||||
// (broadcast, not a literal PE-to-PE systolic shift register -- see
|
||||
// systolic_group.v's own header for the real, user-confirmed design
|
||||
// choice). Same real discipline as every other new module in this
|
||||
// project (act_tile_fetch.v EXP-0079, ddr_prefetch_mgr.v EXP-0083,
|
||||
// result_writeback.v EXP-0088): verify in isolation, with a real
|
||||
// memory backend and a real, independently-reproduced golden model,
|
||||
// BEFORE any Director/SPI-protocol integration.
|
||||
//
|
||||
// Real backend: burst_mem_model32.v (same EXP-0084 model every other
|
||||
// v3 isolated testbench uses) + sdram_arbiter_n.v with NUM_REQ=5 (1
|
||||
// group-level weight-fetch requester + 4 independent per-PE
|
||||
// activation-fetch/writeback requesters) -- sdram_arbiter_n.v's own
|
||||
// NUM_REQ already generalizes to this without any change, confirmed
|
||||
// by direct reuse here, not by inspection.
|
||||
//
|
||||
// Runs TWO consecutive group jobs (different positions/weights each
|
||||
// time) specifically to catch any "forgot to clear a per-job latch"
|
||||
// bug in the group's own barrier state (pe_acked/pe_done_latch) --
|
||||
// a single-job test would not exercise that reset path at all.
|
||||
// ============================================================
|
||||
module tb;
|
||||
localparam BURST_LEN = 8;
|
||||
localparam SDRAM_ADDR_WIDTH = 25;
|
||||
localparam CLK_FREQ_MHZ = 64;
|
||||
localparam CLK_PERIOD_NS = 1000.0/CLK_FREQ_MHZ;
|
||||
|
||||
localparam DATA_WIDTH = 8;
|
||||
localparam P_IN = 8;
|
||||
localparam ACC_WIDTH = 32;
|
||||
localparam ADDR_WIDTH = 26;
|
||||
localparam N_INPUTS = 128;
|
||||
localparam N_TILES = N_INPUTS/P_IN;
|
||||
localparam LAYER_BYTES = N_INPUTS;
|
||||
localparam WORDS_PER_LAYER = LAYER_BYTES/2;
|
||||
localparam NUM_REQ = 5; // 1 group weight-fetch + 4 PE activation/writeback
|
||||
|
||||
reg clk = 0;
|
||||
always #(CLK_PERIOD_NS/2.0) clk = ~clk;
|
||||
reg rst;
|
||||
integer cyc;
|
||||
always @(posedge clk) if (!rst) cyc <= cyc + 1;
|
||||
|
||||
// ---- real burst-memory backend, shared via a real 5-way arbiter ----
|
||||
wire ctrl_req, ctrl_wr;
|
||||
wire [SDRAM_ADDR_WIDTH-1:0] ctrl_addr;
|
||||
wire [32*BURST_LEN-1:0] ctrl_wdata;
|
||||
wire [4*BURST_LEN-1:0] ctrl_wmask;
|
||||
wire [32*BURST_LEN-1:0] ctrl_rdata;
|
||||
wire ctrl_ready, ctrl_busy;
|
||||
|
||||
reg wpre_req, wpre_wr;
|
||||
reg [SDRAM_ADDR_WIDTH-1:0] wpre_addr;
|
||||
reg [32*BURST_LEN-1:0] wpre_wdata;
|
||||
reg pre_active;
|
||||
|
||||
wire [NUM_REQ-1:0] arb_active, arb_grant, arb_req, arb_wr;
|
||||
wire [NUM_REQ*SDRAM_ADDR_WIDTH-1:0] arb_addr;
|
||||
wire [NUM_REQ*32*BURST_LEN-1:0] arb_wdata, arb_rdata;
|
||||
wire [NUM_REQ*4*BURST_LEN-1:0] arb_wmask;
|
||||
wire [NUM_REQ-1:0] arb_ready, arb_busy;
|
||||
|
||||
wire real_ctrl_req, real_ctrl_wr;
|
||||
wire [SDRAM_ADDR_WIDTH-1:0] real_ctrl_addr;
|
||||
wire [32*BURST_LEN-1:0] real_ctrl_wdata;
|
||||
wire [4*BURST_LEN-1:0] real_ctrl_wmask;
|
||||
|
||||
sdram_arbiter_n #(.NUM_REQ(NUM_REQ), .ADDR_WIDTH(SDRAM_ADDR_WIDTH), .BURST_LEN(BURST_LEN)) u_arb (
|
||||
.clk(clk), .rst(rst),
|
||||
.req_active(arb_active), .req_grant(arb_grant),
|
||||
.req_req(arb_req), .req_wr(arb_wr), .req_addr(arb_addr),
|
||||
.req_wdata(arb_wdata), .req_wmask(arb_wmask),
|
||||
.req_rdata(arb_rdata), .req_ready(arb_ready), .req_busy(arb_busy),
|
||||
.ctrl_req(real_ctrl_req), .ctrl_wr(real_ctrl_wr), .ctrl_addr(real_ctrl_addr),
|
||||
.ctrl_wdata(real_ctrl_wdata), .ctrl_wmask(real_ctrl_wmask),
|
||||
.ctrl_rdata(ctrl_rdata), .ctrl_ready(ctrl_ready), .ctrl_busy(ctrl_busy)
|
||||
);
|
||||
|
||||
assign ctrl_req = pre_active ? wpre_req : real_ctrl_req;
|
||||
assign ctrl_wr = pre_active ? wpre_wr : real_ctrl_wr;
|
||||
assign ctrl_addr = pre_active ? wpre_addr : real_ctrl_addr;
|
||||
assign ctrl_wdata = pre_active ? wpre_wdata : real_ctrl_wdata;
|
||||
assign ctrl_wmask = pre_active ? {(4*BURST_LEN){1'b0}} : real_ctrl_wmask;
|
||||
|
||||
burst_mem_model32 #(
|
||||
.BURST_LEN(BURST_LEN), .ADDR_WIDTH(SDRAM_ADDR_WIDTH)
|
||||
) u_mem (
|
||||
.clk(clk), .rst(rst),
|
||||
.req(ctrl_req), .wr(ctrl_wr), .addr(ctrl_addr), .wdata(ctrl_wdata), .wmask(ctrl_wmask),
|
||||
.rdata(ctrl_rdata), .ready(ctrl_ready), .busy(ctrl_busy)
|
||||
);
|
||||
|
||||
function automatic signed [7:0] weight_byte(input integer li, input integer t);
|
||||
reg [7:0] tmp;
|
||||
begin
|
||||
tmp = (li*17 + t*29 + 13) & 8'hFF;
|
||||
weight_byte = $signed(tmp);
|
||||
end
|
||||
endfunction
|
||||
function automatic signed [7:0] input_byte(input integer li, input integer pos, input integer t);
|
||||
reg [7:0] tmp;
|
||||
begin
|
||||
tmp = (li*11 + pos*41 + t*7 + 3) & 8'hFF;
|
||||
input_byte = $signed(tmp);
|
||||
end
|
||||
endfunction
|
||||
|
||||
task automatic sdram_write_burst(input [SDRAM_ADDR_WIDTH-1:0] word_addr, input [32*BURST_LEN-1:0] data);
|
||||
begin
|
||||
@(posedge clk); while (ctrl_busy) @(posedge clk);
|
||||
wpre_req = 1'b1; wpre_wr = 1'b1; wpre_addr = word_addr; wpre_wdata = data;
|
||||
@(posedge clk); wpre_req = 1'b0;
|
||||
while (!ctrl_ready) @(posedge clk);
|
||||
end
|
||||
endtask
|
||||
|
||||
task automatic preload_sdram_layer(input integer li);
|
||||
integer bi, wb, tt;
|
||||
reg [32*BURST_LEN-1:0] burst_data;
|
||||
begin
|
||||
for (bi = 0; bi < (LAYER_BYTES/(4*BURST_LEN)); bi = bi + 1) begin
|
||||
for (wb = 0; wb < BURST_LEN; wb = wb + 1) begin
|
||||
tt = bi*(4*BURST_LEN) + wb*4;
|
||||
burst_data[wb*32 +: 32] = {weight_byte(li, tt+3), weight_byte(li, tt+2),
|
||||
weight_byte(li, tt+1), weight_byte(li, tt)};
|
||||
end
|
||||
sdram_write_burst((li*WORDS_PER_LAYER + bi*BURST_LEN), burst_data);
|
||||
end
|
||||
end
|
||||
endtask
|
||||
|
||||
localparam [ADDR_WIDTH-1:0] ACT_MEM_BASE = 26'h10000;
|
||||
function automatic [ADDR_WIDTH-1:0] act_x_base(input integer li, input integer pos);
|
||||
act_x_base = ACT_MEM_BASE + (li*16 + pos) * ((N_TILES/4)*BURST_LEN);
|
||||
endfunction
|
||||
|
||||
task automatic preload_sdram_activation(input integer li, input integer pos);
|
||||
integer tq, qi;
|
||||
reg [32*BURST_LEN-1:0] burst_data;
|
||||
reg [ADDR_WIDTH-1:0] base;
|
||||
begin
|
||||
base = act_x_base(li, pos);
|
||||
for (tq = 0; tq < N_TILES/4; tq = tq + 1) begin
|
||||
burst_data = {(32*BURST_LEN){1'b0}};
|
||||
for (qi = 0; qi < 4; qi = qi + 1)
|
||||
burst_data[qi*64 +: 64] = {input_byte(li, pos, (4*tq+qi)*P_IN + 7), input_byte(li, pos, (4*tq+qi)*P_IN + 6),
|
||||
input_byte(li, pos, (4*tq+qi)*P_IN + 5), input_byte(li, pos, (4*tq+qi)*P_IN + 4),
|
||||
input_byte(li, pos, (4*tq+qi)*P_IN + 3), input_byte(li, pos, (4*tq+qi)*P_IN + 2),
|
||||
input_byte(li, pos, (4*tq+qi)*P_IN + 1), input_byte(li, pos, (4*tq+qi)*P_IN + 0)};
|
||||
sdram_write_burst(base[SDRAM_ADDR_WIDTH-1:0] + tq*BURST_LEN, burst_data);
|
||||
end
|
||||
end
|
||||
endtask
|
||||
|
||||
// ---- systolic_group.v (DUT) ----
|
||||
reg job_start;
|
||||
reg [ADDR_WIDTH-1:0] w_base;
|
||||
reg [15:0] n_tiles_in;
|
||||
reg [4*ADDR_WIDTH-1:0] pe_x_base_a, pe_x_base_b, pe_result_addr_a, pe_result_addr_b;
|
||||
reg [4*16-1:0] pe_node_id_a, pe_node_id_b;
|
||||
wire job_done;
|
||||
wire [4*DATA_WIDTH-1:0] pe_result_data_a, pe_result_data_b;
|
||||
wire [4*16-1:0] pe_result_node_id_a, pe_result_node_id_b;
|
||||
wire [4*ADDR_WIDTH-1:0] pe_result_addr_a_out, pe_result_addr_b_out;
|
||||
|
||||
wire grp_mem_active;
|
||||
wire [3:0] pe_mem_active;
|
||||
|
||||
systolic_group #(
|
||||
.DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .ACC_WIDTH(ACC_WIDTH),
|
||||
.BURST_LEN(BURST_LEN), .ADDR_WIDTH(ADDR_WIDTH), .LAYER_BYTES(LAYER_BYTES)
|
||||
) dut (
|
||||
.clk(clk), .rst(rst),
|
||||
.job_start(job_start), .w_base(w_base), .n_tiles(n_tiles_in),
|
||||
.pe_x_base_a(pe_x_base_a), .pe_x_base_b(pe_x_base_b),
|
||||
.pe_result_addr_a(pe_result_addr_a), .pe_result_addr_b(pe_result_addr_b),
|
||||
.pe_node_id_a(pe_node_id_a), .pe_node_id_b(pe_node_id_b),
|
||||
.job_done(job_done),
|
||||
.pe_result_data_a(pe_result_data_a), .pe_result_data_b(pe_result_data_b),
|
||||
.pe_result_node_id_a(pe_result_node_id_a), .pe_result_node_id_b(pe_result_node_id_b),
|
||||
.pe_result_addr_a_out(pe_result_addr_a_out), .pe_result_addr_b_out(pe_result_addr_b_out),
|
||||
.mem_active(grp_mem_active), .mem_grant(arb_grant[0]),
|
||||
.ctrl_req(arb_req[0]), .ctrl_wr(arb_wr[0]), .ctrl_addr(arb_addr[0*SDRAM_ADDR_WIDTH +: SDRAM_ADDR_WIDTH]),
|
||||
.ctrl_wdata(arb_wdata[0*32*BURST_LEN +: 32*BURST_LEN]), .ctrl_wmask(arb_wmask[0*4*BURST_LEN +: 4*BURST_LEN]),
|
||||
.ctrl_rdata(arb_rdata[0*32*BURST_LEN +: 32*BURST_LEN]), .ctrl_ready(arb_ready[0]), .ctrl_busy(arb_busy[0]),
|
||||
.pe_mem_active(pe_mem_active), .pe_mem_grant(arb_grant[4:1]),
|
||||
.pe_ctrl_req(arb_req[4:1]), .pe_ctrl_wr(arb_wr[4:1]),
|
||||
.pe_ctrl_addr(arb_addr[1*SDRAM_ADDR_WIDTH +: 4*SDRAM_ADDR_WIDTH]),
|
||||
.pe_ctrl_wdata(arb_wdata[1*32*BURST_LEN +: 4*32*BURST_LEN]),
|
||||
.pe_ctrl_wmask(arb_wmask[1*4*BURST_LEN +: 4*4*BURST_LEN]),
|
||||
.pe_ctrl_rdata(arb_rdata[1*32*BURST_LEN +: 4*32*BURST_LEN]),
|
||||
.pe_ctrl_ready(arb_ready[4:1]), .pe_ctrl_busy(arb_busy[4:1])
|
||||
);
|
||||
|
||||
assign arb_active[0] = grp_mem_active;
|
||||
assign arb_active[4:1] = pe_mem_active;
|
||||
|
||||
integer errors, tests;
|
||||
integer li_i, gi, wd;
|
||||
integer acc, s;
|
||||
reg signed [DATA_WIDTH-1:0] expected [0:7]; // 4 PEs x 2 lanes
|
||||
|
||||
task automatic run_group_job(input integer li, input integer pos_base);
|
||||
integer pe, lane, pos, tt;
|
||||
reg [DATA_WIDTH-1:0] got_a, got_b;
|
||||
begin
|
||||
tests = tests + 1;
|
||||
@(posedge clk);
|
||||
job_start = 1'b1;
|
||||
w_base = li*WORDS_PER_LAYER;
|
||||
n_tiles_in = N_TILES[15:0];
|
||||
for (pe = 0; pe < 4; pe = pe + 1) begin
|
||||
pe_x_base_a[pe*ADDR_WIDTH +: ADDR_WIDTH] = act_x_base(li, pos_base + pe*2);
|
||||
pe_x_base_b[pe*ADDR_WIDTH +: ADDR_WIDTH] = act_x_base(li, pos_base + pe*2 + 1);
|
||||
pe_result_addr_a[pe*ADDR_WIDTH +: ADDR_WIDTH] = 26'h9000 + pe*2;
|
||||
pe_result_addr_b[pe*ADDR_WIDTH +: ADDR_WIDTH] = 26'h9000 + pe*2 + 1;
|
||||
pe_node_id_a[pe*16 +: 16] = li*100 + pos_base + pe*2;
|
||||
pe_node_id_b[pe*16 +: 16] = li*100 + pos_base + pe*2 + 1;
|
||||
end
|
||||
@(posedge clk);
|
||||
job_start = 1'b0;
|
||||
|
||||
// real, independently-reproduced golden model (same formula
|
||||
// as tb_packed_slot.v's own, applied per PE/lane)
|
||||
for (pe = 0; pe < 4; pe = pe + 1) begin
|
||||
for (lane = 0; lane < 2; lane = lane + 1) begin
|
||||
pos = pos_base + pe*2 + lane;
|
||||
acc = 0;
|
||||
for (tt = 0; tt < N_INPUTS; tt = tt + 1)
|
||||
acc = acc + (input_byte(li, pos, tt) * weight_byte(li, tt));
|
||||
s = acc;
|
||||
if (s <= 0) expected[pe*2+lane] = 0;
|
||||
else if (s > 127) expected[pe*2+lane] = 8'sd127;
|
||||
else expected[pe*2+lane] = s[DATA_WIDTH-1:0];
|
||||
end
|
||||
end
|
||||
|
||||
wd = 0;
|
||||
while (!job_done && wd < 4000) begin @(posedge clk); wd = wd + 1; end
|
||||
if (!job_done) begin
|
||||
$display("FAIL li=%0d pos_base=%0d: TIMEOUT waiting for group job_done", li, pos_base);
|
||||
errors = errors + 1;
|
||||
end else begin
|
||||
for (pe = 0; pe < 4; pe = pe + 1) begin
|
||||
got_a = pe_result_data_a[pe*DATA_WIDTH +: DATA_WIDTH];
|
||||
got_b = pe_result_data_b[pe*DATA_WIDTH +: DATA_WIDTH];
|
||||
if (got_a !== expected[pe*2] || got_b !== expected[pe*2+1]) begin
|
||||
$display("FAIL li=%0d pos_base=%0d PE%0d: got_a=%0d got_b=%0d expected_a=%0d expected_b=%0d",
|
||||
li, pos_base, pe, $signed(got_a), $signed(got_b),
|
||||
$signed(expected[pe*2]), $signed(expected[pe*2+1]));
|
||||
errors = errors + 1;
|
||||
end else begin
|
||||
$display("PASS li=%0d pos_base=%0d PE%0d: a=%0d b=%0d (systolic_group.v)",
|
||||
li, pos_base, pe, $signed(got_a), $signed(got_b));
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
endtask
|
||||
|
||||
initial begin
|
||||
errors = 0; tests = 0; cyc = 0;
|
||||
rst = 1; pre_active = 1'b1;
|
||||
wpre_req = 0; wpre_wr = 0; wpre_addr = 0; wpre_wdata = 0;
|
||||
job_start = 0; w_base = 0; n_tiles_in = 0;
|
||||
pe_x_base_a = 0; pe_x_base_b = 0; pe_result_addr_a = 0; pe_result_addr_b = 0;
|
||||
pe_node_id_a = 0; pe_node_id_b = 0;
|
||||
repeat(5) @(posedge clk);
|
||||
rst = 0;
|
||||
@(posedge clk); while (ctrl_busy) @(posedge clk);
|
||||
|
||||
$display("=== preload SDRAM: 2 layers' weights + 16 activation positions ===");
|
||||
for (li_i = 0; li_i < 2; li_i = li_i + 1) begin
|
||||
preload_sdram_layer(li_i);
|
||||
for (gi = 0; gi < 8; gi = gi + 1) preload_sdram_activation(li_i, gi);
|
||||
end
|
||||
@(posedge clk);
|
||||
pre_active = 1'b0;
|
||||
|
||||
$display("=== systolic_group.v: 2 group jobs (4 PEs x 2 lanes each) ===");
|
||||
run_group_job(0, 0);
|
||||
run_group_job(1, 0);
|
||||
|
||||
$display("=== %0d/%0d tests, %0d errors ===", tests-errors, tests, errors);
|
||||
if (errors == 0) $display("ALL TESTS PASSED (tb_systolic_group)");
|
||||
$finish;
|
||||
end
|
||||
endmodule
|
||||
Reference in New Issue
Block a user