michele/FPGA-Neural
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

perf(v2): shared activation cache - further 1.66-2.00x real speedup (DEC-0016)

Browse Source 
Implements optimization #2 from the final benchmark campaign's own
recommendation, on top of DEC-0015's word-level burst rewrite: a new
shared activation_cache.v module fetches a given activation (X)
vector from PSRAM once instead of once per neuron sharing it - the
exact redundant traffic pattern the dense-layer workloads in this
project's benchmark suite exhibit.

Each memory_manager's own prefetch_engine now fetches WEIGHTS only;
the activation half is requested from the shared cache instead
(single-tag, tile-granular, N_SLOTS request ports, its own real
word-level PSRAM backend via a new dedicated arbiter port).
dataflow_core.v/slot_mem_arbiter.v/neural_multiprocessor.v widened to
N_SLOTS+1 ports to arbitrate the cache's traffic alongside each
slot's weight traffic.

Two real bugs found and fixed during implementation (ERR-0010): a
target-bank/pending-bank race in memory_manager.v's activation-cache
wiring (the same bug class ERR-0006 already fixed once for
pf_target_bank - a later handoff's queued request can overwrite which
bank an earlier, still-in-flight request's ack applies to), and a
repeat of ERR-0009's N_SLOTS=1 zero-width replication bug in
activation_cache.v itself.

Real, measured results: the full final-benchmark campaign (24/24
workload/config combinations) re-verified bit-exact. D-Stress cycles
fall a further 1.66-2.00x on top of DEC-0015 (~4x combined vs the
original byte-level baseline). But the cache's real Fmax cost is much
steeper than DEC-0015's own: N_SLOTS=2 (the recommended default,
DEC-0014) drops from 133.58 to 87.72 MHz (-34%, margin over 80MHz
shrinks from +67% to +9.7%), and N_SLOTS=4 drops to 65.01 MHz - now
FAILING the 80MHz target it previously passed. Combined real
wall-clock speedup vs the original baseline: N=1 3.86x, N=2 2.45x
(both real net wins); N=4 is a real regression once its own now-failing
Fmax is honestly used, though N=4 was never the recommended
configuration.

N_SLOTS=2 remains the recommended default (DEC-0014 unaffected) with
a thinner but still real Fmax margin. Cache hit-detection pipelining
is flagged as concrete follow-up work if N_SLOTS>2 is ever needed with
the cache active - not attempted this round.

Logged: simulation/synthesis/timing/benchmark/decisions (DEC-0016)/
experiments (EXP-0016)/errors (ERR-0010)/development.log, ROADMAP.md
updated.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013xXuuRUWZScuo1DeYJxs3v
This commit is contained in:
michele
2026-09-05 20:59:14 +02:00
co-authored by Claude Sonnet 5
parent e4a5540b6e
commit 63cac6a7e5
16 changed files with 931 additions and 211 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hardware/v2/rtl/memory_manager.v
+150 -87
View File
@@ -10,42 +10,41 @@
// The processor sees only "data available" (operand_valid/ready,
// tile_last) -- never PSRAM request/wait cycles directly (§12).
//
// Post-M10 (decisions.log DEC-0015): this port talks directly to
// memory_interface.v's own 16-bit word interface instead of routing
// through int8_memory_access.v's byte-splitting layer -- every real
// transaction now moves a full PSRAM word (2 bytes) instead of
// discarding half of one, halving the real transaction count for
// prefetch_engine's own reads. int8_memory_access.v itself is
// Post-M10 (decisions.log DEC-0015): the WEIGHT backend port talks
// directly to memory_interface.v's own 16-bit word interface instead
// of routing through int8_memory_access.v's byte-splitting layer --
// every real transaction now moves a full PSRAM word (2 bytes)
// instead of discarding half of one. int8_memory_access.v itself is
// untouched (still frozen V1); V2 simply no longer instantiates it in
// this datapath, reusing the lower (word-level) layer directly
// instead, the same "reuse what fits" precedent slot_mem_arbiter.v
// already set for hardware/v1/rtl/mem_arbiter.v.
//
// Post-M10 (decisions.log DEC-0016): the ACTIVATION (X) side is no
// longer fetched from PSRAM by this module's own prefetch_engine at
// all -- it is requested from a shared activation_cache.v instance
// (one per dataflow_core, not one per slot), which fetches a given
// X vector from PSRAM once and serves every memory_manager sharing
// that same x_base directly on-chip. Each bank therefore becomes
// ready only once BOTH its activation half (cache ack) AND its
// weight half (prefetch_engine's own pf_done, now W-only) have
// arrived -- bank_ready[b] = bank_x_ready[b] && bank_w_ready[b].
//
// Double-buffered prefetch (§13): while the processor consumes tile
// N from bank "current", this module retargets the single
// prefetch_engine instance (M4) at bank "next" to fetch tile N+1
// N from bank "current", this module retargets its single
// prefetch_engine instance (M4, W-only) and issues a fresh
// activation_cache request at bank "next" to fetch tile N+1
// concurrently. On tile handoff, banks swap; if a bank isn't ready in
// time (prefetch slower than compute for this run), operand_valid
// simply stays low until it is -- a real stall, not hidden, so its
// frequency is genuinely measurable (§22, deferred to M9). NOTE
// (measured characteristic, not yet optimized -- see
// hardware/v2/logs/decisions.log DEC-0006): the bank-swap-and-check
// control path itself costs a minimum 1 idle cycle per tile handoff
// even when the next bank was already prefetched in time, unlike
// neural_processor.v's own zero-gap tile acceptance -- a real,
// deliberately-not-hidden overhead of this first Memory Manager
// implementation, left for M10 (Optimization) to revisit with real
// stall-percentage data (§22) rather than optimized blindly now.
// time, operand_valid simply stays low until it is -- a real stall,
// not hidden (§22). NOTE (measured characteristic, not yet optimized
// -- see decisions.log DEC-0006): the bank-swap-and-check control path
// itself costs a minimum 1 idle cycle per tile handoff even when the
// next bank was already prefetched in time, unlike neural_processor.v's
// own zero-gap tile acceptance.
//
// One job = one neuron's worth of tiles (n_tiles), read from x_base/
// w_base (PSRAM byte addresses), followed by writing the single
// INT8 result back to result_addr. The result write only happens
// after the last tile has been handed off and prefetch_engine is
// idle (temporally disjoint from prefetching by construction), so no
// separate backend arbiter is needed at this milestone -- see
// decisions.log DEC-0006 for why, and what changes once multiple
// concurrent jobs/processors need to share one backend port
// (deferred, not yet needed).
// w_base (PSRAM byte addresses), followed by writing the single INT8
// result back to result_addr.
// ================================================================
module memory_manager #(
@@ -56,8 +55,7 @@ module memory_manager #(
input wire clk,
input wire rst,
// ---- job control (from a future Neural Director, M5; driven
// directly by a testbench at M4) ----
// ---- job control (from Neural Director, M5) ----
input wire job_start,
input wire [ADDR_WIDTH-1:0] x_base,
input wire [ADDR_WIDTH-1:0] w_base,
@@ -78,12 +76,19 @@ module memory_manager #(
output reg result_ready,
input wire signed [DATA_WIDTH-1:0] result_data,
// ---- Memory Backend Interface (word-level, matches
// ---- shared activation_cache.v request port (post-M10 DEC-0016
// -- one per memory_manager instance, cache is shared/instantiated
// once per dataflow_core) ----
output reg xc_req, // one-cycle pulse
output reg [ADDR_WIDTH-1:0] xc_x_base,
output reg [15:0] xc_tile_idx,
input wire xc_ack, // one-cycle pulse
input wire signed [DATA_WIDTH*P_IN-1:0] xc_tile_x,
// ---- WEIGHT Memory Backend Interface (word-level, matches
// hardware/v1/rtl/memory_interface.v's contract exactly -- see
// prefetch_engine.v's own header and decisions.log DEC-0015 for
// why this is now word- rather than byte-level: int8_memory_access.v
// is no longer in the datapath, each transaction moves a full
// 16-bit PSRAM word instead of discarding half of it) ----
// prefetch_engine.v's own header and decisions.log DEC-0015/
// DEC-0016 for why this is word-level and weight-only) ----
output wire mem_req,
output wire mem_wr,
output wire [ADDR_WIDTH-1:0] mem_addr, // WORD address
@@ -108,17 +113,55 @@ module memory_manager #(
reg [15:0] tile_idx; // tile currently presented (bank `current`)
reg current_bank; // 0 or 1
reg [1:0] bank_ready; // bank_ready[b] = bank b holds valid, unconsumed prefetched data
// ---- double-buffer storage: X half filled by the shared cache,
// W half filled by this module's own prefetch_engine -- a bank is
// usable once BOTH halves have arrived. ----
reg [1:0] bank_x_ready, bank_w_ready;
wire [1:0] bank_ready = bank_x_ready & bank_w_ready;
// ---- double-buffer storage (owned here, filled by prefetch_engine) ----
reg signed [DATA_WIDTH*P_IN-1:0] bank_x [0:1];
reg signed [DATA_WIDTH*P_IN-1:0] bank_w [0:1];
// ---- single prefetch_engine instance, retargeted per bank ----
// ---- activation_cache request bookkeeping: single-entry pending
// (same idiom as pf_pending below -- only one outstanding cache
// request at a time, one instance to serve, one bank as its target). ----
reg xc_pending;
reg [ADDR_WIDTH-1:0] xc_pending_x_base;
reg [15:0] xc_pending_tile_idx;
reg xc_pending_bank;
// xc_target_bank is the bank the CURRENTLY-outstanding (already
// issued) cache request will fill -- set ONLY by the issue rule
// below, from xc_pending_bank, at the exact moment xc_req fires.
// Queueing logic (MM_IDLE/MM_PREFETCH_FIRST/MM_STREAM) writes
// xc_pending_bank, NEVER xc_target_bank directly -- writing
// xc_target_bank directly from queueing was a real bug (found via
// simulation): a later handoff can queue a NEW request (targeting
// a DIFFERENT bank) in the same cycle an EARLIER request is being
// issued, and program-order NBA "last write wins" would silently
// overwrite which bank the EARLIER (already in-flight) request's
// eventual ack gets applied to -- the exact same class of bug
// ERR-0006 already found and fixed once for pf_target_bank/
// pf_pending_bank (which already used this two-register pattern
// correctly; this module's activation-cache side did not, until
// now).
reg xc_target_bank;
// Tracks whether THIS instance's own cache request is still
// awaiting its ack (real PSRAM miss latency can easily exceed one
// neural_processor tile's own compute time, so a later handoff's
// "queue the next request" can genuinely race an earlier request
// still in flight -- the same class of race ERR-0006 already found
// and fixed once for pf_pending/pf_busy; fixed here the same way,
// with an explicit outstanding flag this module controls directly
// rather than inferring busy-ness from a signal with its own
// latency quirk).
reg xc_outstanding;
// ---- single prefetch_engine instance (WEIGHT-only post-DEC-0016),
// retargeted per bank ----
reg pf_start;
reg [ADDR_WIDTH-1:0] pf_x_addr, pf_w_addr;
reg [ADDR_WIDTH-1:0] pf_w_addr;
wire pf_busy, pf_done;
wire signed [DATA_WIDTH*P_IN-1:0] pf_tile_x, pf_tile_w;
wire signed [DATA_WIDTH*P_IN-1:0] pf_tile_w;
reg pf_target_bank; // which bank the CURRENTLY-running (or just-launched) prefetch fills
@@ -132,15 +175,17 @@ module memory_manager #(
// descriptor instead of touching pf_start directly; a single
// always-active rule issues pf_start once the engine is free.
reg pf_pending;
reg [ADDR_WIDTH-1:0] pf_pending_x, pf_pending_w;
reg [ADDR_WIDTH-1:0] pf_pending_w;
reg pf_pending_bank;
// prefetch_engine drives its OWN internal backend wires; the
// result-write FSM below drives its own. A combinational mux
// (never both at once, by construction -- see file header)
// selects which one actually reaches the real output port,
// avoiding a two-driver conflict on mem_req/mem_wr/mem_addr/
// mem_wdata/mem_lb_n/mem_ub_n.
// prefetch_engine drives its OWN internal weight-backend wires;
// the result-write FSM below drives its own. A combinational mux
// (never both at once, by construction -- MM_WRITE_RESULT/MM_DONE
// only run after every tile for this job has already been fetched,
// so prefetch_engine is guaranteed idle) selects which one actually
// reaches the real output port, same pattern as the pre-DEC-0015
// design, just weight-only now (the activation side moved to the
// shared activation_cache.v, DEC-0016).
wire pf_mem_req, pf_mem_wr;
wire [ADDR_WIDTH-1:0] pf_mem_addr;
wire [15:0] pf_mem_wdata;
@@ -150,9 +195,9 @@ module memory_manager #(
.DATA_WIDTH(DATA_WIDTH), .P_IN(P_IN), .ADDR_WIDTH(ADDR_WIDTH)
) u_prefetch (
.clk(clk), .rst(rst),
.fetch_start(pf_start), .x_addr(pf_x_addr), .w_addr(pf_w_addr),
.fetch_start(pf_start), .w_addr(pf_w_addr),
.fetch_busy(pf_busy), .fetch_done(pf_done),
.tile_x(pf_tile_x), .tile_w(pf_tile_w),
.tile_w(pf_tile_w),
.mem_req(pf_mem_req), .mem_wr(pf_mem_wr), .mem_addr(pf_mem_addr), .mem_wdata(pf_mem_wdata),
.mem_lb_n(pf_mem_lb_n), .mem_ub_n(pf_mem_ub_n),
.mem_rdata(mem_rdata), .mem_ready(mem_ready)
@@ -163,12 +208,6 @@ module memory_manager #(
reg [15:0] wr_mem_wdata;
reg wr_mem_lb_n, wr_mem_ub_n;
// wr_mem_req is SET while state==MM_WRITE_RESULT but only becomes
// valid (via NBA) the FOLLOWING cycle, i.e. while state==MM_DONE --
// the mux must select the write-back source across BOTH states,
// not just the one that issues it (an off-by-one here silently
// dropped the write request entirely -- found and fixed here, see
// hardware/v2/logs/errors.log ERR-0006).
wire wr_active = (state == MM_WRITE_RESULT) || (state == MM_DONE);
assign mem_req = wr_active ? wr_mem_req : pf_mem_req;
assign mem_wr = wr_active ? 1'b1 : pf_mem_wr;
@@ -188,45 +227,63 @@ module memory_manager #(
result_ready <= 1'b0;
pf_start <= 1'b0;
current_bank <= 1'b0;
bank_ready <= 2'b00;
bank_x_ready <= 2'b00;
bank_w_ready <= 2'b00;
tile_idx <= 16'h0;
pf_pending <= 1'b0;
xc_req <= 1'b0;
xc_pending <= 1'b0;
xc_outstanding <= 1'b0;
wr_mem_req <= 1'b0;
wr_mem_addr <= {ADDR_WIDTH{1'b0}};
wr_mem_wdata <= 16'h0000;
wr_mem_lb_n <= 1'b1;
wr_mem_ub_n <= 1'b1;
pf_pending <= 1'b0;
end else begin
job_done <= 1'b0;
pf_start <= 1'b0;
xc_req <= 1'b0;
result_ready <= 1'b0;
// Latch a completed prefetch into its target bank.
// Latch a completed weight prefetch into its target bank.
if (pf_done) begin
bank_x[pf_target_bank] <= pf_tile_x;
bank_w[pf_target_bank] <= pf_tile_w;
bank_ready[pf_target_bank] <= 1'b1;
bank_w_ready[pf_target_bank] <= 1'b1;
end
// Issue a pending fetch request as soon as the (single)
// Latch a completed activation-cache fetch into its target
// bank and clear the outstanding flag (see its own
// declaration comment above).
if (xc_ack) begin
bank_x[xc_target_bank] <= xc_tile_x;
bank_x_ready[xc_target_bank] <= 1'b1;
xc_outstanding <= 1'b0;
end
// Issue a pending weight fetch as soon as the (single)
// prefetch engine is genuinely free. The `!pf_start` guard
// is required, not cosmetic: pf_busy does not read 1 until
// the cycle AFTER pf_start was first observed (prefetch_
// engine's own fetch_busy<=1 is one clock behind its own
// fetch_start sampling), so checking !pf_busy alone leaves
// a genuine one-cycle window where a second pending
// request would fire on top of the one just launched,
// silently corrupting pf_target_bank for the fetch already
// in flight (found and fixed here -- see
// hardware/v2/logs/errors.log ERR-0006).
// is required, not cosmetic -- see hardware/v2/logs/
// errors.log ERR-0006.
if (pf_pending && !pf_busy && !pf_start) begin
pf_start <= 1'b1;
pf_x_addr <= pf_pending_x;
pf_w_addr <= pf_pending_w;
pf_target_bank <= pf_pending_bank;
pf_pending <= 1'b0;
end
// Issue a pending activation-cache request only once this
// instance's own PREVIOUS request has been genuinely acked
// (xc_outstanding low) -- see that flag's own declaration
// comment for why checking xc_req alone is not enough.
if (xc_pending && !xc_outstanding) begin
xc_req <= 1'b1;
xc_outstanding <= 1'b1;
xc_x_base <= xc_pending_x_base;
xc_tile_idx <= xc_pending_tile_idx;
xc_target_bank <= xc_pending_bank;
xc_pending <= 1'b0;
end
case (state)
MM_IDLE: begin
@@ -237,30 +294,37 @@ module memory_manager #(
result_addr_reg <= result_addr;
tile_idx <= 16'h0;
current_bank <= 1'b0;
bank_ready <= 2'b00;
bank_x_ready <= 2'b00;
bank_w_ready <= 2'b00;
operand_valid <= 1'b0;
// kick off the very first fetch (tile 0 into bank 0)
pf_pending <= 1'b1;
pf_pending_x <= x_base;
pf_pending_w <= w_base;
pf_pending_bank <= 1'b0;
xc_pending <= 1'b1;
xc_pending_x_base <= x_base;
xc_pending_tile_idx <= 16'h0;
xc_pending_bank <= 1'b0;
state <= MM_PREFETCH_FIRST;
end
end
MM_PREFETCH_FIRST: begin
if (bank_ready[0] || (pf_done && pf_target_bank == 1'b0)) begin
if (bank_ready[0]) begin
// Present tile 0; concurrently start prefetching
// tile 1 into bank 1, if there is one.
operand_valid <= 1'b1;
input_data <= pf_done ? pf_tile_x : bank_x[0];
weight_data <= pf_done ? pf_tile_w : bank_w[0];
input_data <= bank_x[0];
weight_data <= bank_w[0];
tile_last <= (n_tiles_reg == 16'h1);
if (n_tiles_reg > 16'h1) begin
pf_pending <= 1'b1;
pf_pending_x <= x_base_reg + P_IN[ADDR_WIDTH-1:0];
pf_pending_w <= w_base_reg + P_IN[ADDR_WIDTH-1:0];
pf_pending_bank <= 1'b1;
xc_pending <= 1'b1;
xc_pending_x_base <= x_base_reg;
xc_pending_tile_idx <= 16'h1;
xc_pending_bank <= 1'b1;
end
state <= MM_STREAM;
end
@@ -269,7 +333,8 @@ module memory_manager #(
MM_STREAM: begin
if (operand_valid && operand_ready) begin
// This tile consumed; free its bank, swap.
bank_ready[current_bank] <= 1'b0;
bank_x_ready[current_bank] <= 1'b0;
bank_w_ready[current_bank] <= 1'b0;
current_bank <= ~current_bank;
tile_idx <= tile_idx + 16'h1;
operand_valid <= 1'b0; // re-asserted below once the new bank is ready
@@ -279,17 +344,14 @@ module memory_manager #(
state <= MM_WAIT_RESULT;
end else if (tile_idx + 16'h2 < n_tiles_reg) begin
// Queue a prefetch for the tile AFTER next into
// the bank we just freed (current_bank, pre-swap)
// -- it will actually launch once the (single)
// prefetch engine is free (see the pf_pending
// issue rule above); it is very likely still
// busy with the tile-N+1 fetch kicked off on the
// PREVIOUS handoff, so this almost always queues
// rather than launching immediately.
// the bank we just freed (current_bank, pre-swap).
pf_pending <= 1'b1;
pf_pending_x <= x_base_reg + (tile_idx + 16'h2) * P_IN[ADDR_WIDTH-1:0];
pf_pending_w <= w_base_reg + (tile_idx + 16'h2) * P_IN[ADDR_WIDTH-1:0];
pf_pending_bank <= current_bank; // the one just freed
xc_pending <= 1'b1;
xc_pending_x_base <= x_base_reg;
xc_pending_tile_idx <= tile_idx + 16'h2;
xc_pending_bank <= current_bank;
end
end else if (!operand_valid) begin
// Waiting for the new current bank to become ready
@@ -322,8 +384,9 @@ module memory_manager #(
MM_WRITE_RESULT: begin
// prefetch_engine is guaranteed idle here (no more
// tiles to fetch for this job), so driving the shared
// backend port directly is safe -- see file header.
// tiles to fetch for this job), so driving the
// shared weight backend port directly is safe --
// see file header/wr_active above.
wr_mem_req <= 1'b1;
wr_mem_addr <= result_addr_reg[ADDR_WIDTH-1:1]; // byte -> word
state <= MM_DONE;