michele/FPGA-Neural
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

perf(v2): shared activation cache - further 1.66-2.00x real speedup (DEC-0016)

Browse Source 
Implements optimization #2 from the final benchmark campaign's own
recommendation, on top of DEC-0015's word-level burst rewrite: a new
shared activation_cache.v module fetches a given activation (X)
vector from PSRAM once instead of once per neuron sharing it - the
exact redundant traffic pattern the dense-layer workloads in this
project's benchmark suite exhibit.

Each memory_manager's own prefetch_engine now fetches WEIGHTS only;
the activation half is requested from the shared cache instead
(single-tag, tile-granular, N_SLOTS request ports, its own real
word-level PSRAM backend via a new dedicated arbiter port).
dataflow_core.v/slot_mem_arbiter.v/neural_multiprocessor.v widened to
N_SLOTS+1 ports to arbitrate the cache's traffic alongside each
slot's weight traffic.

Two real bugs found and fixed during implementation (ERR-0010): a
target-bank/pending-bank race in memory_manager.v's activation-cache
wiring (the same bug class ERR-0006 already fixed once for
pf_target_bank - a later handoff's queued request can overwrite which
bank an earlier, still-in-flight request's ack applies to), and a
repeat of ERR-0009's N_SLOTS=1 zero-width replication bug in
activation_cache.v itself.

Real, measured results: the full final-benchmark campaign (24/24
workload/config combinations) re-verified bit-exact. D-Stress cycles
fall a further 1.66-2.00x on top of DEC-0015 (~4x combined vs the
original byte-level baseline). But the cache's real Fmax cost is much
steeper than DEC-0015's own: N_SLOTS=2 (the recommended default,
DEC-0014) drops from 133.58 to 87.72 MHz (-34%, margin over 80MHz
shrinks from +67% to +9.7%), and N_SLOTS=4 drops to 65.01 MHz - now
FAILING the 80MHz target it previously passed. Combined real
wall-clock speedup vs the original baseline: N=1 3.86x, N=2 2.45x
(both real net wins); N=4 is a real regression once its own now-failing
Fmax is honestly used, though N=4 was never the recommended
configuration.

N_SLOTS=2 remains the recommended default (DEC-0014 unaffected) with
a thinner but still real Fmax margin. Cache hit-detection pipelining
is flagged as concrete follow-up work if N_SLOTS>2 is ever needed with
the cache active - not attempted this round.

Logged: simulation/synthesis/timing/benchmark/decisions (DEC-0016)/
experiments (EXP-0016)/errors (ERR-0010)/development.log, ROADMAP.md
updated.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013xXuuRUWZScuo1DeYJxs3v
This commit is contained in:
michele
2026-09-05 20:59:14 +02:00
co-authored by Claude Sonnet 5
parent e4a5540b6e
commit 63cac6a7e5
16 changed files with 931 additions and 211 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hardware/v2/rtl/prefetch_engine.v
+36 -61
View File
@@ -1,43 +1,44 @@
`timescale 1ns/1ps
// ================================================================
// FPGA-Neural V2 -- Prefetch Engine (M4, docs/v2-description.md §13;
// word-level burst rewrite post-M10 -- see hardware/v2/logs/
// decisions.log DEC-0015)
// FPGA-Neural V2 -- Weight Prefetch Engine (M4, docs/v2-description.md
// §13; word-level burst rewrite post-M10 DEC-0015; X-fetch moved out
// to a shared activation_cache.v post-M10 DEC-0016)
//
// Fetches ONE tile (P_IN activation bytes + P_IN weight bytes) from
// the WORD-level Memory Backend Interface, P_IN/2 sixteen-bit
// transactions per array instead of P_IN single-byte ones.
// Fetches ONE tile's P_IN WEIGHT bytes from the WORD-level Memory
// Backend Interface, P_IN/2 sixteen-bit transactions instead of P_IN
// single-byte ones (DEC-0015 -- see this rationale in full below).
//
// WHY: hardware/v1/rtl/int8_memory_access.v (the byte-level backend
// this engine originally sat on) converts every 8-bit logical request
// into a FULL 16-bit PSRAM word access internally (mem_addr <= addr
// >> 1, one byte lane selected via lb_n/ub_n) -- so a byte-at-a-time
// fetch was ALREADY paying for two bytes of real PSRAM bandwidth per
// transaction while only using one. This engine now talks directly to
// hardware/v1/rtl/memory_interface.v's own 16-bit word interface
// (skipping int8_memory_access.v entirely -- both are frozen V1 files,
// unmodified either way, §1/§34; V2 is simply choosing to reuse the
// lower layer instead of the byte-splitting one on top of it, the
// same "reuse what fits" precedent already set by slot_mem_arbiter.v
// not reusing hardware/v1/rtl/mem_arbiter.v verbatim). psram_controller.v's
// own real page-mode support (already implemented, unmodified) then
// serves consecutive same-page word reads faster than a cold access --
// this engine's job is simply to stop discarding half of every word it
// already paid for, and to halve the number of real backend
// round-trips needed per tile.
// Historical note: this module used to ALSO fetch the P_IN
// ACTIVATION (X) bytes for the same tile. DEC-0016 moved that
// responsibility to a new shared activation_cache.v instead: in the
// realistic dense-layer workloads this project actually benchmarks
// (hardware/v2/docs/benchmarks/final-benchmark.md), many neurons
// share the exact same X vector, and each of memory_manager.v's own
// N_SLOTS instances re-fetching that identical vector from PSRAM
// independently was real, measured, redundant traffic on the one
// shared PSRAM port -- exactly the kind of real recommendation the
// benchmark campaign was built to surface. Weights (W) are NOT shared
// across neurons (each neuron has its own trained weight vector), so
// there is no equivalent caching opportunity on the W side -- this
// engine keeps fetching W directly from PSRAM, unchanged in spirit
// from DEC-0015, just no longer also fetching X.
//
// CONSTRAINT: P_IN must be even, and x_addr/w_addr must be word-
// aligned (even BYTE addresses) -- each 16-bit transaction covers
// BYTE addresses {addr, addr+1} as {low byte, high byte} (matches
// int8_memory_access.v's own addr[0] convention exactly, replicated
// here since that module is no longer in the datapath). A host/loader
// placing X/W tile arrays at even byte offsets (already true of every
// address used in this project's own testbenches) satisfies this
// with no special handling.
// WHY word-level (DEC-0015, unchanged rationale): int8_memory_access.v
// (the byte-level backend this engine originally sat on) converts
// every 8-bit logical request into a FULL 16-bit PSRAM word access
// internally (mem_addr <= addr >> 1, one byte lane selected via
// lb_n/ub_n) -- so a byte-at-a-time fetch was ALREADY paying for two
// bytes of real PSRAM bandwidth per transaction while only using one.
// This engine talks directly to hardware/v1/rtl/memory_interface.v's
// own 16-bit word interface (skipping int8_memory_access.v entirely --
// both are frozen V1 files, unmodified either way, §1/§34).
//
// The double-buffering strategy itself (§13) remains memory_manager.v's
// responsibility -- unchanged by this rewrite.
// CONSTRAINT: P_IN must be even, and w_addr must be word-aligned (even
// BYTE address) -- each 16-bit transaction covers BYTE addresses
// {addr, addr+1} as {low byte, high byte} (matches int8_memory_access.v's
// own addr[0] convention exactly, replicated here since that module is
// no longer in the datapath).
// ================================================================
module prefetch_engine #(
@@ -49,11 +50,9 @@ module prefetch_engine #(
input wire rst,
input wire fetch_start,
input wire [ADDR_WIDTH-1:0] x_addr, // BYTE address, word-aligned
input wire [ADDR_WIDTH-1:0] w_addr, // BYTE address, word-aligned
output reg fetch_busy,
output reg fetch_done, // one-cycle pulse
output reg signed [DATA_WIDTH*P_IN-1:0] tile_x,
output reg signed [DATA_WIDTH*P_IN-1:0] tile_w,
// ---- word-level Memory Backend Interface (matches
@@ -69,7 +68,6 @@ module prefetch_engine #(
);
localparam ST_IDLE = 2'd0;
localparam ST_READ_X = 2'd1;
localparam ST_READ_W = 2'd2;
localparam ST_DONE = 2'd3;
@@ -79,7 +77,6 @@ module prefetch_engine #(
reg [1:0] state;
reg [WIW-1:0] word_idx;
wire [ADDR_WIDTH-1:0] x_word_base = x_addr[ADDR_WIDTH-1:1];
wire [ADDR_WIDTH-1:0] w_word_base = w_addr[ADDR_WIDTH-1:1];
always @(posedge clk) begin
@@ -106,32 +103,10 @@ module prefetch_engine #(
word_idx <= 0;
mem_req <= 1'b1;
mem_wr <= 1'b0;
mem_addr <= x_word_base;
mem_addr <= w_word_base;
mem_lb_n <= 1'b0; // both byte lanes -- fetch the whole word
mem_ub_n <= 1'b0;
state <= ST_READ_X;
end
end
ST_READ_X: begin
if (mem_ready) begin
tile_x[word_idx*16 +: 16] <= mem_rdata;
if (word_idx == WORDS_PER_TILE[WIW-1:0] - 1'b1) begin
word_idx <= 0;
mem_req <= 1'b1;
mem_wr <= 1'b0;
mem_addr <= w_word_base;
mem_lb_n <= 1'b0;
mem_ub_n <= 1'b0;
state <= ST_READ_W;
end else begin
word_idx <= word_idx + 1'b1;
mem_req <= 1'b1;
mem_wr <= 1'b0;
mem_addr <= x_word_base + word_idx + 1'b1;
mem_lb_n <= 1'b0;
mem_ub_n <= 1'b0;
end
state <= ST_READ_W;
end
end