From 376ccb6ee2fe06c88c9a4d883f9ab1ef81dd78a3 Mon Sep 17 00:00:00 2001 From: manvalan Date: Sun, 20 Sep 2026 13:22:17 +0200 Subject: [PATCH] docs: capture two more exploratory scaling directions (BRAM cache, ESP32-side scheduling) S5.6.1: opportunistic BRAM cache for activation tiles - exploits real, currently 0%-utilized Block RAM to catch whatever locality the workload happens to have, without committing to a specific reuse pattern the way the systolic direction does. No cache-invalidation problem given the current write-once-before-job protocol. S5.6.2: host-side (ESP32) job-queue reordering - a software-only "DDRManager" upstream of ddr_prefetch_mgr.v, grouping jobs with nearby DDR3 addresses before submission to reduce row-switch cost with zero RTL and zero timing-margin risk. Both marked exploratory, not decided, not built - same as S5.6's systolic direction. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01MUG92aM9m68TRc4rG55BcC --- docs/ARCHITECTURE_ANALYSIS.md | 66 +++++++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) diff --git a/docs/ARCHITECTURE_ANALYSIS.md b/docs/ARCHITECTURE_ANALYSIS.md index ba6f28d..0c5a75c 100644 --- a/docs/ARCHITECTURE_ANALYSIS.md +++ b/docs/ARCHITECTURE_ANALYSIS.md @@ -547,6 +547,72 @@ will tell us whether flat scaling is "good enough" up to some N, making this restructuring unnecessary, or whether the real congestion at N=8/16 justifies it. +#### 5.6.1 [EXPLORATORY] Opportunistic BRAM cache for activation tiles + +Smaller and more incremental than §5.6's systolic restructuring — doesn't +require knowing anything about the target network's structure in advance. +Artix-7 100T's Block RAM is real and currently **0% utilized** (§3, every +real P&R signoff to date) — real, free, unused capacity. + +**The idea**: a small direct-mapped or low-associativity cache, in BRAM, +remembering the last few activation tiles fetched from DDR3 (address + +data). Before `act_tile_fetch.v` (or `ddr_prefetch_mgr.v`, EXP-0083) issues +a real DDR3 request, check the cache first — on a hit (e.g. two jobs, on +the same or different slots, requesting overlapping/adjacent tiles, common +in convolution with sliding-window overlap), skip the DDR3 round-trip +entirely. + +**Why it's attractive**: catches real reuse the design doesn't have to +predict or assume in advance — unlike §5.6's systolic chains (which commit +to a specific reuse *pattern*, weight-stationary), a cache opportunistically +exploits WHATEVER locality the real workload happens to have, including +patterns nobody designed for. Composable with everything else already +built or decided (§5.1 packing, §5.4 widening, §5.6 systolic groups if that +direction is taken) — it's a cache in front of the existing fetch path, not +a replacement for it. + +**Real open questions**: cache size vs. real hit rate is workload-dependent +and NOT measured — would need a real trace-driven estimate (or a real +simulation with representative test data) before sizing it, not guessed. +Coherency is simple here (activation data in DDR3 is written once by the +host before a job runs and never modified during compute, per the current +protocol) — no cache-invalidation problem to solve, a real simplification +versus a general-purpose cache design. + +#### 5.6.2 [EXPLORATORY] Host-side (ESP32) job-queue optimization — a "software DDRManager" + +A different kind of lever than anything else in this section: instead of +adding hardware intelligence inside the FPGA, exploit the fact that the +**ESP32 already has full visibility of the whole job queue before +submitting it** — `neural_director_packed.v`'s own `QUEUE_DEPTH=8` hardware +queue only sees jobs one at a time as they're written over SPI; the ESP32 +firmware, upstream of that, could see and reorder ALL pending jobs at once. + +**The idea**: the ESP32's own job-submission firmware groups/reorders jobs +before writing them over the management SPI bus (§2.2 of +`docs/PHYSICAL_REALIZATION.md`), so that jobs whose DDR3 addresses are +close together (same or adjacent rows) are submitted close together in +time — directly reducing the real row-switch cost (§3.3) that dominates +per-tile latency, WITHOUT any new RTL at all. A real, software-only +"DDRManager" living in ESP32 firmware, upstream of and complementary to +`ddr_prefetch_mgr.v` (EXP-0083, which only looks ahead within one already- +submitted job). + +**Why this is worth capturing seriously, not just as a curiosity**: it's +the cheapest possible lever in this whole list — zero RTL, zero real P&R +risk, zero timing-margin cost (the project's real margin is thin, §2, and +every RTL addition risks it; this doesn't touch RTL at all) — and it's far +faster to iterate on than Verilog (the user's own established preference +for where complexity is easiest to absorb). It doesn't compete with any +other idea in this section — it can be built independently, at any time, +by whoever writes the ESP32-side firmware, and composes with all of them. + +**Real open question**: requires the host firmware to know DDR3 addresses +well enough to group by row locality (`ROW_BITS`/`COL_BITS`/`BANK_BITS` +convention, §2 of `docs/PHYSICAL_REALIZATION.md`) — a real firmware-side +design task, not yet scoped, and out of this repository's own RTL scope +(ESP32 firmware isn't part of `hardware/v3/`). + --- ## 6. Summary table: what's real vs. what's a calculation