diff --git a/docs/ARCHITECTURE_ANALYSIS.md b/docs/ARCHITECTURE_ANALYSIS.md index ba6f28d..0c5a75c 100644 --- a/docs/ARCHITECTURE_ANALYSIS.md +++ b/docs/ARCHITECTURE_ANALYSIS.md @@ -547,6 +547,72 @@ will tell us whether flat scaling is "good enough" up to some N, making this restructuring unnecessary, or whether the real congestion at N=8/16 justifies it. +#### 5.6.1 [EXPLORATORY] Opportunistic BRAM cache for activation tiles + +Smaller and more incremental than §5.6's systolic restructuring — doesn't +require knowing anything about the target network's structure in advance. +Artix-7 100T's Block RAM is real and currently **0% utilized** (§3, every +real P&R signoff to date) — real, free, unused capacity. + +**The idea**: a small direct-mapped or low-associativity cache, in BRAM, +remembering the last few activation tiles fetched from DDR3 (address + +data). Before `act_tile_fetch.v` (or `ddr_prefetch_mgr.v`, EXP-0083) issues +a real DDR3 request, check the cache first — on a hit (e.g. two jobs, on +the same or different slots, requesting overlapping/adjacent tiles, common +in convolution with sliding-window overlap), skip the DDR3 round-trip +entirely. + +**Why it's attractive**: catches real reuse the design doesn't have to +predict or assume in advance — unlike §5.6's systolic chains (which commit +to a specific reuse *pattern*, weight-stationary), a cache opportunistically +exploits WHATEVER locality the real workload happens to have, including +patterns nobody designed for. Composable with everything else already +built or decided (§5.1 packing, §5.4 widening, §5.6 systolic groups if that +direction is taken) — it's a cache in front of the existing fetch path, not +a replacement for it. + +**Real open questions**: cache size vs. real hit rate is workload-dependent +and NOT measured — would need a real trace-driven estimate (or a real +simulation with representative test data) before sizing it, not guessed. +Coherency is simple here (activation data in DDR3 is written once by the +host before a job runs and never modified during compute, per the current +protocol) — no cache-invalidation problem to solve, a real simplification +versus a general-purpose cache design. + +#### 5.6.2 [EXPLORATORY] Host-side (ESP32) job-queue optimization — a "software DDRManager" + +A different kind of lever than anything else in this section: instead of +adding hardware intelligence inside the FPGA, exploit the fact that the +**ESP32 already has full visibility of the whole job queue before +submitting it** — `neural_director_packed.v`'s own `QUEUE_DEPTH=8` hardware +queue only sees jobs one at a time as they're written over SPI; the ESP32 +firmware, upstream of that, could see and reorder ALL pending jobs at once. + +**The idea**: the ESP32's own job-submission firmware groups/reorders jobs +before writing them over the management SPI bus (§2.2 of +`docs/PHYSICAL_REALIZATION.md`), so that jobs whose DDR3 addresses are +close together (same or adjacent rows) are submitted close together in +time — directly reducing the real row-switch cost (§3.3) that dominates +per-tile latency, WITHOUT any new RTL at all. A real, software-only +"DDRManager" living in ESP32 firmware, upstream of and complementary to +`ddr_prefetch_mgr.v` (EXP-0083, which only looks ahead within one already- +submitted job). + +**Why this is worth capturing seriously, not just as a curiosity**: it's +the cheapest possible lever in this whole list — zero RTL, zero real P&R +risk, zero timing-margin cost (the project's real margin is thin, §2, and +every RTL addition risks it; this doesn't touch RTL at all) — and it's far +faster to iterate on than Verilog (the user's own established preference +for where complexity is easiest to absorb). It doesn't compete with any +other idea in this section — it can be built independently, at any time, +by whoever writes the ESP32-side firmware, and composes with all of them. + +**Real open question**: requires the host firmware to know DDR3 addresses +well enough to group by row locality (`ROW_BITS`/`COL_BITS`/`BANK_BITS` +convention, §2 of `docs/PHYSICAL_REALIZATION.md`) — a real firmware-side +design task, not yet scoped, and out of this repository's own RTL scope +(ESP32 firmware isn't part of `hardware/v3/`). + --- ## 6. Summary table: what's real vs. what's a calculation