docs: capture two more exploratory scaling directions (BRAM cache, ESP32-side scheduling)

S5.6.1: opportunistic BRAM cache for activation tiles - exploits real,
currently 0%-utilized Block RAM to catch whatever locality the workload
happens to have, without committing to a specific reuse pattern the way
the systolic direction does. No cache-invalidation problem given the
current write-once-before-job protocol.

S5.6.2: host-side (ESP32) job-queue reordering - a software-only
"DDRManager" upstream of ddr_prefetch_mgr.v, grouping jobs with nearby
DDR3 addresses before submission to reduce row-switch cost with zero
RTL and zero timing-margin risk. Both marked exploratory, not decided,
not built - same as S5.6's systolic direction.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUG92aM9m68TRc4rG55BcC
This commit is contained in:
2026-09-20 13:22:17 +02:00
co-authored by Claude Sonnet 5
parent 32d32f4c0e
commit 376ccb6ee2
+66
View File
@@ -547,6 +547,72 @@ will tell us whether flat scaling is "good enough" up to some N, making
this restructuring unnecessary, or whether the real congestion at N=8/16 this restructuring unnecessary, or whether the real congestion at N=8/16
justifies it. justifies it.
#### 5.6.1 [EXPLORATORY] Opportunistic BRAM cache for activation tiles
Smaller and more incremental than §5.6's systolic restructuring — doesn't
require knowing anything about the target network's structure in advance.
Artix-7 100T's Block RAM is real and currently **0% utilized** (§3, every
real P&R signoff to date) — real, free, unused capacity.
**The idea**: a small direct-mapped or low-associativity cache, in BRAM,
remembering the last few activation tiles fetched from DDR3 (address +
data). Before `act_tile_fetch.v` (or `ddr_prefetch_mgr.v`, EXP-0083) issues
a real DDR3 request, check the cache first — on a hit (e.g. two jobs, on
the same or different slots, requesting overlapping/adjacent tiles, common
in convolution with sliding-window overlap), skip the DDR3 round-trip
entirely.
**Why it's attractive**: catches real reuse the design doesn't have to
predict or assume in advance — unlike §5.6's systolic chains (which commit
to a specific reuse *pattern*, weight-stationary), a cache opportunistically
exploits WHATEVER locality the real workload happens to have, including
patterns nobody designed for. Composable with everything else already
built or decided (§5.1 packing, §5.4 widening, §5.6 systolic groups if that
direction is taken) — it's a cache in front of the existing fetch path, not
a replacement for it.
**Real open questions**: cache size vs. real hit rate is workload-dependent
and NOT measured — would need a real trace-driven estimate (or a real
simulation with representative test data) before sizing it, not guessed.
Coherency is simple here (activation data in DDR3 is written once by the
host before a job runs and never modified during compute, per the current
protocol) — no cache-invalidation problem to solve, a real simplification
versus a general-purpose cache design.
#### 5.6.2 [EXPLORATORY] Host-side (ESP32) job-queue optimization — a "software DDRManager"
A different kind of lever than anything else in this section: instead of
adding hardware intelligence inside the FPGA, exploit the fact that the
**ESP32 already has full visibility of the whole job queue before
submitting it** — `neural_director_packed.v`'s own `QUEUE_DEPTH=8` hardware
queue only sees jobs one at a time as they're written over SPI; the ESP32
firmware, upstream of that, could see and reorder ALL pending jobs at once.
**The idea**: the ESP32's own job-submission firmware groups/reorders jobs
before writing them over the management SPI bus (§2.2 of
`docs/PHYSICAL_REALIZATION.md`), so that jobs whose DDR3 addresses are
close together (same or adjacent rows) are submitted close together in
time — directly reducing the real row-switch cost (§3.3) that dominates
per-tile latency, WITHOUT any new RTL at all. A real, software-only
"DDRManager" living in ESP32 firmware, upstream of and complementary to
`ddr_prefetch_mgr.v` (EXP-0083, which only looks ahead within one already-
submitted job).
**Why this is worth capturing seriously, not just as a curiosity**: it's
the cheapest possible lever in this whole list — zero RTL, zero real P&R
risk, zero timing-margin cost (the project's real margin is thin, §2, and
every RTL addition risks it; this doesn't touch RTL at all) — and it's far
faster to iterate on than Verilog (the user's own established preference
for where complexity is easiest to absorb). It doesn't compete with any
other idea in this section — it can be built independently, at any time,
by whoever writes the ESP32-side firmware, and composes with all of them.
**Real open question**: requires the host firmware to know DDR3 addresses
well enough to group by row locality (`ROW_BITS`/`COL_BITS`/`BANK_BITS`
convention, §2 of `docs/PHYSICAL_REALIZATION.md`) — a real firmware-side
design task, not yet scoped, and out of this repository's own RTL scope
(ESP32 firmware isn't part of `hardware/v3/`).
--- ---
## 6. Summary table: what's real vs. what's a calculation ## 6. Summary table: what's real vs. what's a calculation