docs: capture two more exploratory scaling directions (BRAM cache, ESP32-side scheduling)
S5.6.1: opportunistic BRAM cache for activation tiles - exploits real, currently 0%-utilized Block RAM to catch whatever locality the workload happens to have, without committing to a specific reuse pattern the way the systolic direction does. No cache-invalidation problem given the current write-once-before-job protocol. S5.6.2: host-side (ESP32) job-queue reordering - a software-only "DDRManager" upstream of ddr_prefetch_mgr.v, grouping jobs with nearby DDR3 addresses before submission to reduce row-switch cost with zero RTL and zero timing-margin risk. Both marked exploratory, not decided, not built - same as S5.6's systolic direction. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUG92aM9m68TRc4rG55BcC
This commit is contained in:
@@ -547,6 +547,72 @@ will tell us whether flat scaling is "good enough" up to some N, making
|
|||||||
this restructuring unnecessary, or whether the real congestion at N=8/16
|
this restructuring unnecessary, or whether the real congestion at N=8/16
|
||||||
justifies it.
|
justifies it.
|
||||||
|
|
||||||
|
#### 5.6.1 [EXPLORATORY] Opportunistic BRAM cache for activation tiles
|
||||||
|
|
||||||
|
Smaller and more incremental than §5.6's systolic restructuring — doesn't
|
||||||
|
require knowing anything about the target network's structure in advance.
|
||||||
|
Artix-7 100T's Block RAM is real and currently **0% utilized** (§3, every
|
||||||
|
real P&R signoff to date) — real, free, unused capacity.
|
||||||
|
|
||||||
|
**The idea**: a small direct-mapped or low-associativity cache, in BRAM,
|
||||||
|
remembering the last few activation tiles fetched from DDR3 (address +
|
||||||
|
data). Before `act_tile_fetch.v` (or `ddr_prefetch_mgr.v`, EXP-0083) issues
|
||||||
|
a real DDR3 request, check the cache first — on a hit (e.g. two jobs, on
|
||||||
|
the same or different slots, requesting overlapping/adjacent tiles, common
|
||||||
|
in convolution with sliding-window overlap), skip the DDR3 round-trip
|
||||||
|
entirely.
|
||||||
|
|
||||||
|
**Why it's attractive**: catches real reuse the design doesn't have to
|
||||||
|
predict or assume in advance — unlike §5.6's systolic chains (which commit
|
||||||
|
to a specific reuse *pattern*, weight-stationary), a cache opportunistically
|
||||||
|
exploits WHATEVER locality the real workload happens to have, including
|
||||||
|
patterns nobody designed for. Composable with everything else already
|
||||||
|
built or decided (§5.1 packing, §5.4 widening, §5.6 systolic groups if that
|
||||||
|
direction is taken) — it's a cache in front of the existing fetch path, not
|
||||||
|
a replacement for it.
|
||||||
|
|
||||||
|
**Real open questions**: cache size vs. real hit rate is workload-dependent
|
||||||
|
and NOT measured — would need a real trace-driven estimate (or a real
|
||||||
|
simulation with representative test data) before sizing it, not guessed.
|
||||||
|
Coherency is simple here (activation data in DDR3 is written once by the
|
||||||
|
host before a job runs and never modified during compute, per the current
|
||||||
|
protocol) — no cache-invalidation problem to solve, a real simplification
|
||||||
|
versus a general-purpose cache design.
|
||||||
|
|
||||||
|
#### 5.6.2 [EXPLORATORY] Host-side (ESP32) job-queue optimization — a "software DDRManager"
|
||||||
|
|
||||||
|
A different kind of lever than anything else in this section: instead of
|
||||||
|
adding hardware intelligence inside the FPGA, exploit the fact that the
|
||||||
|
**ESP32 already has full visibility of the whole job queue before
|
||||||
|
submitting it** — `neural_director_packed.v`'s own `QUEUE_DEPTH=8` hardware
|
||||||
|
queue only sees jobs one at a time as they're written over SPI; the ESP32
|
||||||
|
firmware, upstream of that, could see and reorder ALL pending jobs at once.
|
||||||
|
|
||||||
|
**The idea**: the ESP32's own job-submission firmware groups/reorders jobs
|
||||||
|
before writing them over the management SPI bus (§2.2 of
|
||||||
|
`docs/PHYSICAL_REALIZATION.md`), so that jobs whose DDR3 addresses are
|
||||||
|
close together (same or adjacent rows) are submitted close together in
|
||||||
|
time — directly reducing the real row-switch cost (§3.3) that dominates
|
||||||
|
per-tile latency, WITHOUT any new RTL at all. A real, software-only
|
||||||
|
"DDRManager" living in ESP32 firmware, upstream of and complementary to
|
||||||
|
`ddr_prefetch_mgr.v` (EXP-0083, which only looks ahead within one already-
|
||||||
|
submitted job).
|
||||||
|
|
||||||
|
**Why this is worth capturing seriously, not just as a curiosity**: it's
|
||||||
|
the cheapest possible lever in this whole list — zero RTL, zero real P&R
|
||||||
|
risk, zero timing-margin cost (the project's real margin is thin, §2, and
|
||||||
|
every RTL addition risks it; this doesn't touch RTL at all) — and it's far
|
||||||
|
faster to iterate on than Verilog (the user's own established preference
|
||||||
|
for where complexity is easiest to absorb). It doesn't compete with any
|
||||||
|
other idea in this section — it can be built independently, at any time,
|
||||||
|
by whoever writes the ESP32-side firmware, and composes with all of them.
|
||||||
|
|
||||||
|
**Real open question**: requires the host firmware to know DDR3 addresses
|
||||||
|
well enough to group by row locality (`ROW_BITS`/`COL_BITS`/`BANK_BITS`
|
||||||
|
convention, §2 of `docs/PHYSICAL_REALIZATION.md`) — a real firmware-side
|
||||||
|
design task, not yet scoped, and out of this repository's own RTL scope
|
||||||
|
(ESP32 firmware isn't part of `hardware/v3/`).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 6. Summary table: what's real vs. what's a calculation
|
## 6. Summary table: what's real vs. what's a calculation
|
||||||
|
|||||||
Reference in New Issue
Block a user