# FPGA-Neural — ESP32 Host Firmware Specification Real, from-source specification for the ESP32-S3 host firmware, derived directly from the actual RTL protocol implementation (`spi_host_bridge_v3.v`, `neural_director_grouped.v`, `result_writeback.v`, `host_mem_bridge.v`, `flash_spi_master.v`) and the real, closed timing signoff in `docs/PHYSICAL_REALIZATION.md`. Nothing below is guessed — every field layout, address formula, and timing number is read directly from the committed RTL's own header comments or real, measured project data. **Status**: this is a specification only. No ESP32-side firmware exists yet in this repository — this document exists to make that firmware buildable without re-deriving the protocol from RTL by hand. Real, disclosed scope: management SPI protocol, job submission, result readback, raw memory access, config-flash relay, boot/bootstrap. Out of scope: FreeRTOS task structure, Wi-Fi/network layer, higher-level model-graph scheduling above "submit one job" — those are real firmware design decisions, not yet made anywhere in this project. **Real fabrication target (2026-09-22)**: N=16 (`n16_system_ddr3_top.v`, 16 real parallel processing elements, 4 groups × 4 PEs). This document is written for N=16 throughout; N=2/N=8 fallback configurations use the byte-for-byte identical protocol (§3), differing only in the real octet/pairing requirement of §4.2. --- ## 1. Physical interfaces Two electrically separate SPI buses, plus JTAG. Full real pin assignments: `docs/PINOUT.md` / `docs/PHYSICAL_REALIZATION.md` §2. | Bus | FPGA role | Purpose | Pins (bank 15/16, LVCMOS33) | |---|---|---|---| | Management SPI | **slave** | Job submission, status, raw memory access, config-flash relay | sclk=A15, mosi=B16, miso=B17, cs_n=A16 | | Config-flash SPI | **master** (relayed) | Bitstream/config flash access, reached only via `FLASH_XFER` (§6) | not directly host-visible — internal to the FPGA | | JTAG | target | Factory/recovery programming | TCK=E10, TDI=E11, TMS=E12, TDO=E13 | Additional real, dedicated signals on the management SPI bus's own bank: | Signal | Pin | Direction (ESP32 side) | Purpose | |---|---|---|---| | `sys_rst` | G13 (tentative real pin, §7 of PHYSICAL_REALIZATION.md) | output | Board-level reset, separate from the SPI-driven soft-reset (§3) | | `data_ready_n` | D14 | input, active-low | Sticky IRQ — see §5.1 | **Real, important note**: the ESP32 has **no direct electrical connection** to the config flash. The only path is `FLASH_XFER` (§6), relayed through the FPGA over the management SPI bus. --- ## 2. Boot / bootstrap sequence Three real, distinct paths (`docs/PHYSICAL_REALIZATION.md` §5): 1. **Factory-first / recovery (JTAG, ESP32-driven, one-time or rare)** — a factory-fresh board has a **blank** config flash. The FPGA cannot relay `FLASH_XFER` commands until it is already running logic that does that — a real chicken-and-egg constraint. The only way to bootstrap a blank board is for the ESP32 to bit-bang JTAG (TAP state machine, IR/DR shifting) and either (a) load a bitstream directly into the FPGA's configuration memory (volatile — lost on power-cycle), or (b) run Vivado's own real "indirect SPI flash programming" sequence over JTAG to write the flash for the first time. **Real, disclosed gap: this JTAG bit-banging firmware does not exist yet anywhere in this project** — it is real, necessary, unbuilt software work, out of scope for this document beyond flagging it. 2. **Normal boot (Master SPI, autonomous, every power-on)** — the FPGA self-configures from the flash via its own dedicated configuration hardware (mode pins `M[2:0]=001`, §2.4 of PHYSICAL_REALIZATION.md). **No ESP32 involvement at all.** Firmware's own real startup sequence should simply wait for `init_calib_complete` (readable via `REG_READ` on STATUS, §3.3, bit3) before issuing any job or memory access — DDR3 traffic before real PHY calibration completes is unsafe. 3. **Field firmware update (`FLASH_XFER`, §6)** — once the FPGA is already running (path 2 completed), the ESP32 can rewrite the config flash by relaying raw SPI-NOR bytes through the FPGA. This is the **only** path from ESP32 to flash during normal operation. ### 2.1 Real, recommended firmware startup sequence ``` 1. Assert sys_rst (G13), hold >= a few clk periods, release. (Real note: sys_rst's own real, permanent board location is still tentative per PHYSICAL_REALIZATION.md §7 -- confirm against the real board schematic before relying on this pin number.) 2. Wait for the FPGA's own real Master-SPI autonomous boot to complete -- there is no host-visible "boot done" signal separate from init_calib_complete; DONE (pin P10) is a real, physical configuration-status pin if the ESP32 has a GPIO wired to it (board-dependent, not guaranteed). 3. REG_READ(0x00 DEVICE_ID) -- confirm 32'h4E50_5601 ("NPV" + protocol version 1) before trusting anything else. A mismatch means either the flash holds the wrong bitstream, or config-boot has not actually completed -- do not proceed. 4. Poll REG_READ(0x02 STATUS) bit3 (init_calib_complete) until set. Real, measured DDR3 calibration timing has not been characterized on real hardware yet (only in simulation, where SIM_BYPASS_INIT_CAL shortcuts it) -- budget real time for this, do not assume it is instantaneous. 5. Only once bit3 is set: safe to issue WRITE_MEM (weight/activation preload, SS3) and WRITE_JOB (SS5). ``` --- ## 3. Management SPI protocol — real opcode reference One opcode byte (MSB-first), sent as the first byte of a CS-low SPI transaction; multi-byte fields are MSB-first. Authoritative source: `spi_host_bridge_v3.v`'s own header comment — this table is a summary. | Opcode | Name | Direction | Payload (MOSI) | Response (MISO) | |---|---|---|---|---| | `0x00` | NOP | — | 0 bytes | inert (0x00) | | `0x0F` | RESET | host→FPGA | 0 bytes | pulses `soft_rst_pulse` for 1 real clk cycle after CS rises | | `0x10` | WRITE_JOB | host→FPGA | 16 bytes (§4.1) | — | | `0x20` | STATUS | host→FPGA | 0 bytes | 1 byte (§3.1) | | `0x01` | WRITE_MEM | host→FPGA | 4 + 2·N bytes (§6.1) | — | | `0x02` | READ_MEM | host→FPGA | 6 bytes | 2·N bytes (§6.1) | | `0x30` | REG_WRITE | host→FPGA | 5 bytes (§3.2) | — | | `0x31` | REG_READ | host→FPGA | 1 byte | 4 bytes (§3.3) | | `0x40` | FLASH_XFER | bidir | N bytes | N+2 bytes (§6.2) | **Real, load-bearing rule**: any opcode byte not in this table is treated as NOP — the bus is never wedged by an unrecognized command. Do not rely on this for real protocol negotiation; it exists purely as a safety fallback. ### 3.1 `0x20` STATUS — 1 response byte | Bit | Meaning | |---|---| | 0 | `job_busy` — a `WRITE_JOB` is currently waiting on `job_in_ready` | | 1 | `mem_busy` — a `WRITE_MEM`/`READ_MEM` is waiting on `mem_ready` | | 2 | `last_job_accepted` (sticky, cleared by the next `WRITE_JOB`) | | 7:3 | reserved, always 0 | ### 3.2 `0x30` REG_WRITE / `0x31` REG_READ `REG_WRITE`: `byte0=reg_addr[7:0]`, `byte1:4=value[31:0]` MSB-first — applied the instant the last byte lands (purely internal, no backend handshake). `REG_READ`: `byte0=reg_addr[7:0]`, then 4 response bytes MSB-first. An unmapped address reads back `0xFFFFFFFF` (deliberately distinct from a real `0`, so firmware can tell "unmapped register" from "real zero"). Writing a read-only or unmapped register is inert (accepted, no effect). ### 3.3 Real register map (v1) | Addr | Name | R/W | Real meaning | |---|---|---|---| | `0x00` | DEVICE_ID | RO | `32'h4E50_5601` ("NPV" + protocol version 1) | | `0x01` | CONTROL | RW | bit0: write 1 to pulse `soft_rst_pulse` (same physical effect as `0x0F`); always reads back 0; bits[31:1] reserved | | `0x02` | STATUS | RO | bit0 `job_busy`, bit1 `mem_busy`, bit2 `last_job_accepted`, bit3 `init_calib_complete`, bit4 `dir_error`; bits[31:5] reserved | | `0x03` | N_SLOTS | RO | real, total parallel-PE count this bitstream was built with (**16** for the current N=16 fabrication target) — read this instead of hardcoding the core count in firmware | --- ## 4. Job submission ### 4.1 `0x10` WRITE_JOB — real 16-byte payload | Bytes | Field | Real convention | |---|---|---| | 0:1 | `node_id[15:0]` | host-assigned identifier, returned unchanged with the result (§5) | | 2:5 | `x_base[25:0]` | byte-address convention (26-bit `JOB_ADDR_WIDTH`); byte2's MSBs are `{6'b0, x_base[25:24]}` | | 6:9 | `w_base[25:0]` | same convention — **must match, together with `n_tiles`, across the real octet requirement of §4.2** | | 10:11 | `n_tiles[15:0]` | number of `P_IN=8`-wide tiles to stream (real activation length = `n_tiles × 8` bytes) | | 12:15 | `result_addr[25:0]` | where `result_writeback.v` will write the real result — see §5 for the real readback formula | `job_in_valid` is asserted and **held** until `job_in_ready` also reads 1 in the same real cycle — never a blind, unacknowledged pulse. Firmware does not need to do anything special for this; it is entirely internal to the bridge. ### 4.2 REAL, CRITICAL N=16 requirement: octet dispatch **This is the single most important real constraint in this document.** `neural_director_grouped.v` (the real N=16 job scheduler) only dispatches jobs in **groups of 8** — it inspects the **8 oldest** entries in its own internal queue and requires **all eight** to share the exact same `w_base` **and** `n_tiles`. If they do, they are dispatched together (one octet = one real systolic group's own job, weight fetched once, broadcast to 4 PEs × 2 lanes). If they do not, **the queue stalls — permanently, with no in-band recovery.** Real, verified behavior (`tb_neural_director_grouped.v`): a mismatched octet does not corrupt anything and does not silently mis-pair jobs — it simply stops draining the queue. The **only** real recovery is a hardware reset (`sys_rst`, or `CONTROL` register bit0 / `RESET` opcode) — there is no software-visible "skip this job" or "flush the queue" command. **Real, mandatory firmware discipline**: submit exactly 8 `WRITE_JOB` transactions sharing the same `w_base` and `n_tiles`, back-to-back, with no other job submitted in between, for every real layer-position batch. If a real workload's position count is not a multiple of 8, **pad the batch** (e.g. repeat the last real position, or submit a real dummy position with a `result_addr` the firmware will simply ignore) rather than submit a short, mismatched batch — the RTL has no concept of a "partial octet." **Real, deliberate, unchanged property**: the `WRITE_JOB` payload itself is byte-for-byte identical whether the octet requirement applies (N=16, N=8's own pair-of-4 requirement) or not (N=2's own simple pairing) — only the batch **size** firmware must submit before a batch reliably drains differs (8 for N=16, 2 for N=2). Always read `N_SLOTS` (§3.3) at startup and derive the real required batch size as `N_SLOTS / 2` rather than hardcoding 8. --- ## 5. Result readback Results are **not** returned over SPI directly — they are written by `result_writeback.v` into DDR3 at the job's own `result_addr`, and must be read back via the existing `READ_MEM` (`0x02`) opcode. No new opcode exists or is needed for this. **Real, exact addressing formula** (from `result_writeback.v`'s own header — `result_addr` here is the 26-bit value submitted in the job's own `WRITE_JOB` payload, §4.1): ``` mem_addr (value) = result_addr[24:0] * 2 mem_addr (node_id) = result_addr[24:0] * 2 + 1 ``` Two `READ_MEM` transactions per real result lane (one job produces **two** results, lane A and lane B — see `packed_pe.v`'s own real A/B lane structure): - `mem_addr` above reads back a 16-bit word: low byte = the real INT8 result value (`result_data`, zero-extended into the low 8 bits of the word); the other bytes are `8'h00`. - `mem_addr+1` reads back the real 16-bit `node_id` the job was submitted with — use this to match a result back to the job that produced it, since results may complete out of submission order across different groups. **Real, disclosed firmware design question, not yet decided at the RTL level**: how the host knows a specific result is ready (vs. polling `READ_MEM` speculatively) is answered by §5.1 below (`data_ready_n`), but `data_ready_n` is a single, whole-device sticky flag, not per-job — firmware must track its own real outstanding-job count/set and read back results as they become plausible, not per-job-precise from the hardware alone. ### 5.1 `data_ready_n` — real, sticky, active-low interrupt Real, board-level pin (D14). Lets firmware be interrupt-driven instead of polling `STATUS` in a loop. - **Asserted (driven low)** when a real job/octet completes (`job_out_done`) **or** while `dir_error` is active. - **Sticky**: the job-completion contribution stays low even after the underlying completion pulse ends, until the host acknowledges by completing a real `STATUS` (`0x20`) **or** `REG_READ` of STATUS (`0x31`, reg `0x02`) transaction. A `REG_READ` of any *other* register does **not** acknowledge it. - The `dir_error` contribution is **live**, not latched — it clears the instant `dir_error` itself does (typically only after a real reset, since the Director has no in-band error-clear per §4.2). **Real, recommended firmware pattern**: wire `data_ready_n` to an ESP32 GPIO interrupt; on falling edge, read `STATUS` (acknowledges the sticky latch and tells firmware whether it was a real completion or a `dir_error` condition), then read back whichever real results are now plausible. --- ## 6. Raw memory access (weight/activation preload, debug) ### 6.1 `0x01` WRITE_MEM / `0x02` READ_MEM **Real, different address convention from `WRITE_JOB`**: these opcodes use `host_mem_bridge.v`'s own **16-bit-WORD** address space (`MEM_ADDR_WIDTH=25` bits), not the 26-bit byte-base convention `x_base`/`w_base`/`result_addr` use. Do not mix the two conventions. - `WRITE_MEM`: `byte0:3 = addr[24:0]` (word address; `byte0`'s MSBs are `{7'b0, addr[24]}`), then `len_words` (2 bytes, right after addr) × 2 payload bytes, MSB-first per word. Each word is written via one real `mem_req`/`mem_ready` handshake before the next word's bytes are accepted — a real, sequential, per-word transaction, not a burst. - `READ_MEM`: same 6-byte header (4 addr + 2 `len_words`), 0 further MOSI payload; `2·len_words` response bytes clocked out on MISO starting at payload byte 7, MSB-first per word. **Real, disclosed limitation** (not a bug, a deliberate, documented design choice, `host_mem_bridge.v`'s own header): this path halves the host's own reachable byte range for a given `ADDR_WIDTH`, since the underlying shared bus is 32-bit-word-native (EXP-0084's real DDR3 widening) while this host-facing contract stays a fixed 16-bit word — `mem_addr`'s own LSB additionally selects which 16-bit half of the real 32-bit ctrl-bus word to target. Not currently a practical constraint at this project's real usage scale. **Real use**: preloading weight/activation data into DDR3 before submitting jobs that reference it. See `docs/PHYSICAL_REALIZATION.md` §4 for the real, exact DDR3 memory layout convention (weight-per-layer addressing, 4-tiles-per-burst activation packing) firmware must follow when computing `x_base`/`w_base` values for `WRITE_JOB`. ### 6.2 `0x40` FLASH_XFER — config-flash relay Raw, byte-for-byte SPI passthrough to the FPGA's own config flash. The FPGA relays every MOSI byte it receives onto the physical flash's own MOSI line, and relays whatever the flash returns back on MISO. **The FPGA knows nothing about SPI-NOR command semantics** — firmware is responsible for sending a real, complete flash command sequence, exactly as if directly wired to the flash. **Real SPI-NOR opcodes** (Winbond W25Q32JV, verified against the actual datasheet): | Opcode | Command | |---|---| | `0x06` | Write Enable | | `0x04` | Write Disable | | `0x05` | Read Status Register-1 (bit0=BUSY, bit1=WEL) | | `0x02` | Page Program | | `0x03` | Read Data | | `0x20` | Sector Erase (4KB) | | `0x52` | 32KB Block Erase | | `0xD8` | 64KB Block Erase | | `0xC7` / `0x60` | Chip Erase | **Real, critical timing requirement** (measured via simulation, `spi_host_bridge_v3.v`'s own header, EXP-0077): a relayed byte's real flash response only becomes stable starting the transfer's own **byte N+2** — one extra host-clocked byte is not enough (`flash_spi_ master.v`'s own internal transfer, ~640ns at 155.039MHz/DIV=4, starts only once byte N is fully received, overlapping byte N+1's own window). Firmware **must** clock **two** trailing dummy bytes at the end of a `FLASH_XFER` transaction to safely receive the final real response — one dummy byte is a real, previously-reproduced bug, not a hypothetical concern. **Real update procedure**: after writing a new bitstream to the flash via `FLASH_XFER`, reconfigure either by pulsing `PROGRAM_B` externally (if the ESP32 has a GPIO wired to it — board-dependent), or (real, disclosed future work, not built) via an `ICAPE2`-based warm self-reconfiguration triggered over the same SPI bus. --- ## 7. Error handling `dir_error` (readable via `STATUS`, contributes to `data_ready_n`) is the only real, hardware-visible error signal. It is a **latch**, not a transient flag — per `neural_director_grouped.v`'s own real, disclosed design (§4.2), there is **no in-band recovery**: once set (e.g. by a real octet-mismatch stall), only a hardware reset clears it. **Real, recommended firmware policy**: treat `dir_error` as fatal to the current batch of in-flight jobs — issue a real reset (`RESET` opcode or `CONTROL` register bit0), re-establish `DEVICE_ID`/`STATUS` sanity (§2.1 steps 3–4), and re-submit any real jobs that were in-flight at the time of the error, since the Director's own internal queue state is not preserved across a reset. --- ## 8. Real, disclosed open items - No ESP32-side firmware exists yet in this repository — this document is the specification, not an implementation. - JTAG bit-banging firmware (§2, path 1) is real, necessary, unbuilt software. - Real DDR3 calibration timing (§2.1 step 4) has not been measured on real hardware — only in simulation with `SIM_BYPASS_INIT_CAL`. - Whether/how the ESP32 firmware tracks outstanding jobs per-node_id (§5) — a real firmware design decision, not yet made. - `ICAPE2`-based warm self-reconfiguration (§6.2) is disclosed future work, not built. - `sys_rst`'s own real, permanent board pin location is still tentative (`docs/PHYSICAL_REALIZATION.md` §7) — confirm against the real board schematic before finalizing firmware GPIO assignments.