FPGA-Neural/docs/FIRMWARE_SPEC.md

FPGA-Neural — ESP32 Host Firmware Specification

Real, from-source specification for the ESP32-S3 host firmware, derived directly from the actual RTL protocol implementation (spi_host_bridge_v3.v, neural_director_grouped.v, result_writeback.v, host_mem_bridge.v, flash_spi_master.v) and the real, closed timing signoff in docs/PHYSICAL_REALIZATION.md. Nothing below is guessed — every field layout, address formula, and timing number is read directly from the committed RTL's own header comments or real, measured project data.

Status: this is a specification only. No ESP32-side firmware exists yet in this repository — this document exists to make that firmware buildable without re-deriving the protocol from RTL by hand. Real, disclosed scope: management SPI protocol, job submission, result readback, raw memory access, config-flash relay, boot/bootstrap. Out of scope: FreeRTOS task structure, Wi-Fi/network layer, higher-level model-graph scheduling above "submit one job" — those are real firmware design decisions, not yet made anywhere in this project.

Real fabrication target (2026-09-22): N=16 (n16_system_ddr3_top.v, 16 real parallel processing elements, 4 groups × 4 PEs). This document is written for N=16 throughout; N=2/N=8 fallback configurations use the byte-for-byte identical protocol (§3), differing only in the real octet/pairing requirement of §4.2.

---

1. Physical interfaces

Two electrically separate SPI buses, plus JTAG. Full real pin assignments: docs/PINOUT.md / docs/PHYSICAL_REALIZATION.md §2.

Bus	FPGA role	Purpose	Pins (bank 15/16, LVCMOS33)
Management SPI	slave	Job submission, status, raw memory access, config-flash relay	sclk=A15, mosi=B16, miso=B17, cs_n=A16
Config-flash SPI	master (relayed)	Bitstream/config flash access, reached only via FLASH_XFER (§6)	not directly host-visible — internal to the FPGA
JTAG	target	Factory/recovery programming	TCK=E10, TDI=E11, TMS=E12, TDO=E13

Additional real, dedicated signals on the management SPI bus's own bank:

Signal	Pin	Direction (ESP32 side)	Purpose
sys_rst	G13 (tentative real pin, §7 of PHYSICAL_REALIZATION.md)	output	Board-level reset, separate from the SPI-driven soft-reset (§3)
data_ready_n	D14	input, active-low	Sticky IRQ — see §5.1

Real, important note: the ESP32 has no direct electrical connection to the config flash. The only path is FLASH_XFER (§6), relayed through the FPGA over the management SPI bus.

---

2. Boot / bootstrap sequence

Three real, distinct paths (docs/PHYSICAL_REALIZATION.md §5):

1. Factory-first / recovery (JTAG, ESP32-driven, one-time or rare) — a factory-fresh board has a blank config flash. The FPGA cannot relay FLASH_XFER commands until it is already running logic that does that — a real chicken-and-egg constraint. The only way to bootstrap a blank board is for the ESP32 to bit-bang JTAG (TAP state machine, IR/DR shifting) and either (a) load a bitstream directly into the FPGA's configuration memory (volatile — lost on power-cycle), or (b) run Vivado's own real "indirect SPI flash programming" sequence over JTAG to write the flash for the first time. Real, disclosed gap: this JTAG bit-banging firmware does not exist yet anywhere in this project — it is real, necessary, unbuilt software work, out of scope for this document beyond flagging it.
2. Normal boot (Master SPI, autonomous, every power-on) — the FPGA self-configures from the flash via its own dedicated configuration hardware (mode pins M[2:0]=001, §2.4 of PHYSICAL_REALIZATION.md). No ESP32 involvement at all. Firmware's own real startup sequence should simply wait for init_calib_complete (readable via REG_READ on STATUS, §3.3, bit3) before issuing any job or memory access — DDR3 traffic before real PHY calibration completes is unsafe.
3. Field firmware update (FLASH_XFER, §6) — once the FPGA is already running (path 2 completed), the ESP32 can rewrite the config flash by relaying raw SPI-NOR bytes through the FPGA. This is the only path from ESP32 to flash during normal operation.

2.1 Real, recommended firmware startup sequence

1. Assert sys_rst (G13), hold >= a few clk periods, release.
   (Real note: sys_rst's own real, permanent board location is still
   tentative per PHYSICAL_REALIZATION.md §7 -- confirm against the
   real board schematic before relying on this pin number.)
2. Wait for the FPGA's own real Master-SPI autonomous boot to complete
   -- there is no host-visible "boot done" signal separate from
   init_calib_complete; DONE (pin P10) is a real, physical
   configuration-status pin if the ESP32 has a GPIO wired to it
   (board-dependent, not guaranteed).
3. REG_READ(0x00 DEVICE_ID) -- confirm 32'h4E50_5601 ("NPV" + protocol
   version 1) before trusting anything else. A mismatch means either
   the flash holds the wrong bitstream, or config-boot has not
   actually completed -- do not proceed.
4. Poll REG_READ(0x02 STATUS) bit3 (init_calib_complete) until set.
   Real, measured DDR3 calibration timing has not been characterized
   on real hardware yet (only in simulation, where SIM_BYPASS_INIT_CAL
   shortcuts it) -- budget real time for this, do not assume it is
   instantaneous.
5. Only once bit3 is set: safe to issue WRITE_MEM (weight/activation
   preload, SS3) and WRITE_JOB (SS5).

---

3. Management SPI protocol — real opcode reference

One opcode byte (MSB-first), sent as the first byte of a CS-low SPI transaction; multi-byte fields are MSB-first. Authoritative source: spi_host_bridge_v3.v's own header comment — this table is a summary.

Opcode	Name	Direction	Payload (MOSI)	Response (MISO)
0x00	NOP	—	0 bytes	inert (0x00)
0x0F	RESET	host→FPGA	0 bytes	pulses soft_rst_pulse for 1 real clk cycle after CS rises
0x10	WRITE_JOB	host→FPGA	16 bytes (§4.1)	—
0x20	STATUS	host→FPGA	0 bytes	1 byte (§3.1)
0x01	WRITE_MEM	host→FPGA	4 + 2·N bytes (§6.1)	—
0x02	READ_MEM	host→FPGA	6 bytes	2·N bytes (§6.1)
0x30	REG_WRITE	host→FPGA	5 bytes (§3.2)	—
0x31	REG_READ	host→FPGA	1 byte	4 bytes (§3.3)
0x40	FLASH_XFER	bidir	N bytes	N+2 bytes (§6.2)

Real, load-bearing rule: any opcode byte not in this table is treated as NOP — the bus is never wedged by an unrecognized command. Do not rely on this for real protocol negotiation; it exists purely as a safety fallback.

3.1 0x20 STATUS — 1 response byte

Bit	Meaning
0	job_busy — a WRITE_JOB is currently waiting on job_in_ready
1	mem_busy — a WRITE_MEM/READ_MEM is waiting on mem_ready
2	last_job_accepted (sticky, cleared by the next WRITE_JOB)
7:3	reserved, always 0

3.2 0x30 REG_WRITE / 0x31 REG_READ

REG_WRITE: byte0=reg_addr[7:0], byte1:4=value[31:0] MSB-first — applied the instant the last byte lands (purely internal, no backend handshake).

REG_READ: byte0=reg_addr[7:0], then 4 response bytes MSB-first. An unmapped address reads back 0xFFFFFFFF (deliberately distinct from a real 0, so firmware can tell "unmapped register" from "real zero"). Writing a read-only or unmapped register is inert (accepted, no effect).

3.3 Real register map (v1)

Addr	Name	R/W	Real meaning
0x00	DEVICE_ID	RO	32'h4E50_5601 ("NPV" + protocol version 1)
0x01	CONTROL	RW	bit0: write 1 to pulse soft_rst_pulse (same physical effect as 0x0F); always reads back 0; bits[31:1] reserved
0x02	STATUS	RO	bit0 job_busy, bit1 mem_busy, bit2 last_job_accepted, bit3 init_calib_complete, bit4 dir_error; bits[31:5] reserved
0x03	N_SLOTS	RO	real, total parallel-PE count this bitstream was built with (16 for the current N=16 fabrication target) — read this instead of hardcoding the core count in firmware

---

4. Job submission

4.1 0x10 WRITE_JOB — real 16-byte payload

Bytes	Field	Real convention
0:1	node_id[15:0]	host-assigned identifier, returned unchanged with the result (§5)
2:5	x_base[25:0]	byte-address convention (26-bit JOB_ADDR_WIDTH); byte2's MSBs are {6'b0, x_base[25:24]}
6:9	w_base[25:0]	same convention — must match, together with n_tiles, across the real octet requirement of §4.2
10:11	n_tiles[15:0]	number of P_IN=8-wide tiles to stream (real activation length = n_tiles × 8 bytes)
12:15	result_addr[25:0]	where result_writeback.v will write the real result — see §5 for the real readback formula

job_in_valid is asserted and held until job_in_ready also reads 1 in the same real cycle — never a blind, unacknowledged pulse. Firmware does not need to do anything special for this; it is entirely internal to the bridge.

4.2 REAL, CRITICAL N=16 requirement: octet dispatch

This is the single most important real constraint in this document. neural_director_grouped.v (the real N=16 job scheduler) only dispatches jobs in groups of 8 — it inspects the 8 oldest entries in its own internal queue and requires all eight to share the exact same w_base and n_tiles. If they do, they are dispatched together (one octet = one real systolic group's own job, weight fetched once, broadcast to 4 PEs × 2 lanes). If they do not, the queue stalls — permanently, with no in-band recovery. Real, verified behavior (tb_neural_director_grouped.v): a mismatched octet does not corrupt anything and does not silently mis-pair jobs — it simply stops draining the queue. The only real recovery is a hardware reset (sys_rst, or CONTROL register bit0 / RESET opcode) — there is no software-visible "skip this job" or "flush the queue" command.

Real, mandatory firmware discipline: submit exactly 8 WRITE_JOB transactions sharing the same w_base and n_tiles, back-to-back, with no other job submitted in between, for every real layer-position batch. If a real workload's position count is not a multiple of 8, pad the batch (e.g. repeat the last real position, or submit a real dummy position with a result_addr the firmware will simply ignore) rather than submit a short, mismatched batch — the RTL has no concept of a "partial octet."

Real, deliberate, unchanged property: the WRITE_JOB payload itself is byte-for-byte identical whether the octet requirement applies (N=16, N=8's own pair-of-4 requirement) or not (N=2's own simple pairing) — only the batch size firmware must submit before a batch reliably drains differs (8 for N=16, 2 for N=2). Always read N_SLOTS (§3.3) at startup and derive the real required batch size as N_SLOTS / 2 rather than hardcoding 8.

---

5. Result readback

Results are not returned over SPI directly — they are written by result_writeback.v into DDR3 at the job's own result_addr, and must be read back via the existing READ_MEM (0x02) opcode. No new opcode exists or is needed for this.

Real, exact addressing formula (from result_writeback.v's own header — result_addr here is the 26-bit value submitted in the job's own WRITE_JOB payload, §4.1):

mem_addr (value)    = result_addr[24:0] * 2
mem_addr (node_id)   = result_addr[24:0] * 2 + 1

Two READ_MEM transactions per real result lane (one job produces two results, lane A and lane B — see packed_pe.v's own real A/B lane structure):

• mem_addr above reads back a 16-bit word: low byte = the real INT8 result value (result_data, zero-extended into the low 8 bits of the word); the other bytes are 8'h00.
• mem_addr+1 reads back the real 16-bit node_id the job was submitted with — use this to match a result back to the job that produced it, since results may complete out of submission order across different groups.

Real, disclosed firmware design question, not yet decided at the RTL level: how the host knows a specific result is ready (vs. polling READ_MEM speculatively) is answered by §5.1 below (data_ready_n), but data_ready_n is a single, whole-device sticky flag, not per-job — firmware must track its own real outstanding-job count/set and read back results as they become plausible, not per-job-precise from the hardware alone.

5.1 data_ready_n — real, sticky, active-low interrupt

Real, board-level pin (D14). Lets firmware be interrupt-driven instead of polling STATUS in a loop.

• Asserted (driven low) when a real job/octet completes (job_out_done) or while dir_error is active.
• Sticky: the job-completion contribution stays low even after the underlying completion pulse ends, until the host acknowledges by completing a real STATUS (0x20) or REG_READ of STATUS (0x31, reg 0x02) transaction. A REG_READ of any other register does not acknowledge it.
• The dir_error contribution is live, not latched — it clears the instant dir_error itself does (typically only after a real reset, since the Director has no in-band error-clear per §4.2).

Real, recommended firmware pattern: wire data_ready_n to an ESP32 GPIO interrupt; on falling edge, read STATUS (acknowledges the sticky latch and tells firmware whether it was a real completion or a dir_error condition), then read back whichever real results are now plausible.

---

6. Raw memory access (weight/activation preload, debug)

6.1 0x01 WRITE_MEM / 0x02 READ_MEM

Real, different address convention from WRITE_JOB: these opcodes use host_mem_bridge.v's own 16-bit-WORD address space (MEM_ADDR_WIDTH=25 bits), not the 26-bit byte-base convention x_base/w_base/result_addr use. Do not mix the two conventions.

• WRITE_MEM: byte0:3 = addr[24:0] (word address; byte0's MSBs are {7'b0, addr[24]}), then len_words (2 bytes, right after addr) × 2 payload bytes, MSB-first per word. Each word is written via one real mem_req/mem_ready handshake before the next word's bytes are accepted — a real, sequential, per-word transaction, not a burst.
• READ_MEM: same 6-byte header (4 addr + 2 len_words), 0 further MOSI payload; 2·len_words response bytes clocked out on MISO starting at payload byte 7, MSB-first per word.

Real, disclosed limitation (not a bug, a deliberate, documented design choice, host_mem_bridge.v's own header): this path halves the host's own reachable byte range for a given ADDR_WIDTH, since the underlying shared bus is 32-bit-word-native (EXP-0084's real DDR3 widening) while this host-facing contract stays a fixed 16-bit word — mem_addr's own LSB additionally selects which 16-bit half of the real 32-bit ctrl-bus word to target. Not currently a practical constraint at this project's real usage scale.

Real use: preloading weight/activation data into DDR3 before submitting jobs that reference it. See docs/PHYSICAL_REALIZATION.md §4 for the real, exact DDR3 memory layout convention (weight-per-layer addressing, 4-tiles-per-burst activation packing) firmware must follow when computing x_base/w_base values for WRITE_JOB.

6.2 0x40 FLASH_XFER — config-flash relay

Raw, byte-for-byte SPI passthrough to the FPGA's own config flash. The FPGA relays every MOSI byte it receives onto the physical flash's own MOSI line, and relays whatever the flash returns back on MISO. The FPGA knows nothing about SPI-NOR command semantics — firmware is responsible for sending a real, complete flash command sequence, exactly as if directly wired to the flash.

Real SPI-NOR opcodes (Winbond W25Q32JV, verified against the actual datasheet):

Opcode	Command
0x06	Write Enable
0x04	Write Disable
0x05	Read Status Register-1 (bit0=BUSY, bit1=WEL)
0x02	Page Program
0x03	Read Data
0x20	Sector Erase (4KB)
0x52	32KB Block Erase
0xD8	64KB Block Erase
0xC7 / 0x60	Chip Erase

Real, critical timing requirement (measured via simulation, spi_host_bridge_v3.v's own header, EXP-0077): a relayed byte's real flash response only becomes stable starting the transfer's own byte N+2 — one extra host-clocked byte is not enough (flash_spi_ master.v's own internal transfer, ~640ns at 155.039MHz/DIV=4, starts only once byte N is fully received, overlapping byte N+1's own window). Firmware must clock two trailing dummy bytes at the end of a FLASH_XFER transaction to safely receive the final real response — one dummy byte is a real, previously-reproduced bug, not a hypothetical concern.

Real update procedure: after writing a new bitstream to the flash via FLASH_XFER, reconfigure either by pulsing PROGRAM_B externally (if the ESP32 has a GPIO wired to it — board-dependent), or (real, disclosed future work, not built) via an ICAPE2-based warm self-reconfiguration triggered over the same SPI bus.

---

7. Error handling

dir_error (readable via STATUS, contributes to data_ready_n) is the only real, hardware-visible error signal. It is a latch, not a transient flag — per neural_director_grouped.v's own real, disclosed design (§4.2), there is no in-band recovery: once set (e.g. by a real octet-mismatch stall), only a hardware reset clears it.

Real, recommended firmware policy: treat dir_error as fatal to the current batch of in-flight jobs — issue a real reset (RESET opcode or CONTROL register bit0), re-establish DEVICE_ID/STATUS sanity (§2.1 steps 3–4), and re-submit any real jobs that were in-flight at the time of the error, since the Director's own internal queue state is not preserved across a reset.

---

8. Real, disclosed open items

• No ESP32-side firmware exists yet in this repository — this document is the specification, not an implementation.
• JTAG bit-banging firmware (§2, path 1) is real, necessary, unbuilt software.
• Real DDR3 calibration timing (§2.1 step 4) has not been measured on real hardware — only in simulation with SIM_BYPASS_INIT_CAL.
• Whether/how the ESP32 firmware tracks outstanding jobs per-node_id (§5) — a real firmware design decision, not yet made.
• ICAPE2-based warm self-reconfiguration (§6.2) is disclosed future work, not built.
• sys_rst's own real, permanent board pin location is still tentative (docs/PHYSICAL_REALIZATION.md §7) — confirm against the real board schematic before finalizing firmware GPIO assignments.