FPGA-Neural/docs/PHYSICAL_REALIZATION.md

FPGA-Neural V3 — Physical Realization Reference

Real, verified data for the custom PCB (bare XC7A100T-CSG324-2 + DDR3, no dev board). Every pin/part/setting below comes from a real Vivado-generated constraint file, a real datasheet, or a real place-and-route run — none of it is guessed. See hardware/v2/logs/experiments.log (EXP-0059 onward) for the full derivation history.

1. Core components (real, verified availability)

Component	Part	Notes
FPGA	XC7A100T-CSG324-2	Speed grade -2 (corrected from an initial -1 assumption, EXP-0074) — same die/package/footprint as -1, strictly better timing margin.
DDR3 SDRAM	Micron MT41J128M16JT-125:K	2Gb, x16, DDR3-1600-rated (run at 310.078MHz here due to -2 timing closure, see §3). Verified in-stock on LCSC.
Config flash	Winbond W25Q32JVSSIQ	32Mbit/4MB, SOIC-8. Comfortably fits the ~30.5Mbit full XC7A100T bitstream. Verified in-stock on LCSC. Wired exclusively to the FPGA (see §5).

2. FPGA pin assignments (real, from the routed design)

2.1 DDR3 (fixed by the FPGA's own PHY hardware — not a free choice)

EXP-0084 UPDATE: real 32-bit channel (two MT41J128M16JT-125:K chips ganged in parallel) — dq/dqs/dm pin counts DOUBLED versus the original 16-bit design (dq 16→32, dqs 2→4 pairs, dm 2→4). Address/command/control lines are unchanged and fanned out identically to both chips. Real, board-accurate pin placement below (confirmed from the actual routed design, mig_7series_0.xdc + this project's own top-level XDC) — functionally verified, but see §3 for the real, honest timing-closure status before treating this as a final signoff.

Generated by the Vivado MIG wizard (mig_7series_0.xdc), all SSTL15 / DIFF_SSTL15 (1.5V), banks 34/35:

Signal	Pin	Signal	Pin	Signal	Pin
ddr3_dq[0]	C5	ddr3_dq[16]	F4	ddr3_addr[0]	U3
ddr3_dq[1]	B7	ddr3_dq[17]	F3	ddr3_addr[1]	U4
ddr3_dq[2]	B6	ddr3_dq[18]	E2	ddr3_addr[2]	V1
ddr3_dq[3]	D8	ddr3_dq[19]	D2	ddr3_addr[3]	U1
ddr3_dq[4]	C7	ddr3_dq[20]	C1	ddr3_addr[4]	L5
ddr3_dq[5]	E6	ddr3_dq[21]	H1	ddr3_addr[5]	L6
ddr3_dq[6]	E5	ddr3_dq[22]	G1	ddr3_addr[6]	L4
ddr3_dq[7]	E7	ddr3_dq[23]	F1	ddr3_addr[7]	K5
ddr3_dq[8]	B4	ddr3_dq[24]	F6	ddr3_addr[8]	M2
ddr3_dq[9]	A4	ddr3_dq[25]	G4	ddr3_addr[9]	M3
ddr3_dq[10]	A3	ddr3_dq[26]	G3	ddr3_addr[10]	L3
ddr3_dq[11]	B3	ddr3_dq[27]	J3	ddr3_addr[11]	K3
ddr3_dq[12]	B2	ddr3_dq[28]	J2	ddr3_addr[12]	M1
ddr3_dq[13]	D5	ddr3_dq[29]	K2	ddr3_addr[13]	L1
ddr3_dq[14]	D4	ddr3_dq[30]	K1	ddr3_ba[0]	V5
ddr3_dq[15]	E3	ddr3_dq[31]	H6	ddr3_ba[1]	V2
ddr3_dm[0]	C6	ddr3_dm[2]	C2	ddr3_ba[2]	U2
ddr3_dm[1]	C4	ddr3_dm[3]	G6
ddr3_dqs_p[0]	A6 (DIFF)	ddr3_dqs_n[0]	A5 (DIFF)	ddr3_cas_n	R3
ddr3_dqs_p[1]	B1 (DIFF)	ddr3_dqs_n[1]	A1 (DIFF)	ddr3_cke[0]	P2
ddr3_dqs_p[2]	H2 (DIFF)	ddr3_dqs_n[2]	G2 (DIFF)	ddr3_cs_n[0]	T5
ddr3_dqs_p[3]	J4 (DIFF)	ddr3_dqs_n[3]	H4 (DIFF)	ddr3_odt[0]	R2
ddr3_ck_p[0]	N2 (DIFF)	ddr3_ck_n[0]	N1 (DIFF)	ddr3_ras_n	V4
ddr3_reset_n	D7 (LVCMOS15)	ddr3_we_n	T3
sys_clk_p	N5 (DIFF_SSTL15, bank 34)	sys_clk_n	P5 (DIFF_SSTL15, bank 34)
clk_ref_p	T14 (LVDS_25, bank 14)	clk_ref_n	T15 (LVDS_25, bank 14)

Real board implication (EXP-0084): both sys_clk and clk_ref are now genuine differential pairs — the board needs a differential oscillator (or a differential buffer stage), not a single-ended one. clk_ref lives in bank 14 (not 34/35) because the MIG wizard's own UG586 placement rules restrict that specific net to bank 14 for this part/package — not a free choice. This directly displaced the config-flash SPI bus, which used to share bank 14 (see §2.3 — moved to bank 16 in EXP-0084 to resolve a real VCCO conflict, LVCMOS33 vs LVDS_25 cannot coexist in one bank).

Bank voltage requirements: bank 34/35 → 1.5V (DDR3 SSTL15, plus sys_clk_p/n at bank 34's own 1.5V-domain DIFF_SSTL15), bank 14 → 2.5V (clk_ref_p/n, LVDS_25).

INTERNAL_VREF for banks 34/35 is set to 0.750V by the MIG constraints (required for SSTL15 single-ended inputs) — this is a Vivado-side setting, not a board component, but note it if you ever inspect bitstream generation warnings about VREF.

2.2 Neural-processor management SPI (ESP32 ↔ FPGA, FPGA is slave)

Bank 15, package edge column A/B, physically adjacent (short traces), LVCMOS33:

Signal	Pin	Direction (FPGA side)
sclk	A15	input
mosi	B16	input
miso	B17	output
cs_n	A16	input

Bank 15 VCCO: assumed 3.3V — change the XDC's IOSTANDARD if your board power plan uses a different rail for this bank.

2.3 Config flash SPI (FPGA ↔ flash, FPGA is master)

EXP-0084 REAL PIN CHANGE: originally these were the FPGA's own dedicated Master-SPI configuration pins (K17/K18/L13, bank 14), reclaimed as ordinary fabric I/O post-configuration. EXP-0084's real 32-bit DDR3 widening moved the differential reference clock (clk_ref_p/n, §2.1) into bank 14 too — a real VCCO conflict (the flash needs LVCMOS33/3.3V, clk_ref needs LVDS_25/2.5V, one bank can only have one VCCO), confirmed by a real place_design failure, not hypothetical. Fixed by moving the flash bus to bank 16 (completely unconstrained before this, no VCCO commitment):

Signal	Pin	Direction (FPGA side)	Notes
flash_mosi	D9	output	bank 16 (was K17/D00_MOSI, bank 14)
flash_miso	D10	input	bank 16 (was K18/D01_DIN, bank 14)
flash_cs_n	C9	output	bank 16 (was L13/FCS_B, bank 14)
(CCLK)	E9	output	Not a top-level port — driven internally via the STARTUPE2 primitive. Wire the flash's own CLK pin to package pin E9. Unaffected by the bank-14→16 move (CCLK was never a plain port).

Still requires BITSTREAM.CONFIG.PERSIST = NO (the Vivado default — set explicitly in the project XDC; EXP-0084 note: the XDC previously had this as FALSE, an invalid enum value in this Vivado version — silently not applied at all until fixed to the real NO/YES enum).

Bank 16 VCCO: 3.3V (matches the flash's own VCC, typically 1.8–3.6V for the W25Q32JV) — chosen freely since bank 16 had no prior VCCO commitment.

Real board implication: since the flash bridge no longer uses the FPGA's own dedicated D00_MOSI/D01_DIN/FCS_B configuration pins at all (K17/K18/L13 are now unused by this design), the flash chip's SPI lines on the PCB connect to D9/D10/C9 instead — a real routing change versus any earlier board layout draft based on the pre-EXP-0084 pinout.

Reserved, do not use (bank 14, unused by this design, kept clear for any future Quad-SPI/BPI expansion — no longer near the flash bus, now just general config-mode-reserved pins): L16 (EMCCLK), R16 (RDWR_B), V15 (CSI_B). The project's own XDC PROHIBITs the underlying sites for these (fixed in EXP-0084 — PROHIBIT is not a valid property directly on package_pin objects in this Vivado version, only on their site; this constraint had silently never been enforced before).

2.4 FPGA configuration control (dedicated, bank 0, not negotiable)

Signal	Pin	Purpose
PROGRAM_B	P9	pulse low to force a full reconfiguration from flash
INIT_B	P7	goes low during config; can indicate a config error if it re-asserts
DONE	P10	goes high once configuration succeeds — wire to a status LED if desired
M0	P12	mode select
M1	P13	mode select
M2	P11	mode select
CFGBVS	P8	tie to match bank 0's VCCO logic level (see UG470)

Mode pin setting for Master SPI boot (the flash-based autonomous boot path, see §5): M[2:0] = 001 (per UG470's mode pin table) — tie via pull-up/ pull-down resistors on the board, not driven dynamically.

2.5 JTAG (always available, independent of flash content)

Signal	Pin
TCK	E10
TDI	E11
TMS	E12
TDO	E13

Used for: (a) first-ever/factory programming when the flash is blank (see §5), (b) recovery, (c) development/debug. This project's own plan drives these from an ESP32 doing real JTAG bit-banging (TAP state machine, IR/DR shifting) rather than a bench programmer — that firmware is separate, software-side work, not covered here.

3. Real timing signoff (EXP-0083, the current, trustworthy number)

Real in-context Vivado place-and-route (not out-of-context, not estimated):

Metric	Value
DDR3 PHY clock (sys_clk_i)	310.078 MHz (3.225ns period)
Compute domain clock (ui_clk, PLL-derived 2:1 from sys_clk_i)	155.039 MHz
WNS (setup slack)	+0.073 ns — real, still thin but improved vs every prior real run. Re-verify with a fresh P&R after ANY further logic addition.
WHS (hold slack)	+0.036 ns
Failing endpoints	0 (setup), 0 (hold)
LUTs used	5644 / 63400 (8.90%)
DSP48E1 used	16 / 240 (6.67%) — 8 per compute core × 2 cores, unchanged since EXP-0059
Block RAM used	0
STARTUPE2 used	1 / 1 (100%) — the config-flash bridge

Signoff history (every real change, same target, in-context P&R):

EXP	WNS (ns)	LUTs	DSP48E1
0074 (first real DDR3 + pins)	+0.040	5140	16
0076 (+regfile, +pins, +SPI fix)	+0.056	5173	16
0078 (+flash bridge, STARTUPE2)	+0.013	5213	16
0079 (+real activation engine)	+0.030	5379	16
0082 (+denser activation packing)	+0.068	5437	16
0083 (+DDRManager phase 1, ddr_prefetch_mgr.v)	+0.073	5644	16
0084 (32-bit DDR3 widening, 2900ps/172.414MHz)	-0.618 (FAILS)	6418	16

3.1 EXP-0084: real 32-bit DDR3 widening — functionally verified, timing NOT yet closed

Honest status, not glossed over: the real 32-bit channel (two MT41J128M16JT-125:K chips) is functionally complete and real-verified (real xsim against the real 2-chip DDR3 model: tb_mig_native_adapter.v 12/12 PASS, tb_n2_system_ddr3.v 8/8 PASS) — but its own real P&R does not close timing at the paired clock speedup (2900ps sys_clk / 172.414MHz ui_clk): WNS = -0.618ns, 213 failing endpoints. Route completed 100% (a real, physically valid placement/routing), only timing itself fails.

Real root cause (traced to the actual worst path): the violation is inside neural_processor_packed.v's own packed-MAC accumulation tree (a DSP48E1 output through a 4-deep CARRY4 chain), unchanged since EXP-0059 — it had real positive margin at the old 155.039MHz ui_clk (EXP-0083: +0.073ns) but not at the new 172.414MHz (an 11.2% real frequency increase). This is not caused by the 32-bit width change itself — every module touched for the widening (adapter, arbiter, activation fetch, etc.) is real-verified functionally correct; the failure is in unrelated, pre- existing compute logic that simply wasn't budgeted for a faster clock.

The real, honest decoupling this reveals: bandwidth = width × clock rate. Widening to 32 bits alone, even at the OLD, already-closing 3225ps/155.039MHz, already delivers the FULL intended 2× bandwidth gain (1.24 → ~2.48 GB/s physical ceiling) — the clock speedup to 2900ps was a separate, additional optimization stacked on top in the same wizard session, and it is specifically that stacking, not the width, that breaks timing.

Current real, trustworthy signoff therefore remains EXP-0083 (16-bit, 155.039MHz, +0.073ns) until a real P&R closes for the 32-bit configuration — expected once the Input Clock Period is reverted toward 3225ps (keeping Data Width=32), a real, user-gated MIG wizard step, not yet done. See hardware/v2/logs/experiments.log EXP-0084 for the complete story, including 5 real P&R/XDC bugs found and fixed along the way (stale differential-clock ports, a real VCCO bank conflict, a stale imported XDC, missing IOSTANDARDs, and two cosmetic but previously-silently-broken XDC property bugs).

4. Real DDR3 memory layout convention

Both weight data and activation data share the same DDR3 address space, word-addressed. EXP-0084: the native word width is now 32 bits (BURST_LEN=8 per transaction = 256 bits/burst, up from 128 bits at the old 16-bit width) — this is the real, current RTL (hardware/v3/rtl/), functionally verified; see §3.1 for its pending timing-closure status.

• Weights: one layer's weight set starts at word address layer_index * WORDS_PER_LAYER (WORDS_PER_LAYER = LAYER_BYTES/2). Densely packed — layer_prefetch_ctrl.v reads full bursts sequentially into the on-chip weight buffer once per job. BYTES_PER_BURST is now 4*BURST_LEN (32 bytes/burst, up from 16) — the byte-drain logic itself is word-width- agnostic and needed no change beyond this.
• Activations (real engine, act_tile_fetch.v; current layout is the v3 convention, EXP-0084): FOUR consecutive tiles (P_IN=8 INT8 values each, 64 bits each) share ONE full BURST_LEN=8-word (256-bit) burst — tile parity 0/1/2/3 (tcnt[1:0]) selects bits [63:0]/[127:64]/[191:128]/[255:192] of the burst response. Tile t's burst address is base + (t>>2)*BURST_LEN (integer division — four tiles per burst), always burst-aligned by construction. This is not a further bytes-per-MAC reduction beyond EXP-0081's already- optimal 1 byte/MAC — it's what's required to keep that same 100% packing utilization at the new, larger burst size instead of leaving half of it newly wasted.
  • Why this is timing-safe despite selecting a sub-burst quarter at read time: the tile index's own low 2 bits (which quarter of the burst a given tile lives in) are known at request time, not at response time. They're latched into a register (sel_lat) the same cycle the request is accepted — many ui_clk cycles before the real DDR3 round-trip completes and ctrl_rdata becomes valid. The eventual data-select mux is an explicit 4-way case on constant byte offsets (not a runtime-indexed part-select expression) — extending the exact same discipline EXP-0081 established for the 1-bit case to 2 bits. This mux is real and functionally verified, but is unrelated to §3.1's own real timing failure (that violation is inside the compute core's accumulation tree, not this fetch path).
  • Real measured effect: back-to-back same-row DDR3 throughput was a fixed 1.24 GB/s at 16-bit width (measured, EXP-0080); at 32-bit width the real physical ceiling doubles to ~2.48 GB/s, independent of this packing convention — packing controls how much of that ceiling is wasted, not the ceiling itself (see docs/ARCHITECTURE_ANALYSIS.md §3 and §5.1/§5.4).
  • base (a job's own x_base_a/x_base_b) is chosen freely by whoever submits jobs (the SPI host) — just keep each position's own activation array in its own non-overlapping (N_TILES/4) * BURST_LEN-word region.

5. FPGA configuration (boot) procedure

Two complementary paths, both present on this board by design:

1. Factory-first / recovery (JTAG, ESP32-driven): the flash starts blank on a fresh board — no other path can bootstrap it (a real chicken- and-egg constraint: the FPGA can't relay flash-programming commands over SPI, §5.2, until it's already running logic that does that). The ESP32 bit-bangs JTAG (§2.5) to load a bitstream directly, or to run Vivado's own "indirect SPI flash programming" sequence to write the flash for the first time. One-time (or rare/recovery-only) step.
2. Normal boot (Master SPI, autonomous): every subsequent power-on, the FPGA self-configures from the flash via its own dedicated hardware (mode pins set to Master SPI, §2.4) — no ESP32 involvement needed.
3. Field firmware updates (SPI-through-FPGA, FLASH_XFER opcode 0x40): once the FPGA is running, the ESP32 can rewrite the flash by relaying raw SPI-NOR bytes through the FPGA over the management SPI bus (§2.2) — the FPGA then re-transmits them as master on the flash bus (§2.3). This is the only electrical path from ESP32 to the flash; there is no direct connection (by design, per explicit requirement).
   • Real SPI-NOR opcodes (verified against the actual W25Q32JV datasheet, for whoever writes the ESP32-side flashing routine): 0x06 Write Enable, 0x04 Write Disable, 0x05 Read Status Register-1 (bit0=BUSY, bit1=WEL), 0x02 Page Program, 0x03 Read Data, 0x20 Sector Erase (4KB), 0x52 32KB Block Erase, 0xD8 64KB Block Erase, 0xC7/0x60 Chip Erase.
   • Protocol timing note: FLASH_XFER relays are NOT instantaneous — each relayed byte's real flash response is only stable starting two host-clocked bytes later (not one), so the host must clock 2 trailing dummy bytes after its last real command byte to safely receive the final response. See spi_host_bridge_v3.v's own header for the full real-measured reasoning (EXP-0077).
   • After writing a new bitstream to the flash, reconfigure either by pulsing PROGRAM_B externally, or (future work, not built yet) via a ICAPE2-based warm self-reconfiguration triggered over the same SPI bus.

6. Management SPI protocol summary (for ESP32 firmware)

One opcode byte (MSB-first) per CS-low transaction, driven by spi_host_bridge_v3.v:

Opcode	Name	Payload	Purpose
0x00	NOP	0 bytes	inert
0x0F	RESET	0 bytes	pulses a soft-reset
0x10	WRITE_JOB	16 bytes	submit one inference job (node_id, x_base, w_base, n_tiles, result_addr)
0x20	STATUS	0 bytes → 1 byte out	job_busy / mem_busy / last_job_accepted bits
0x01	WRITE_MEM	4+2N bytes	raw DDR3 word write (N words)
0x02	READ_MEM	6 bytes → 2N bytes out	raw DDR3 word read (N words)
0x30	REG_WRITE	5 bytes	write a control register
0x31	REG_READ	1 byte → 4 bytes out	read a status/ID register (0x00 DEVICE_ID, 0x01 CONTROL, 0x02 STATUS incl. DDR3-ready + Director-error, 0x03 N_SLOTS)
0x40	FLASH_XFER	N bytes → N bytes out (+2 margin)	raw passthrough to the config flash, see §5.3

Full byte-level field layouts are documented in spi_host_bridge_v3.v's own header comment — treat that file as the authoritative protocol spec, this table is a summary/index.

7. Known-open items (honestly disclosed, not hidden)

• Scaling past N=2 compute cores (silicon budget allows up to ~30 per the DSP48E1 count) is not yet built or timing-verified. A result-writeback engine (currently result_data_a/b are literal top-level pins, fine at N=2 but a real pin-budget blocker beyond that) must land first — see docs/ARCHITECTURE_ANALYSIS.md §5.3.
• sys_rst has only a tentative real pin (G13, bank 15, EXP-0084 — chosen just to unblock real P&R, not a final board decision) — assign its real, permanent location once the rest of the board layout (reset circuit, status LEDs, etc.) is decided.
• The 32-bit DDR3 channel's own real timing does NOT currently close (EXP-0084: WNS -0.618ns at 2900ps/172.414MHz) — the current real, trustworthy signoff remains EXP-0083 (16-bit, 155.039MHz, +0.073ns) until the Input Clock Period is reverted toward 3225ps (keeping Data Width=32) in one more real, user-gated MIG wizard session. See §3.1 for the full honest story and real root cause (unrelated to the width change itself).
• ESP32-side JTAG bit-banging firmware (§5.1) does not exist yet — it's software work on the host side, not part of this FPGA RTL.
• ddr_prefetch_mgr.v (DDRManager phase 1, EXP-0083) is real, built, and timing-verified (at the OLD 16-bit/155MHz signoff), but its measured benefit (2.86% real reduction in total simulated time) is modest — see docs/ARCHITECTURE_ANALYSIS.md §5.2 for the honest number and why. The larger multi-slot DDRManager is not built. Its own real benefit should be re-measured once the 32-bit channel's timing actually closes.
• User-requested, not yet built: an active-low data_ready_n/IRQ pin so the ESP32 can be interrupt-driven (job completion or error pending) instead of polling the STATUS register — planned as a sticky bit (set on job_out_done/dir_error, cleared when the ESP32 reads STATUS), deliberately deferred until the §3.1 timing-closure work lands so it doesn't further complicate the currently-tight I/O/VCCO budget mid-fix.