FPGA-Neural-Datasheet/chapters/01-overview.tex

\chapter{Overview and design philosophy}
\label{ch:overview}

\section{From sequential accelerator to dataflow machine}
V1 is, structurally, a single pipeline: one neuron computes at a time,
driven by the host over SPI, one MAC group at a time, one layer at a
time. It is fast for what it is (the V1 datasheet's own ``ECP5
implementation'' chapter documents its real Fmax/timing-closure history),
but it cannot keep
more than one computational unit genuinely busy at once, and it has no
notion of a dependency graph --- the host sequences everything.

V2 keeps V1's own proven INT8 datapath (bit-exact, byte-for-byte reused
math) but wraps it in a fundamentally different control architecture:
a \textbf{Dependency Manager} tracks a graph of neuron ``jobs'', each
with an explicit list of producer nodes it depends on; a \textbf{Neural
Director} dispatches every node whose dependencies have resolved to
whichever of \code{N\_SLOTS} concurrent (Memory Manager $+$ Neural
Processor) pairs is free; a slot's completion feeds back to wake up any
node that was waiting on it. Once a graph is loaded, the whole system
runs autonomously --- no per-neuron host intervention.

\section{What did NOT change}
\begin{itemize}
\item The INT8$\times$INT8$\to$INT32 MAC math, the balanced adder tree,
      ReLU/linear activation with saturation --- \code{neural\_processor.v}
      is a direct, bit-exact-verified port of V1's own
      \code{neuron\_parallel.v}/\code{mac8.v}/\code{mac\_unit.v}.
\item V1's own PSRAM backend files (\code{memory\_interface.v},
      \code{psram\_controller.v}) remain byte-for-byte, unmodified
      copies throughout the repository --- V1 itself, as a tree
      (\code{hardware/v1/}), is frozen and was never touched.
      \textbf{Not currently part of V2's physical board}, however: the
      project has since replaced external memory with a single SDR
      SDRAM device (\S\ref{sec:sdram-mem-addendum}); the PSRAM-era
      chapters that follow document real, correctly-measured work for
      the architecture it was measured on, not the current board.
\item The target device (Lattice ECP5 \code{LFE5U-45F-8BG381C}) and the
      real-toolchain-only measurement discipline: every number in this
      datasheet is labelled \textsc{Theoretical}, \textsc{Simulated},
      \textsc{Post-P\&R measured}, or \textsc{Derived}, and no result was
      invented to make V2 look better than it measured (§\ref{ch:impl2}).
\end{itemize}

\section{What DID change}
\begin{itemize}
\item \textbf{Concurrency}: from one active neuron to \code{N\_SLOTS}
      independent Neural Processor instances, each fed by its own Memory
      Manager.
\item \textbf{Scheduling}: from host-sequenced SPI opcodes to an on-chip
      dependency graph, resolved autonomously.
\item \textbf{Memory backend granularity}: from byte-at-a-time fetches
      (through \code{int8\_memory\_access.v}, still frozen V1 but no
      longer instantiated in V2's own datapath) to word-level bursts
      talking to \code{memory\_interface.v} directly --- a real, measured
      2.24--2.37$\times$ speedup (ch.~\ref{ch:mem}).
\item \textbf{Memory traffic pattern}: a new shared on-chip
      \textbf{activation cache} eliminates redundant re-fetching of an
      input vector shared by many neurons of the same layer --- a
      further real 1.66--2.00$\times$ cycle reduction, at a real, honestly
      reported Fmax cost (ch.~\ref{ch:mem}).
\end{itemize}

\section{The central, measured finding}
The single most important result of this project's own benchmark
campaign is that \textbf{V2 is memory-bound, not compute-bound}: the
real compute-to-memory-wait ratio is on the order of 1:170--1:220, and
the one physical PSRAM port saturates at $\approx$90\% utilization
regardless of \code{N\_SLOTS}$\ge$2. Real parallel scaling from
\code{N\_SLOTS}=1 to \code{N\_SLOTS}=8 is essentially flat for
large/sustained workloads (1.05--1.06$\times$), and once real,
place\&route-measured Fmax degradation from added routing congestion is
also accounted for, \code{N\_SLOTS}=4 measures as \emph{slower} in real
wall-clock time than \code{N\_SLOTS}=1 for the largest workload tested
--- more hardware parallelism made that specific configuration worse,
not better, because the bottleneck was never compute. This finding
directly shaped both post-campaign optimizations in ch.~\ref{ch:mem}.

\begin{fnwarn}[Architecture changed since this finding: SDRAM, not PSRAM]
This memory-bound finding was measured on the PSRAM-era architecture
described above. The project has since replaced PSRAM with a single
SDR SDRAM device (\S\ref{sec:sdram-mem-addendum}) and closed on
\textbf{\code{N\_SLOTS}=4 as the production configuration} --- chosen
primarily because it is the largest slot count that reliably closes
real timing (8/8 seeds @ 64\,MHz, ch.~\ref{ch:hw}
\S\ref{sec:clock-closure-current}), not from a re-run of this specific
utilization/scaling study. Whether the SDRAM backend's own
utilization/saturation ratio matches the PSRAM-era $\approx$90\% figure
above has \textbf{not been independently re-measured} --- disclosed as
an open item, not assumed to carry over.
\end{fnwarn}

\begin{fnnote}[Reproducibility]
Every real number in this datasheet traces to a specific, append-only
log entry (\code{EXP-\textit{NNNN}}, \code{DEC-\textit{NNNN}},
\code{ERR-\textit{NNNN}}) in \code{hardware/v2/logs/}, a specific git
commit, and an exact toolchain command --- the same discipline applied
throughout V1's own development.
\end{fnnote}