From 8439ec4055f6be80dedeebb84bcc1f37dd11b34a Mon Sep 17 00:00:00 2001 From: Michele Bigi Date: Tue, 7 Jul 2026 07:49:07 +0200 Subject: [PATCH] Sync all project docs for firmware 0.8.3 completion. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Update READMEs, manual chapters, agent guides, CONTRIBUTING, and TODO to reflect T1–T8 done, encrypted NVS, CI gates, and pending HIL checklist. Co-authored-by: Cursor --- CONTRIBUTING.md | 33 +++-- README.md | 17 +-- Software/.cursor/rules/40-security.mdc | 8 +- Software/AGENTS.md | 23 ++-- Software/README.md | 5 +- Software/docs/TODO.md | 121 +++++++++--------- Software/docs/manual/ch-api.tex | 2 +- Software/docs/manual/ch-build.tex | 41 ++++++- Software/docs/manual/ch-classes.tex | 24 ++-- Software/docs/manual/ch-firmware.tex | 39 +++--- Software/docs/manual/ch-intro.tex | 4 +- Software/docs/manual/ch-sigmastudio.tex | 2 +- Software/docs/manual/manual.tex | 7 +- Software/instructions.md | 155 +++++++++--------------- WHERE-TO-PUT-THESE.md | 62 ++++++---- 15 files changed, 283 insertions(+), 260 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0307ca7..8a54bfc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -5,6 +5,9 @@ development conventions. The full, authoritative rules live in [`Software/AGENTS.md`](Software/AGENTS.md); this is the human-facing summary. +**Current firmware:** **0.8.3** on `main` — agent tasks T1–T8 complete; +device HIL pending first PCB. + DigiRadio is dual-licensed: hardware under **CERN-OHL-S v2**, firmware under **Apache-2.0**. By contributing, you agree your contributions are licensed under the same terms as the part of the project they touch. @@ -16,19 +19,28 @@ licensed under the same terms as the part of the project they touch. | Framework | ESP-IDF v5.5.x (native, not Arduino) | | Language | C++23, pinned `-std=gnu++23` | | Errors | `std::expected`; exceptions off| -| Docs | Doxygen (build must pass, see below) | +| Security | NVS + flash encryption (dev mode) | +| Docs | Doxygen + LaTeX manual sync (CI enforced)| On macOS, the host unit tests need a C++23 standard library: use Homebrew `llvm` (>= 18) or `gcc-14`, not the system Apple Clang. ## Build, test, docs -Device build / flash / monitor: +Si4684 blobs (local only, not in git): + +```bash +cd Software +python3 tools/fetch_si4684_firmware.py --dab-only +python3 tools/fetch_si4684_firmware.py --si46xx-dir /path/to/si46xx_firmware +``` + +Device build / flash / monitor (first encrypted flash: include `erase-flash`): ```bash idf.py set-target esp32s3 idf.py build -idf.py -p flash monitor +idf.py erase-flash flash monitor ``` Host unit tests (pure core, no hardware): @@ -40,17 +52,20 @@ cmake --build build-host ctest --test-dir build-host --output-on-failure ``` -Documentation (must exit 0; run from `Software/`): +Documentation and policy checks (must exit 0; run from `Software/`): ```bash doxygen Doxyfile python3 tools/check-manual-sync.py +python3 tools/check_si4684_blobs.py ``` -The LaTeX manual lives in `Software/docs/manual/` (canonical). The repository -root `docs/` is a symbolic link to that folder for convenience and Overleaf -/GitHub browsing. Design and HTTP JSON API: `ch-api.tex`; rebuild the PDF with -`latexmk -lualatex manual.tex` inside `docs/` or `Software/docs/manual/`. +After editing the web UI: `python3 tools/gzip-www.sh`. + +The LaTeX manual lives in `Software/docs/manual/` (canonical). Design and +HTTP JSON API: `ch-api.tex`; security: `docs/security-flash-nvs.md`. +Rebuild the PDF with `latexmk -lualatex manual.tex` inside +`Software/docs/manual/`. ## Coding conventions @@ -104,6 +119,7 @@ Before opening a PR, confirm: - [ ] Every class and method has its documentation block. - [ ] `doxygen Doxyfile` exits 0 with an empty warnings log. - [ ] `python3 tools/check-manual-sync.py` passes. +- [ ] `python3 tools/check_si4684_blobs.py` passes. - [ ] Manual updated: `ch-classes.tex` for new/changed public classes; `ch-api.tex` for new/changed HTTP endpoints. - [ ] Every method fits 80x24 and complexity <= 7. @@ -112,6 +128,7 @@ Before opening a PR, confirm: - [ ] No secret is loggable or stored in plaintext. - [ ] Register-level decisions cite the datasheet section. - [ ] No dynamic allocation in audio/ISR paths. +- [ ] Web UI changes regenerate `index.html.gz` via `tools/gzip-www.sh`. ## Editor setup (optional) diff --git a/README.md b/README.md index 481d517..85e9a14 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,7 @@ Vertical slices landed on `main` (newest first): | Version | Highlights | |---------|------------| | **0.8.3** | NVS + flash encryption (development mode), `initEncryptedStorage`, security docs | -| **0.8.2** | Tabbed configuration Web UI — now-playing, 6-band EQ, full API coverage | +| **0.8.2** | Tabbed Web UI + Si4684 blob CI policy | | **0.8.1** | `IntegrationService` — boot preset recall, tune orchestration (tuner + audio + NVS `last_preset`); services stub removed | | **0.8.0** | Preset reorder API/UI; broadcast metadata (RDS + DAB DLS); `readDabServiceData` driver path | | **0.7.1** | CI workflow (host tests, Doxygen, manual sync); Doxygen warnings cleared | @@ -101,7 +101,7 @@ Vertical slices landed on `main` (newest first): ``` DigiRadio/ -├── .github/workflows/ CI — host tests, Doxygen, manual class sync +├── .github/workflows/ CI — host tests, Doxygen, manual sync, Si4684 blob policy ├── Hardware/ │ ├── schematics/ Schematic (PDF) │ ├── gerber/ Gerber + drill (fabrication) @@ -121,7 +121,8 @@ DigiRadio/ │ │ └── net/ Wi-Fi, HTTP server, gzipped web UI │ ├── docs/ │ │ ├── manual/ LaTeX technical manual (canonical) -│ │ └── TODO.md Prioritised firmware backlog +│ │ ├── security-flash-nvs.md NVS/flash encryption + HIL checklist +│ │ └── TODO.md Agent backlog and completed tasks │ ├── Firmware/ Si4684 blobs + ADAU1701 SigmaStudio export │ ├── main/ app_main, hardware bootstrap │ └── tools/ Manual sync checker, Si4684 blob helpers @@ -148,7 +149,8 @@ Open **`Software/`** as the Cursor project so `AGENTS.md` and `.cursor/rules/` l cd Software idf.py set-target esp32s3 idf.py build -idf.py -p flash monitor +idf.py erase-flash flash # once when first enabling encryption (0.8.3+) +idf.py -p monitor ``` Host unit tests (no hardware): @@ -167,6 +169,7 @@ Quality gates (also enforced in CI): cd Software doxygen Doxyfile python3 tools/check-manual-sync.py +python3 tools/check_si4684_blobs.py ``` Full build notes, API table, and component map: [`Software/README.md`](Software/README.md). @@ -225,10 +228,8 @@ Generated C++ API reference: run `doxygen Doxyfile` → `Software/docs/api/html/ | **Prototype** | In fabrication (PCBWay) | | **Firmware** | **0.8.3** on `main` — encrypted NVS, full Web UI, integration, RDS/DLS | | **Web UI** | Tabbed SPA covering every REST endpoint | -| **HIL** | Security checklist pending PCB (`Software/docs/security-flash-nvs.md`) | -| **Production hardening** | Si4684 blob policy (T7), NVS encryption (T8) — open | - -Agent task list: [`Software/docs/TODO.md`](Software/docs/TODO.md). +| **HIL** | Device validation pending PCB (`Software/docs/security-flash-nvs.md`) | +| **Agent backlog** | Feature-complete on `main` (fw 0.8.3); see [`Software/docs/TODO.md`](Software/docs/TODO.md) | --- diff --git a/Software/.cursor/rules/40-security.mdc b/Software/.cursor/rules/40-security.mdc index 5393c1d..749d161 100644 --- a/Software/.cursor/rules/40-security.mdc +++ b/Software/.cursor/rules/40-security.mdc @@ -8,10 +8,10 @@ alwaysApply: false Full spec: @AGENTS.md §7.5. -- Stores Wi-Fi SSID/password, user credentials, station list. Encrypted - at rest (NVS encryption on an encrypted partition, or a device key in - eFuse). Confirm the mechanism against current ESP-IDF security docs - before implementing. +- Stores Wi-Fi SSID/password, station list, audio profiles, last-preset index. + **Encrypted at rest** — `CONFIG_NVS_ENCRYPTION` + flash encryption in + `sdkconfig.defaults` (development mode); `secure_store::initEncryptedStorage()` + before any NVS access. See `docs/security-flash-nvs.md`. - A `Secret` wrapper: no operator<<, no implicit conversion to a loggable string, buffer zeroised on destruction. - Secrets are never logged, never placed in URLs, never serialised to diff --git a/Software/AGENTS.md b/Software/AGENTS.md index 361f967..849478e 100644 --- a/Software/AGENTS.md +++ b/Software/AGENTS.md @@ -300,8 +300,8 @@ Layered, dependencies point inward only: ``` Shell (imperative): drivers, web server, NVS, tasks, ISRs — thin, no business logic [ESP-IDF, HW] - Application services: TunerService, AudioService, - ConfigService, NetworkService [orchestration] + Application services: TunerService, AudioService, StationService, + BluetoothService, IntegrationService [orchestration] Domain core (pure, host-testable): Station, Frequency, EqProfile, MixerState, Credential, boot-blob framing, validation [no HW headers] @@ -420,11 +420,12 @@ Layered, dependencies point inward only: ### 7.5 Secure storage -- Stores: Wi-Fi SSID + password, user credentials (name + password), - station/frequency list. **Encrypted at rest** — use NVS encryption on - an encrypted partition (with flash encryption enabled), or encrypt - payloads with a device key held in eFuse. Confirm the chosen mechanism - against current ESP-IDF security docs before implementing. +- Stores: Wi-Fi SSID + password, station/frequency list, audio profiles, + last-preset index. **Encrypted at rest** — NVS encryption with flash + encryption enabled in `sdkconfig.defaults` (development mode); keys in + `nvs_keys` partition; init via `secure_store::initEncryptedStorage()`. + Production release mode: `sdkconfig.defaults.production`. See + `docs/security-flash-nvs.md`. - **Secrets never leave their type.** A `Secret` wrapper: no `operator<<`, no implicit conversion to a loggable string, buffer zeroised on destruction. Secrets are never logged, never placed in URLs, never @@ -537,11 +538,11 @@ ESP-IDF header, so the host build stays hardware-free. ### Commands -Device build / flash / monitor: +Device build / flash / monitor (first encrypted flash: erase once): ``` idf.py set-target esp32s3 idf.py build -idf.py -p flash monitor +idf.py erase-flash flash monitor ``` Host unit tests (pure core; needs a C++23 stdlib compiler): @@ -552,9 +553,11 @@ cmake --build build-host ctest --test-dir build-host --output-on-failure ``` -Docs (must exit 0, empty warnings log): +Docs and policy (must exit 0, from `Software/`): ``` doxygen Doxyfile +python3 tools/check-manual-sync.py +python3 tools/check_si4684_blobs.py ``` ### Host toolchain note (macOS) diff --git a/Software/README.md b/Software/README.md index 48760ef..373d558 100644 --- a/Software/README.md +++ b/Software/README.md @@ -83,8 +83,9 @@ C++ signatures: `doxygen Doxyfile` → `docs/api/html/index.html`. | `docs/security-flash-nvs.md` | NVS + flash encryption and HIL checklist | | `docs/TODO.md` | Agent task list (prioritised backlog) | -See [`AGENTS.md`](AGENTS.md) §12 and [`instructions.md`](instructions.md) for -coding rules and slice roadmap. +See [`AGENTS.md`](AGENTS.md), [`instructions.md`](instructions.md), +[`docs/security-flash-nvs.md`](docs/security-flash-nvs.md), and +[`docs/TODO.md`](docs/TODO.md) for coding rules, security, and backlog. ## Licence diff --git a/Software/docs/TODO.md b/Software/docs/TODO.md index 16a27ea..31abb2a 100644 --- a/Software/docs/TODO.md +++ b/Software/docs/TODO.md @@ -1,93 +1,80 @@ # TODO — DigiRadio firmware -Task list for the coding agent. Work top to bottom; each task is a -vertical slice that keeps `main` building and the host tests green. - -**Before writing code, read `AGENTS.md`, `.cursor/rules/`, and -`instructions.md`.** Every task below must satisfy the Definition of Done -in `AGENTS.md §10`: Apache header on new files, doc block on every class -and method, `doxygen Doxyfile` exits 0, host tests pass, -`tools/check-manual-sync.py` passes, no primitive obsession, typed -errors, no plaintext secrets. - -Working directory for all commands is `Software/`. +Agent task list and hardware-in-the-loop backlog. Working directory for all +commands is `Software/`. **Current firmware:** `0.8.3` — NVS + flash encryption (dev mode), tabbed Web -UI, integration service, CI gate. +UI, integration service, RDS/DLS metadata, CI gate (4 jobs). + +**Before writing code, read `AGENTS.md`, `.cursor/rules/`, and +`instructions.md`.** Definition of Done: Apache header, doc blocks, +`doxygen Doxyfile` exits 0, host tests pass, `check-manual-sync.py` and +`check_si4684_blobs.py` pass, no plaintext secrets. --- -## Completed (fw 0.7.0–0.8.3) +## Completed agent tasks (T1–T8, fw 0.7.1–0.8.3) -- **Integration service (T5)** — preset recall with audio profile re-apply, - last-preset NVS, \texttt{app\_main} orchestration. -- **BT1035 pairing** — `AT+PAIR`, `AT+A2DPSTAT`, `AT+A2DPDISC`, - `BluetoothService`, REST + UI (not numbered below; landed with Slice 7). +| Task | Version | Summary | +|------|---------|---------| +| **T1** | 0.7.1 | Doxygen warnings cleared | +| **T2** | 0.7.1 | CI workflow (host tests, Doxygen, manual sync) | +| **T3** | 0.7.2 | Preset reorder API/UI, DAB playing ids in status | +| **T4** | 0.8.0 | RDS/DLS broadcast metadata | +| **T5** | 0.8.1 | `IntegrationService` — startup, preset recall, last-preset NVS | +| **T6** | 0.8.2 | Tabbed configuration Web UI (full REST coverage) | +| **T7** | 0.8.2 | Si4684 blob policy — gitignore, docs, `check_si4684_blobs.py` | +| **T8** | 0.8.3 | NVS + flash encryption — `initEncryptedStorage`, security docs | + +Also landed (not numbered): BT1035 pairing (`BluetoothService`), station presets +(fw 0.7.0), companion-chip boot (Slice 3), ADAU1701 runtime (Slice 5). --- -## P0 — Fix the build gate (do this first) +## P4 — Hardware-in-the-loop (when PCB arrives) -### T1. Clear the 16 Doxygen warnings — **DONE (fw 0.7.1)** -Fixed invalid `\texttt`/`\r`/`\ref` in doc blocks; documented -`Station` accessors and `NetBootstrap`/`SetupWebServer` parameters. -`doxygen Doxyfile` exits 0 with an empty warnings log. +Manual validation only — does not block host CI. -### T2. Add the CI workflow — **DONE (fw 0.7.1)** -`.github/workflows/ci.yml`: host `ctest`, Doxygen, manual sync on every -push/PR to `main`. +### H1. Encrypted NVS boot path +Follow [`docs/security-flash-nvs.md`](security-flash-nvs.md): first flash with +`idf.py erase-flash flash`, verify boot logs, Wi-Fi provisioning survives +reboot, presets and `last_preset` survive power cycle. + +### H2. End-to-end listening +Si4684 DAB/FM tune, ADAU1701 profile apply, BT1035 A2DP to headphones, +now-playing metadata in UI and `/api/tuner/status`. + +### H3. Production flash encryption (optional) +After H1 passes, trial build with `sdkconfig.defaults.production` overlay on +a sacrificial unit; confirm RELEASE mode policy before shipping. --- -## P1 — Missing domain features +## Open firmware polish (non-blocking) -### T3. Station / preset list — polish — **DONE (fw 0.7.2)** -Reorder API (`POST /api/stations/reorder`), DAB playing ids in tuner -status and preset save, UI Up/Dn, host tests. **Remaining:** device HIL -(preset survives reboot) — manual only. - -### T4. Broadcast metadata (RDS / DLS) — **DONE (fw 0.8.0)** -`BroadcastLabel`, RDS accumulator, DAB DLS accumulator, driver -`readDabServiceData`, status JSON fields, UI now-playing lines, host tests. - -### T5. Remove the services stub — integration service — **DONE (fw 0.8.1)** -`integration::IntegrationService` orchestrates startup, preset recall, -audio profile re-apply, and last-preset NVS. Stub removed; `app_main` and -`POST /api/stations/tune` delegate here. +- BT1035: device name, paired-device list, auto-reconnect AT (driver stubs open). +- Si4684: optional commands (STOP_DIGITAL_SERVICE, ensemble info) if product needs them. +- FM seek down (API today is seek-up only). --- -## P2 — User interface +## Quality gates (run from `Software/` before merge) -### T6. Complete the configuration Web UI — **DONE (fw 0.8.2)** -Tabbed SPA (`Now` / `Radio` / `Presets` / `Audio` / `BT` / `Wi‑Fi`): -now-playing hero with 5 s metadata poll, six-band EQ sliders, all REST -endpoints wired, companion-chip badges, `tools/gzip-www.sh` for the -embedded gzip blob. No debug routes in `SetupWebServer`. +```bash +cmake -S components/core/test -B build-host && cmake --build build-host +ctest --test-dir build-host --output-on-failure +doxygen Doxyfile +python3 tools/check-manual-sync.py +python3 tools/check_si4684_blobs.py +``` ---- - -## P3 — Procurement & hardening - -### T7. Si4684 firmware blob strategy (legal) — **DONE (fw 0.8.2)** -`Firmware/Si4684-Firmware/*.bin` gitignored; no blobs in git history. -Procurement documented in `Si4684-Firmware/README.md`; CI job -`si4684-blobs` runs `tools/check_si4684_blobs.py`. - -### T8. Flash/NVS encryption enablement — **DONE (fw 0.8.3)** -`CONFIG_NVS_ENCRYPTION` + flash encryption (development mode) in -`sdkconfig.defaults`; `secure_store::initEncryptedStorage()`; production -overlay `sdkconfig.defaults.production`; HIL checklist in -`docs/security-flash-nvs.md`. **Pending:** device verification when PCB -arrives. +After editing the web UI: `tools/gzip-www.sh`. --- ## Notes for the agent -- Prefer extending existing patterns over inventing new ones: copy the - shape of `AudioProfile` / `AudioProfileJson` / `IAudioProfileStore` for - new persisted models. -- Never invent Si4684 register/command details — cite AN649. -- One logical change per commit; 50/72 commit messages. -- After each task, run: host `ctest`, `doxygen Doxyfile`, - `tools/check-manual-sync.py` — all must pass before moving on. + +- Extend existing patterns (`AudioProfile` / `IAudioProfileStore` shape). +- Never invent Si4684 opcodes — cite AN649. +- One logical change per commit; 50/72 messages. +- Update `ch-classes.tex` / `ch-api.tex` when public API or HTTP changes. diff --git a/Software/docs/manual/ch-api.tex b/Software/docs/manual/ch-api.tex index 66e2356..426f9e5 100644 --- a/Software/docs/manual/ch-api.tex +++ b/Software/docs/manual/ch-api.tex @@ -6,7 +6,7 @@ implemented in \texttt{SetupWebServer}. Request bodies are parsed into domain types in the pure core (\texttt{components/core}) before any persistence or driver call. Exact C++ signatures live in the generated Doxygen output under \texttt{docs/api/}; this chapter documents the -wire protocol and behaviour as shipped in firmware~0.5.0 (Slices~1--5). +wire protocol and behaviour as shipped in firmware~0.8.3. \section{Transport and reachability} diff --git a/Software/docs/manual/ch-build.tex b/Software/docs/manual/ch-build.tex index e4fa63e..88f20f5 100644 --- a/Software/docs/manual/ch-build.tex +++ b/Software/docs/manual/ch-build.tex @@ -14,6 +14,7 @@ builds and is unit-tested on the host machine. Framework & ESP-IDF v5.5.x (native) \\ Language & C++23 (\texttt{-std=gnu++23}) \\ Error model & \texttt{std::expected}; exceptions off \\ + Security & NVS + flash encryption (dev mode); see \texttt{docs/security-flash-nvs.md} \\ Documentation & Doxygen (build must pass) \\ \bottomrule \end{tabular} @@ -21,14 +22,32 @@ builds and is unit-tested on the host machine. \label{tab:build-toolchain} \end{table} +\section{Si4684 firmware blobs (local only)} + +Proprietary Si4684 images are \textbf{not} in git. Before the first device +build: + +\begin{drcode}[Populate blobs (from Software/)] +python3 tools/fetch_si4684_firmware.py --dab-only +python3 tools/fetch_si4684_firmware.py --si46xx-dir /path/to/si46xx_firmware +python3 tools/check_si4684_blobs.py +\end{drcode} + +See \texttt{Firmware/Si4684-Firmware/README.md} for procurement options. + \section{Device build} +First flash after enabling encryption (fw~0.8.3+) requires a one-time erase: + \begin{drcode}[Build, flash, monitor] idf.py set-target esp32s3 idf.py build -idf.py -p flash monitor +idf.py erase-flash flash monitor \end{drcode} +Production flash-encryption release mode uses \texttt{sdkconfig.defaults.production} +as an overlay --- irreversible on the chip; see the security doc before use. + \section{Host unit tests} The pure core is tested on the host, with no board attached. On macOS the @@ -42,9 +61,12 @@ cmake --build build-host ctest --test-dir build-host --output-on-failure \end{drcode} +Thirteen test executables cover JSON parsing, audio design, station list, +integration service, and broadcast metadata accumulators. + \section{Documentation} -Documentation has two enforced checks, run from the \texttt{Software/} +Documentation has enforced checks, run from the \texttt{Software/} directory: \begin{enumerate} @@ -55,17 +77,21 @@ directory: \texttt{\textbackslash label\{cls:ClassName\}} section in \texttt{docs/manual/ch-classes.tex}. Design-level HTTP API documentation lives in Chapter~\ref{ch:api}. + \item \textbf{Si4684 blob policy} --- \texttt{tools/check\_si4684\_blobs.py} + ensures no proprietary \texttt{.bin} is tracked in git. \end{enumerate} \begin{drcode}[Docs (from Software/)] doxygen Doxyfile python3 tools/check-manual-sync.py +python3 tools/check_si4684_blobs.py +python3 tools/gzip-www.sh # after editing components/net/www/index.html \end{drcode} \section{Continuous integration} Every push and pull request to \texttt{main} runs -\texttt{.github/workflows/ci.yml} at the repository root (three parallel +\texttt{.github/workflows/ci.yml} at the repository root (four parallel jobs, all from the \texttt{Software/} directory): \begin{enumerate} @@ -74,6 +100,7 @@ jobs, all from the \texttt{Software/} directory): \item \textbf{Doxygen} --- must exit 0 with an empty \texttt{docs/api/doxygen-warnings.log}. \item \textbf{Manual sync} --- \texttt{tools/check-manual-sync.py}. + \item \textbf{Si4684 blobs} --- \texttt{tools/check\_si4684\_blobs.py}. \end{enumerate} To rebuild the PDF manual (requires a LaTeX installation): @@ -84,8 +111,14 @@ latexmk -lualatex manual.tex \end{drcode} \begin{drcaution}[Keep it green] -Both checks are part of the definition of done. A firmware change that +All checks are part of the definition of done. A firmware change that adds or modifies a public class, a REST endpoint, or its behaviour must update the Doxygen doc blocks, \texttt{ch-classes.tex} (for classes), and \texttt{ch-api.tex} (for HTTP) in the same change. \end{drcaution} + +\section{Hardware validation} + +Automated CI does not attach to a board. When the PCB is available, run the +checklist in \texttt{docs/security-flash-nvs.md} and the HIL items in +\texttt{docs/TODO.md} (section P4). diff --git a/Software/docs/manual/ch-classes.tex b/Software/docs/manual/ch-classes.tex index 507b152..47c2db4 100644 --- a/Software/docs/manual/ch-classes.tex +++ b/Software/docs/manual/ch-classes.tex @@ -16,16 +16,15 @@ added in the same change that introduces the class. A tooling check keeps this chapter in step with the code, so it is always current. \end{drnote} -The firmware is under active development. As each public class lands, its -section appears below, grouped by layer: domain core, application -services, and hardware drivers. +The class reference tracks firmware~0.8.3 on \texttt{main}. Public classes +are grouped by layer: domain core, application services, and hardware drivers. % ------------------------------------------------------------------ % Domain core (Slice 1) % ------------------------------------------------------------------ \section{FirmwareVersion}\label{cls:FirmwareVersion} -Strong type wrapping the firmware release identifier (e.g.\ \texttt{0.1.0}). +Strong type wrapping the firmware release identifier (e.g.\ \texttt{0.8.3}). Used by \texttt{HealthStatus} and the \texttt{/api/health} endpoint so version strings are never passed as bare \texttt{char*} across module boundaries. Invariant: non-empty at construction. @@ -65,11 +64,10 @@ tuner via \texttt{tuner::TunerService}; audio via presets via \texttt{station::StationService}. \section{NetBootstrap}\label{cls:NetBootstrap} -Owns network resources for setup or STA mode. -\texttt{start(store, tuner)} initialises the platform, joins stored Wi-Fi -when credentials exist, or falls back to the \texttt{DigiRadio-setup} -SoftAP. Must outlive \texttt{app\_main} for the process lifetime. -\texttt{start(store, tuner, audio)} also wires the audio REST routes. +Owns network resources for setup or STA mode. Initialises encrypted NVS via +\texttt{secure\_store::initEncryptedStorage()}, then Wi-Fi and the HTTP server. +Wires tuner, audio, Bluetooth, station, and integration services into REST +handlers. Must outlive \texttt{app\_main} for the process lifetime. % ------------------------------------------------------------------ % Domain core + secure store (Slice 2) @@ -89,9 +87,11 @@ Domain value pairing \texttt{WifiSsid} with a \texttt{Secret} password. Open networks use an empty password; WPA-PSK requires 8--63 characters. \section{ISecureStore}\label{cls:ISecureStore} -Abstract persistence boundary for credentials at rest. Slice~2 implements -Wi-Fi credential save/load/clear; station list and user credentials arrive -in later slices. Host tests use fakes; the shell uses \texttt{NvsSecureStore}. +Abstract persistence boundary for credentials and preset data at rest. +Wi-Fi credentials, station list JSON, and last-preset index use +\texttt{NvsSecureStore}; audio profiles use \texttt{NvsAudioProfileStore}. +Host tests use fakes; the shell uses NVS backends after +\texttt{initEncryptedStorage()}. \section{StaClient}\label{cls:StaClient} RAII STA join helper with an explicit connect timeout. Assumes diff --git a/Software/docs/manual/ch-firmware.tex b/Software/docs/manual/ch-firmware.tex index ba1cf15..44456da 100644 --- a/Software/docs/manual/ch-firmware.tex +++ b/Software/docs/manual/ch-firmware.tex @@ -64,7 +64,8 @@ itself. \node[layer, fill=black!8, below=of shell] (services) {% \textbf{Application services}\\[2pt] TunerService \textbullet\ AudioService \textbullet\ - ConfigService \textbullet\ NetworkService}; + StationService \textbullet\ BluetoothService \textbullet\ + IntegrationService}; \node[layer, fill=black!12, below=of services] (core) {% \textbf{Domain core (pure, host-tested)}\\[2pt] Station \textbullet\ Frequency \textbullet\ EqProfile \textbullet\ @@ -165,23 +166,31 @@ through \texttt{core::ISecureStore}, and the device reboots into STA mode on the next boot. The HTTP endpoints and JSON schemas are documented in Chapter~\ref{ch:api}. -\paragraph{Implemented (Slices~1--2).} +\paragraph{Implemented (fw 0.8.3).} \begin{itemize} - \item \texttt{GET /api/health} --- health DTO - (\texttt{core::HealthStatus}, serialised in the pure core). - \item \texttt{POST /api/wifi} --- Wi-Fi provisioning - (\texttt{core::WifiCredentials} via \texttt{parseWifiProvisionJson}). - \item \texttt{secure\_store::NvsSecureStore} --- NVS persistence for - SSID and PSK (\texttt{core::Secret}); station list and user - credentials arrive in later slices on the same - \texttt{ISecureStore} interface. + \item \textbf{Network} --- \texttt{GET /api/health}, \texttt{POST /api/wifi}, + SoftAP/STA state machine (\texttt{NetState}), tabbed gzipped web UI. + \item \textbf{Tuner} --- \texttt{/api/tuner/*} (FM/DAB tune, seek, services, + play); RDS and DAB dynamic labels in status JSON. + \item \textbf{Audio} --- \texttt{/api/audio/*} (profile, reset, stereo/bass + enhance); six-band EQ via \texttt{AudioService}. + \item \textbf{Bluetooth} --- \texttt{/api/bluetooth/*} (pair, stop, disconnect, + status). + \item \textbf{Presets} --- \texttt{/api/stations/*} including reorder and + integrated recall via \texttt{IntegrationService}. + \item \textbf{Storage} --- \texttt{NvsSecureStore}, \texttt{NvsAudioProfileStore}, + encrypted NVS + flash encryption (development mode); keys in + \texttt{nvs\_keys} partition; init via \texttt{initEncryptedStorage()}. \end{itemize} -Sensitive data --- Wi-Fi credentials today; user credentials and the -station/frequency list in later slices --- uses \texttt{core::Secret} so -values cannot be logged or implicitly converted to a string; buffers are -cleared on destruction. Firmware~0.8.3+ enables NVS and flash encryption at -rest via \texttt{secure\_store::initEncryptedStorage()} (see +\paragraph{Legacy note (Slices 1--2).} +The first slices introduced health, Wi-Fi provisioning, and the secure-store +boundary; the bullets above supersede the original slice-scoped list. + +Sensitive data uses \texttt{core::Secret} where applicable so values cannot +be logged or implicitly converted to a string; buffers are cleared on +destruction. Firmware~0.8.3 enables NVS and flash encryption at rest via +\texttt{secure\_store::initEncryptedStorage()} (see \texttt{docs/security-flash-nvs.md} and Chapter~\ref{sec:api-storage}). \section{Companion-chip boot at power-up} diff --git a/Software/docs/manual/ch-intro.tex b/Software/docs/manual/ch-intro.tex index 7beca88..8023d0c 100644 --- a/Software/docs/manual/ch-intro.tex +++ b/Software/docs/manual/ch-intro.tex @@ -4,7 +4,9 @@ DigiRadio is an open-source, high-fidelity digital radio receiver. It receives DAB+ and FM broadcasts, processes the audio through a dedicated signal processor, and streams the result over Bluetooth using a -high-resolution codec. The whole project --- hardware and firmware --- is +high-resolution codec. Firmware~0.8.3 on \texttt{main} provides encrypted +storage, a tabbed configuration web UI, and the full REST API documented +in Chapter~\ref{ch:api}. The whole project --- hardware and firmware --- is released as open source for the maker and audio community to study, build, and improve. diff --git a/Software/docs/manual/ch-sigmastudio.tex b/Software/docs/manual/ch-sigmastudio.tex index 009cd44..fa99354 100644 --- a/Software/docs/manual/ch-sigmastudio.tex +++ b/Software/docs/manual/ch-sigmastudio.tex @@ -218,7 +218,7 @@ SigmaStudio export (Section~\ref{sec:ss-export}). \label{sec:ss-enhancements} The SigmaStudio export does not include dedicated stereo widener or bass -boost blocks. Firmware~0.5.0 maps enhancement levels (0--100) onto the +boost blocks. Firmware maps enhancement levels (0--100) onto the existing Param EQ1 bands at runtime: \begin{itemize} diff --git a/Software/docs/manual/manual.tex b/Software/docs/manual/manual.tex index da66b00..c1dea31 100644 --- a/Software/docs/manual/manual.tex +++ b/Software/docs/manual/manual.tex @@ -29,9 +29,10 @@ \vfill {\color{drInk}\large Michele Bigi\par} -\vspace{1mm} -{\color{drGray} 2026 \quad\textbullet\quad Hardware: CERN-OHL-S v2 - \quad\textbullet\quad Firmware: Apache-2.0\par} +\vspace{2mm} +{\color{drGray}Firmware 0.8.3 \quad\textbullet\quad 2026\par} +\vspace{2mm} +{\color{drGray}Hardware: CERN-OHL-S v2 \quad\textbullet\quad Firmware: Apache-2.0\par} \vspace{2mm} {\color{drGray}\small \url{https://github.com/manvalan/DigiRadio}\par} \end{titlepage} diff --git a/Software/instructions.md b/Software/instructions.md index feb3ae2..91e1954 100644 --- a/Software/instructions.md +++ b/Software/instructions.md @@ -2,7 +2,10 @@ Read this together with `AGENTS.md` and everything under `.cursor/rules/`. Those define *how* to write code; this file defines -*what we are building* and *what to do first*. +*what we are building* and the current state on `main`. + +**Firmware on `main`:** **0.8.3** — all agent tasks T1–T8 complete; device +HIL pending PCB arrival. ## What DigiRadio is @@ -15,8 +18,8 @@ ESP32-S3 and coordinates three companion chips: - **FSC-BT1035 (QCC3056)** — Bluetooth 5.2 out with aptX Adaptive, controlled by AT commands over UART. -Plus: an elegant, essential web UI for network configuration; encrypted -storage for Wi-Fi and user credentials and the station list. +Plus: tabbed web UI for provisioning and control; **encrypted NVS** for +Wi-Fi credentials, presets, audio profiles, and last-preset index. Repository: https://github.com/manvalan/DigiRadio @@ -29,116 +32,72 @@ Repository: https://github.com/manvalan/DigiRadio | Errors | `std::expected` (native); exceptions OFF | | DSP boot | ESP32 writes ADAU1701 RAM at every boot (no EEPROM) | | Architecture| Functional core (pure, host-tested) + imperative shell| -| Docs | Doxygen, build must exit 0 (enforced) | +| Security | NVS + flash encryption (dev mode); see `docs/security-flash-nvs.md` | +| Docs | Doxygen + LaTeX manual sync (CI enforced) | | HW licence | CERN-OHL-S v2 · FW licence: Apache-2.0 | +## Roadmap status + +| Slice / task | Status | Notes | +|--------------|--------|-------| +| 1 Walking skeleton | Done | SoftAP, gzipped UI, `/api/health` | +| 2 Secure store + Wi-Fi | Done | `ISecureStore`, STA provisioning | +| 3 Companion-chip boot | Done | Si4684 + ADAU1701 from `Firmware/` | +| 4 Station presets | Done (0.7.0) | NVS `station_list`, full `/api/stations/*` | +| 5 ADAU1701 runtime | Done | EQ, mixer, enhancements, audio API | +| 6 Si4684 tuning | Done | FM/DAB tune, seek, RSQ, RDS, DAB services/DLS | +| 7 BT1035 | Mostly done | Pairing, A2DP stat/disconnect; name/plist AT open | +| 8 Integration | Done (0.8.1) | `IntegrationService`, last-preset NVS | +| T6 Web UI | Done (0.8.2) | Tabbed SPA, all REST endpoints | +| T7 Si4684 blobs | Done (0.8.2) | Local-only `.bin`, CI policy check | +| T8 NVS encryption | Done (0.8.3) | `initEncryptedStorage`; HIL when PCB ready | + +Next work: **hardware-in-the-loop** (`docs/TODO.md` § P4), not new features +unless the user requests them. + ## Working agreement -- **Confirm understanding before writing code.** On kickoff, summarise - the plan and list any blockers or unclear hardware invariants first. -- **Blockers first**, always. State risks before solutions. -- **One vertical slice at a time.** `main` always builds and runs. -- Every file gets the Apache header; every class/method its doc block; - `doxygen Doxyfile` stays green. Small commits, 50/72 messages. -- Never invent a register/opcode/boot step — cite the datasheet or stop. - -## Roadmap (slices, in order) - -1. **Walking skeleton** — done (Slice 1). -2. **Secure store + Wi-Fi provisioning** — done (Slice 2). -3. **Companion-chip boot** — done (Slice 3): Si4684 DAB + ADAU1701 RAM load. -4. **Station/frequency list model + persistence + UI** — done (fw 0.7.0): - `Station`/`StationList`, NVS key `station_list`, `/api/stations/*`, Presets UI. -5. Si4684 tuning: RSQ, station list, DAB properties. -6. **ADAU1701 runtime** — done (Slice 5): safeload EQ + input mixer + HTTP. -7. **FSC-BT1035 driver** — init + pairing (AT+PAIR, A2DP stat/disconnect) done; - name/plist/reconnect AT still open. -8. Integration: TunerService + AudioService end to end. +- **Blockers first** — state risks before solutions. +- **One vertical slice at a time** — `main` always builds; host tests green. +- Apache header + Doxygen doc blocks on every file/class/method. +- Never invent register/opcode/boot steps — cite the datasheet or stop. +- After changes: `ctest`, `doxygen`, `check-manual-sync.py`, `check_si4684_blobs.py`. ## Slice 1 — Walking skeleton (complete) -Goal: exercise the whole toolchain end to end with zero chip hardware, -so every later slice drops into a working frame. - -Build: -- Top-level ESP-IDF project targeting `esp32s3`. -- `sdkconfig.defaults` sets C++23, exceptions off, NVS + flash encryption - (development mode). Production overlay: `sdkconfig.defaults.production`. -- The `components/core` component compiles both under ESP-IDF and - standalone on the host. - -Behaviour: -- `app_main` starts a FreeRTOS task that logs a heartbeat on a timer. -- Bring up SoftAP with a known SSID (e.g. `DigiRadio-setup`). -- Start an HTTP server serving one minimal gzipped page from flash. -- Expose `GET /api/health` returning a typed DTO serialised by the pure - core, e.g. `{"status":"ok","fw":"0.3.0"}`. - -Documentation (required): -- Doxygen doc blocks on every class/method; `doxygen Doxyfile` green. -- Manual: class sections in `docs/manual/ch-classes.tex`; - HTTP API in `docs/manual/ch-api.tex`. -- `python3 tools/check-manual-sync.py` green. +- ESP-IDF `esp32s3`, C++23, `components/core` host-testable. +- SoftAP `DigiRadio-setup`, gzipped page, `GET /api/health`. +- Current health JSON includes `fw` (today **0.8.3**) and companion-chip flags. ## Slice 2 — Secure store + Wi-Fi STA (complete) -Goal: persist Wi-Fi credentials and join the configured network after -provisioning; fall back to SoftAP when no credentials or join fails. - -Build on Slice 1: -- `core::ISecureStore` interface + `secure_store::NvsSecureStore` (NVS). -- `core::Secret`, `WifiSsid`, `WifiCredentials`, `parseWifiProvisionJson`. -- `net::StaClient`, `NetBootstrap::start(store)` state machine. -- `POST /api/wifi` + provisioning form in the web UI; reboot after save. - -Acceptance criteria: -- [x] Provisioning via SoftAP saves credentials and reboots; next boot joins STA. -- [x] Host tests for health JSON and Wi-Fi provision parse/serialise. -- [x] Doxygen green; manual sync green; `ch-api.tex` documents endpoints. -- [x] No ESP-IDF headers in `components/core`. - -Out of scope for Slice 2: station list (later slices). NVS encryption landed -in fw 0.8.3 — see `docs/security-flash-nvs.md`. -(production), chip drivers. +- `ISecureStore`, `NvsSecureStore`, `StaClient`, `NetBootstrap`. +- `POST /api/wifi` + Wi-Fi tab in web UI. +- NVS encryption enabled in fw 0.8.3 via `initEncryptedStorage()`. ## Slice 3 — Companion-chip boot (complete) -Goal: load Si4684 DAB firmware and ADAU1701 SigmaStudio program from -`Firmware/` on every boot, before network bring-up. +- Si4684 blobs local-only (`tools/fetch_si4684_firmware.py`). +- `Si4684Driver`, `Adau1701Driver`, `HardwareBootstrap` before network. +- Device flash: pending HIL on first PCB. -Build: -- `Firmware/Si4684-Firmware/` — `rom_patch_016.bin`, `dab_firmware.bin`, - `fm_firmware.bin` (FM via `tools/fetch_si4684_firmware.py --si46xx-dir`). -- `Firmware/ADAU1701-Firmware/` — SigmaStudio export (`DigiRadio_IC_1.h`, …). -- `core::IFirmwareBlobReader` + `EmbeddedBlobReader` for chunked HOST_LOAD. -- `si4684::Si4684Driver`, `adau1701::Adau1701Driver`, `HardwareBootstrap`. +## Slices 4–8 — Presets, audio, tuner, BT, integration (complete) -Behaviour: -- `app_main` calls `HardwareBootstrap::boot()` first (Si4684, then ADAU1701). -- On failure, firmware logs and halts before Wi-Fi. +- Presets: `StationService`, reorder, integration recall with audio profile. +- Audio: six-band EQ, enhancements, `NvsAudioProfileStore`. +- Tuner: RDS/DLS metadata in status JSON and Now Playing UI. +- Bluetooth: `BluetoothService`, pairing REST + UI. +- Integration: boot loads last preset; `POST /api/stations/tune` orchestrates tune + audio + NVS. -Acceptance criteria: -- [x] AN649 boot sequence with streaming blobs (no full image on heap). -- [x] ADAU1701 reset + I2C + `default_download_IC_1()` replay. -- [x] Host test for `EmbeddedBlobReader`; manual sync green. -- [ ] Device flash verified (requires ESP-IDF toolchain on build host). +## Quality gates (from `Software/`) -## Slice 5 — ADAU1701 runtime + audio API (complete) +```bash +cmake -S components/core/test -B build-host && cmake --build build-host +ctest --test-dir build-host --output-on-failure +doxygen Doxyfile +python3 tools/check-manual-sync.py +python3 tools/check_si4684_blobs.py +``` -Goal: safeload mixer/EQ/master on the ADAU1701 at runtime; persist user -profiles in NVS; expose REST and web UI controls. - -Build on Slice 3–4: -- Pure core: `GainDb`, `EqProfile`, `AudioProfile`, `IDsp`, biquad design, - `parseAudioProfileJson` / `serializeAudioProfileJson`. -- Driver: `sigma_safeload_*`, extended `Adau1701Driver`, `Adau1701Dsp`. -- Service: `audio::AudioService`, `secure_store::NvsAudioProfileStore`. -- HTTP: `GET/PUT /api/audio/profile`, `POST /api/audio/reset`; Audio - section in the web UI. Firmware **0.5.0**. - -Acceptance criteria: -- [x] Safeload volume/mixer/EQ without direct param RAM writes during audio. -- [x] Profile load/apply after ADAU boot; NVS round-trip via JSON. -- [x] Host tests for biquad fixpoint and audio profile JSON. -- [x] Doxygen green; manual sync green; `ch-api.tex` documents audio routes. -- [ ] Device flash verified on hardware. +First device flash with encryption: `idf.py erase-flash flash` — see +`docs/security-flash-nvs.md`. diff --git a/WHERE-TO-PUT-THESE.md b/WHERE-TO-PUT-THESE.md index 70443d3..70dfcb6 100644 --- a/WHERE-TO-PUT-THESE.md +++ b/WHERE-TO-PUT-THESE.md @@ -1,44 +1,54 @@ # DigiRadio — repository documentation package -Copy the contents into your repo keeping this structure. +Canonical layout for this repository: ``` -DigiRadio/ <- repo root -├── LICENSE CERN-OHL-S v2 (hardware) +DigiRadio/ <- repo root (README, Hardware, LICENSE) ├── CONTRIBUTING.md dev conventions (human-facing) -├── .gitignore +├── README.md project overview (fw 0.8.3) └── Software/ firmware project root (open THIS in Cursor) ├── LICENSE Apache-2.0 (firmware) ├── AGENTS.md authoritative coding rules - ├── instructions.md agent kickoff briefing (Slice 1) + ├── instructions.md agent kickoff + roadmap status + ├── sdkconfig.defaults C++23, NVS + flash encryption (dev mode) + ├── sdkconfig.defaults.production release-mode overlay (irreversible) + ├── partitions.csv nvs + nvs_keys partitions ├── Doxyfile API docs generation + enforcement ├── apache-header.txt header to paste in each source file ├── .cursor/rules/*.mdc Cursor scoped rules (6 files) ├── tools/ - │ └── check-manual-sync.py enforces "a section per public class" + │ ├── check-manual-sync.py one LaTeX section per public class + │ ├── check_si4684_blobs.py no proprietary .bin in git + │ ├── fetch_si4684_firmware.py local blob procurement + │ └── gzip-www.sh regenerate embedded web UI gzip └── docs/ - └── manual/ the technical manual (LaTeX) — canonical - ├── manual.tex main file - ├── digiradio-manual.sty style (Optima-like, boxes, listings) - ├── ch-*.tex chapters - └── manual.pdf compiled preview - -At the **repository root**, `docs/` is a **symbolic link** to -`Software/docs/manual/` (one source of truth; do not duplicate .tex here). + ├── TODO.md agent + HIL backlog + ├── security-flash-nvs.md encryption + device checklist + └── manual/ LaTeX technical manual (canonical) + ├── manual.tex + ├── ch-*.tex + └── manual.pdf optional compiled preview ``` -## Build the manual - cd docs # symlink → Software/docs/manual - latexmk -lualatex manual.tex # real Optima on macOS - # or: cd Software/docs/manual && latexmk -lualatex manual.tex +LaTeX sources live in `Software/docs/manual/`. Build the PDF from there: -## Enforcement in CI (run from Software/) - doxygen Doxyfile # API docs must pass - python3 tools/check-manual-sync.py # manual must be in sync +```bash +cd Software/docs/manual && latexmk -lualatex manual.tex +``` + +## CI enforcement (from `Software/`) + +```bash +doxygen Doxyfile +python3 tools/check-manual-sync.py +python3 tools/check_si4684_blobs.py +``` + +Host tests: `cmake -S components/core/test -B build-host && ctest --test-dir build-host`. ## Notes -- Two LICENSE files: CERN-OHL-S at root (hardware), Apache-2.0 in - Software/ (firmware). GitHub auto-detects both. -- docs/api/ (Doxygen output) is git-ignored; the manual PDF is optional - to commit (source .tex is the master). -- Open Software/ as the Cursor project so rules and AGENTS.md load. + +- Two LICENSE files: CERN-OHL-S at repo root (hardware), Apache-2.0 in + `Software/` (firmware). +- Si4684 `.bin` blobs are gitignored — never commit them. +- Open `Software/` as the Cursor project so rules and `AGENTS.md` load.