docs: ESP32 boot chain comparison and bring-up report

- esp32-boot-chain-comparison.md: IDF vs MCUboot->U-Boot comparison covering memory regions, cache/MMU sequences, image formats, and full feature matrix. - esp32-mcuboot-uboot-bringup-report.md: complete bring-up log with root cause analysis (IROM at 0x40800000, I-bus/D-bus split, cache bus masks, 64KB alignment requirement).
2026-07-16 16:44:25 +01:00 · 2026-07-16 16:44:25 +01:00 · 3961c6c42a
commit 3961c6c42a
parent 9b1c94d2d2
2 changed files with 424 additions and 0 deletions
--- a/docs/esp32-boot-chain-comparison.md
+++ b/docs/esp32-boot-chain-comparison.md
@ -0,0 +1,286 @@
+# ESP32 Boot Chain: ESP-IDF vs UniversalisOS (MCUboot→U-Boot→UOS)
+
+## 1. Complete Boot Sequence Comparison
+
+### 1.1 ESP-IDF Flow
+
+```
+ROM (1st stage, mask ROM @ 0x40000000)
+ │  - Reads ESP image header (magic 0xE9) from flash @ 0x1000
+ │  - Parses segment headers, loads segments per Memory Types:
+ │    · BYTE_ACCESSIBLE/DRAM → memcpy to DRAM
+ │    · CACHE_APP → flash-MMU maps flash pages to virtual address
+ │    · IRAM → memcpy to IRAM
+ │  - Sets up SPI flash, initial cache/MMU config
+ │  - Jumps to entry point (bootloader)
+ │
+ └─ 2nd Stage Bootloader (IDF bootloader @ flash 0x1000)
+     │  bootloader_init():
+     │    - bootloader_init_mem()  → clear BSS, check stack
+     │    - bootloader_clock_configure()
+     │    - bootloader_console_init()
+     │    - bootloader_reset_mmu():
+     │        Cache_Read_Disable(0)
+     │        Cache_Flush(0)
+     │        mmu_init(0)              ← HARD reset of MMU controller
+     │        DPORT unmask DROM0
+     │    - bootloader_flash_update_id()
+     │    - bootloader_init_spi_flash()  → SPI flash init
+     │    - bootloader_enable_random()
+     │
+     │  bootloader_utility_load_image():
+     │    - Parse app partition, read ESP image header
+     │    - For each segment: memcpy IRAM/DRAM segments to RAM
+     │    - Record DROM/IROM segment addrs for MMU mapping
+     │    - load_image() → unpack_load_app()
+     │
+     │  set_cache_and_start_app():
+     │    Cache_Read_Disable(0)          ← disable cache
+     │    Cache_Flush(0)                 ← flush D-cache to SRAM
+     │    mmu_hal_unmap_all()            ← SOFT reset: clear page table only
+     │    cache_flash_mmu_set(0, DROM)   ← map DROM flash pages
+     │    cache_flash_mmu_set(1, DROM)   ← map DROM for APP CPU too
+     │    cache_flash_mmu_set(0, IROM)   ← map IROM flash pages
+     │    cache_flash_mmu_set(1, IROM)   ← map IROM for APP CPU too
+     │    cache_ll_l1_get_bus() / enable ← enable cache bus masks
+     │    Cache_Read_Enable(0)           ← re-enable cache
+     │    isync
+     │    bootloader_atexit()            ← clean up (WDT, uart flush)
+     │    (*entry)()                     ← jump to app IROM entry
+     │
+     └─ Application (FreeRTOS app)
+         - Runs from IROM (flash-cached @ 0x400D0000+)
+         - data/bss in DRAM
+```
+
+### 1.2 UniversalisOS Flow (our chain)
+
+```
+ROM (1st stage, mask ROM)
+ │  - Reads ESP image header from flash @ 0x1000
+ │  - Loads MCUboot segments (CACHE_APP + DRAM + IRAM)
+ │  - Jumps to MCUboot entry
+ │
+ └─ MCUboot (2nd stage bootloader @ flash 0x1000)
+     │  bootloader_init():  [same ESP-IDF HAL, no major diffs]
+     │    - bootloader_reset_mmu() (via bootloader_init → init_ext_mem)
+     │
+     │  main() → do_boot():
+     │    - Validate image, check swap type
+     │    - start_cpu0_image():
+     │        esp_app_image_load():
+     │          - Parse MCUboot's esp_image_load_header_t (magic 0xACE637D3)
+     │          - Load DRAM via memcpy (→ 0x3FFAF000+)
+     │          - **IROM path**: cache_flash_mmu_set to map flash→IROM
+     │          - **IRAM path**: memcpy (BROKEN on ESP32 — D-bus can't write I-bus)
+     │        jump to entry_addr
+     │
+     └─ U-Boot (loaded by MCUboot @ IROM 0x400D0000+)
+         │  _start:
+         │    - Print 'X', set VECBASE, unpack reloc, init UART
+         │    - board_init_f → board_init_r → U-Boot prompt
+         │
+         └─ UniversalisOS (loaded by U-Boot)
+```
+
+### 1.3 Critical Difference: IRAM vs IROM
+
+| Aspect | ESP-IDF | UniversalisOS |
+|--------|---------|---------------|
+| Bootloader code location | IRAM (0x40078000) + CACHE_APP segment | IRAM (0x40078000 loader) + IRAM (0x40090000 main) |
+| App code location | **IROM** (flash-cached, 0x400D0000+) | Was **IRAM** (SRAM, 0x40080000) — **BROKEN** |
+| App code loading | flash-MMU maps flash pages to virtual IROM | Was memcpy to SRAM — **ESP32 D-bus can't write I-bus** |
+| Handoff sequence | Disable cache → unmap_all → map DROM → map IROM → bus enable → Enable cache → isync → jump | Same pattern (current code) |
+| MMU reset | `mmu_hal_unmap_all()` (soft, entries only) | Was `mmu_init(0)` (hard, corrupts controller) — **fixed** |
+
+## 2. Memory Region Comparison
+
+### 2.1 ESP32 Physical Memory Map
+
+```
+Bus     Address              Size    Content
+────    ────────             ────    ───────
+        0x4000_0000-0x4005_FFFF  384KB  ROM (mask ROM, read-only)
+I-bus   0x4007_0000-0x4007_7FFF   32KB  SRAM0 — ROM cache (reserved)
+I-bus   0x4007_8000-0x4007_A800   10KB  MCUboot LOADER (CACHE_APP segment)
+I-bus   0x4007_A800-0x4008_FFFF   86KB  GAP — free
+I-bus   0x4008_0000-0x4008_003F   —     UserExceptionVector (Xtensa hw)
+I-bus   0x4008_0100-0x4008_01xx   —     KernelExceptionVector
+I-bus   0x4008_03C0-0x4008_xxxx   —     DoubleExceptionVector
+I-bus   0x4009_0000-0x4009_E800   58KB  MCUboot IRAM MAIN (RECLAIMABLE by app)
+I-bus   0x400A_0000-0x400B_FFFF  128KB  SRAM1 (free, dual-bus)
+I-bus   0x400C_0000-0x40FF_FFFF    4MB  IROM — flash-cached execution
+D-bus   0x3F40_0000-0x3F80_0000    4MB  DROM — flash-cached data
+D-bus   0x3FFA_E000-0x3FFC_FFFF  200KB  SRAM2 — DRAM (data/bss/stack)
+D-bus   0x3FFE_0000-0x3FFF_FFFF  128KB  SRAM1 — DRAM (dual-bus mirror)
+```
+
+### 2.2 IDF App Memory Layout
+
+```
+Code:  0x400D_0000+  (IROM, flash-cached via MMU)
+RO:    0x3F40_0000+  (DROM, flash-cached via MMU)
+Data:  0x3FFB_0000+  (DRAM, SRAM2/SRAM1)
+BSS:   0x3FFx_xxxx   (DRAM, after data)
+Stack: 0x3FFF_xxxx   (DRAM, top of SRAM1)
+```
+
+### 2.3 U-Boot Memory Layout (new IROM design)
+
+```
+Code:   0x400D_0000+  (IROM, flash-cached — same as IDF app)
+RO:     0x3FFB_0000+  (DRAM, same as IDF)
+Data:   0x3FFB_xxxx   (DRAM, after RO)
+BSS:    0x3FFB_xxxx   (DRAM, after data)
+Stack:  0x4000_0000-   (DRAM, top of address space)
+```
+
+### 2.4 The IRAM Problem (why memcpy never worked)
+
+ESP32 has **split bus architecture**:
+- I-bus (instruction bus): fetches code from 0x40000000+
+- D-bus (data bus): reads/writes data from 0x3FFxxxxx+
+
+A `memcpy((void*)0x40080000, ...)` uses D-bus to write to I-bus address.
+The D-bus does NOT route to SRAM0 at that address. The write goes through
+the flash cache controller, which requires MMU entries for the target address.
+If no MMU entry exists, the write either goes nowhere or faults.
+
+**The only safe way to execute code on ESP32 is from IROM (flash-cached).**
+Direct IRAM loading via memcpy is architecturally impossible from software.
+
+## 3. Cache / MMU Handling Comparison
+
+### 3.1 ROM Initial State
+
+After ROM boots the 2nd stage:
+- VECBASE = 0x4000_0000 (ROM exception vectors)
+- DROM0 bus: unmasked (ROM clears mask)
+- IROM0: mapped by ROM to the bootloader's CACHE_APP segments
+- D-cache: enabled, contains bootloader code cache lines
+- I-cache: enabled, contains bootloader IROM cache lines
+- MMU table: populated with bootloader's flash page mappings
+
+### 3.2 IDF Bootloader Init
+
+`bootloader_reset_mmu()` (bootloader_esp32.c:43-69):
+```
+Cache_Read_Disable(0);  // stop cache reads
+Cache_Read_Disable(1);  // APP CPU too
+Cache_Flush(0);         // write back D-cache to SRAM
+Cache_Flush(1);
+mmu_init(0);            // HARD RESET: clear ALL MMU entries + controller state
+mmu_init(1);            // APP CPU too
+DPORT unmask DROM0;      // restore DROM0 bus so flash reads work
+```
+After this, the bootloader must re-initialize SPI flash and re-read
+its own header from flash. ALL previous MMU mappings are destroyed.
+
+### 3.3 IDF App Load
+
+`set_cache_and_start_app()` (bootloader_utility.c:1036-1158):
+```
+Cache_Read_Disable(0);  // stop cache reads
+Cache_Flush(0);         // write back D-cache
+mmu_hal_unmap_all();     // SOFT RESET: clear page table, keep controller state
+cache_flash_mmu_set(0, DROM); // map app's data from flash
+cache_flash_mmu_set(1, DROM); // for APP CPU
+cache_flash_mmu_set(0, IROM); // map app's code from flash
+cache_flash_mmu_set(1, IROM); // for APP CPU
+cache_ll_l1_enable_bus();    // enable bus masks
+Cache_Read_Enable(0);   // re-enable cache
+isync;                   // flush prefetch pipeline
+bootloader_atexit();     // clean up WDT, flush UART
+(*entry)();              // JUMP
+```
+
+Key: uses `mmu_hal_unmap_all()` NOT `mmu_init()`. The soft reset only
+clears the page table entries, leaving the MMU controller configured.
+`mmu_init()` hard-resets the controller which requires full re-init.
+
+### 3.4 Our MCUboot Flow
+
+```
+Cache_Read_Disable(0);  // ✓
+Cache_Flush(0);         // ✓
+// (no mmu_hal_unmap_all — we don't have this HAL function)
+cache_flash_mmu_set(0, IROM); // ✓ rc=0
+Cache_Read_Enable(0);   // ✓
+isync;                   // ✓
+(*entry)();              // JUMP
+```
+
+**Missing vs IDF:**
+1. No `mmu_hal_unmap_all()` — but we don't need it since we're adding
+   entries on top of MCUboot's existing mappings
+2. No DROM mapping for U-Boot — U-Boot's RO data is in DRAM, not DROM
+3. No bus mask enable — MCUboot's existing bus masks should still be active
+4. No APP CPU mapping — ESP32 single-core during boot
+
+## 4. Image Formats
+
+### 4.1 ESP Image Format (ROM loader)
+```
+[ESP image header 24B]    magic=0xE9, segment_count, entry_addr
+[Segment header 8B] × N   load_addr, data_len
+[Segment data]             raw bytes
+[Checksum 1B]
+[SHA256 32B] (optional)
+```
+Segment types determined by esptool from ELF segment VMA:
+- 0x3FFxxxxx → BYTE_ACCESSIBLE, DRAM
+- 0x4007xxxx → CACHE_APP (flash-mapped, no memcpy)
+- 0x4008xxxx → IRAM (memcpy to SRAM)
+- 0x400Dxxxx → CACHE_APP (flash-mapped, no memcpy)
+
+### 4.2 MCUboot Image Format (our chain)
+```
+[MCUboot header 32B]      magic=0x96f3b83d, image size, version
+[Load header 96B]         magic=0xACE637D3, entry_addr, iram_*, irom_*, dram_*
+[IRAM data]               code segment (legacy, memcpy)
+[DRAM data]               data segment
+[MCUboot TLV]             hash, signatures
+```
+Our wrapper uses:
+- irom_map_addr, irom_flash_offset, irom_size → flash-MMU mapped
+- dram_dest_addr, dram_flash_offset, dram_size → memcpy to DRAM
+
+## 5. Feature Comparison Matrix
+
+| Feature | ESP-IDF Bootloader | MCUboot (our chain) | U-Boot (our chain) |
+|---------|-------------------|---------------------|-------------------|
+| **ROM loading** | ESP image (0xE9) | ESP image (0xE9) | N/A (loaded by MCUboot) |
+| **Code execution** | IROM (flash-cached) | IRAM (SRAM) | IROM (flash-cached) |
+| **Data storage** | DRAM (memcpy) | DRAM (memcpy) | DRAM (memcpy) |
+| **Flash MMU** | `mmu_hal_unmap_all` + `cache_flash_mmu_set` × 4 | `cache_flash_mmu_set` × 1 | N/A (runs from MCUboot's MMU) |
+| **Cache handling** | Disable→Flush→Unmap→Map→Enable→isync | Disable→Flush→Map→Enable→isync | Cache already enabled by MCUboot |
+| **VECBASE** | App sets it (IRAM or IROM) | Set to 0x40080000 (old IRAM) | Sets VECBASE to IROM 0x400D0000 |
+| **Exception vectors** | App provides, at VECBASE offsets | None (ROM vectors) | Built-in, at IROM offsets |
+| **Secure boot** | Yes (RSA/ECDSA) | Yes (MCUboot signature validation) | N/A (trusts MCUboot) |
+| **OTA updates** | Yes (factory/OTA partitions) | Yes (primary/secondary slots) | N/A (loaded by MCUboot) |
+| **Flash encryption** | Yes (AES-XTS) | Yes (MCUboot supports) | N/A |
+| **Image validation** | SHA256 checksum | SHA256 via imgtool TLV | N/A |
+| **Multi-image** | No (single app) | Yes (multiple slots) | N/A |
+| **Stack location** | DRAM (top of SRAM1) | DRAM (MCUboot's loader segment) | DRAM (0x40000000 - 16) |
+| **Boot time** | ~50ms | ~100ms (MCUboot validation) | ~200ms (U-Boot init) |
+
+## 6. Root Cause Summary
+
+The fundamental issue was: **ESP32 D-bus cannot write to I-bus addresses.**
+
+Our original approach loaded U-Boot code to IRAM (0x40080000+) via memcpy,
+which uses D-bus stores. ESP32's bus architecture routes D-bus writes to the
+flash cache controller, not directly to SRAM0. The MMU must have valid entries
+for the target address to route writes to physical SRAM.
+
+The fix: **Run U-Boot from IROM (flash-cached at 0x400D0000+).** This matches
+how ESP-IDF runs applications. The flash MMU maps flash pages to IROM virtual
+addresses. The CPU fetches instructions via I-cache → MMU → flash SPI controller.
+No memcpy needed — the code executes directly from flash through the cache.
+
+This required:
+1. U-Boot linker script migrated from IRAM (0x40080000) to IROM (0x400D0000)
+2. MCUboot `esp_loader.c` updated to map IROM via `cache_flash_mmu_set`
+3. Wrapper tool updated to emit IROM-segment load headers
+4. Flash offset aligned to 64KB (ESP32 MMU page size)
+5. VECBASE moved from IRAM to IROM address space
--- a/docs/esp32-mcuboot-uboot-bringup-report.md
+++ b/docs/esp32-mcuboot-uboot-bringup-report.md
@ -0,0 +1,138 @@
+# ESP32 MCUboot → U-Boot Boot Chain — Bring-Up Report
+
+**Date:** 2026-07-16
+**Target:** ESP32-D0WDQ6-V3 (Xtensa LX6)
+**Status:** ✅ Working
+
+## Overview
+
+UniversalisOS boot chain on ESP32: ROM (1st stage) → MCUboot (2nd stage) → U-Boot.
+
+MCUboot validates and loads U-Boot from the primary flash slot. U-Boot runs
+from IROM (flash-cached at 0x40800000) with data in DRAM at 0x3FFAF000.
+
+## The Fundamental Discovery
+
+### ESP32 IROM is at 0x40800000, not 0x400D0000
+
+The ESP32 memory map (from `esp-hal-3rdparty/components/soc/esp32/include/soc/ext_mem_defs.h`):
+
+| Region | Bus | Address Range | Purpose |
+|--------|-----|---------------|---------|
+| IRAM0  | IBUS0 | 0x400D0000-0x40400000 | SRAM / configurable cache |
+| IRAM1  | IBUS1 | 0x40400000-0x40800000 | SRAM / configurable cache |
+| **IROM0** | **IBUS2** | **0x40800000-0x40C00000** | **Flash-cached execution (dedicated)** |
+| DROM0  | DBUS0 | 0x3F400000-0x3F800000 | Flash-cached data |
+| DRAM1  | DBUS1 | 0x3F800000-0x3FC00000 | PSRAM / configurable data cache |
+
+The ESP32's IROM (where flash-MMU-mapped code executes) is at **0x40800000**.
+This differs from newer ESP32 variants (S3, C3, etc.) where IROM starts at
+0x400C0000 or 0x400D0000.
+
+### D-Bus Cannot Write to I-Bus Addresses
+
+The ESP32 has a split bus architecture:
+- **I-Bus** (0x40000000+): instruction fetch, ROM, IRAM, IROM
+- **D-Bus** (0x3FFxxxxx+): data access, DRAM, DROM
+
+A `memcpy((void*)0x40080000, ...)` uses D-bus to write to an I-bus address.
+The D-bus routes such writes through the flash cache controller, which requires
+MMU page table entries for the target address. Without valid entries, the write
+either goes nowhere or faults. **Direct IRAM loading via memcpy is impossible
+on ESP32 from software.**
+
+### The Solution: IROM Execution via Flash MMU
+
+Espressif's own bootloader ("CACHE_APP" segments) and all IDF applications
+run from IROM — the flash MMU maps flash pages to IROM virtual addresses.
+The CPU fetches instructions via I-cache → MMU → SPI flash controller.
+No memcpy needed.
+
+Our MCUboot uses `cache_flash_mmu_set()` to map U-Boot's flash pages to
+IROM0 at 0x40800000, then jumps to the entry point there.
+
+## The Cache Bus Mask Problem
+
+The ESP32 DPORT register `DPORT_PRO_CACHE_CTRL1_REG` (0x3FF00044) controls
+which cache buses are enabled. After `bootloader_reset_mmu()` (called during
+MCUboot init), only DROM0 bus (bit 0) is unmasked. The IROM0 bus (bit 2)
+remains **masked** — meaning instruction fetches from IROM never reach the
+flash cache.
+
+The fix: clear bits 2 and 3 (IROM0 and IROM1 masks) before re-enabling cache.
+
+## Image Format
+
+U-Boot is wrapped with an `esp_image_load_header_t` (magic 0xACE637D3) that
+MCUboot's Espressif port understands:
+
+```
+[ MCUboot header 32B ]  magic=0x96f3b83d
+[ Load header 96B ]     magic=0xACE637D3  entry=0x40801854  irom_map=0x40800000
+[ Padding to 64KB ]     ESP32 MMU requires 64KB-aligned flash addresses
+[ IROM data ]           U-Boot .text + .ResetVector.text
+[ DRAM data ]           U-Boot .rodata + .data
+[ MCUboot TLV ]         SHA256 signature
+```
+
+The `mk_esp_mcuboot_image.py` tool wraps U-Boot's ELF into this format.
+
+## Files Changed
+
+### MCUboot (`portugalfuturista/mcuboot`)
+
+| File | Change |
+|------|--------|
+| `boot/espressif/port/esp_loader.c` | IROM flash-MMU mapping, cache bus unmask, DRAM offset fix |
+| `boot/espressif/CMakeLists.txt` | Relaxed IDF HAL version check (6.0.0 → 6.1.0) |
+| `boot/espressif/port/esp32/bootloader.conf` | Added CONFIG_ESPRESSIF_BOOTLOADER_MCUBOOT |
+
+### U-Boot (`portugalfuturista/universalis-uboot`)
+
+| File | Change |
+|------|--------|
+| `arch/xtensa/include/asm/arch-esp32/core.h` | Rewritten: IROM base 0x40800000, VECBASE, vectors |
+| `board/espressif/esp32/u-boot.lds` | Rewritten: IROM execution, vectors discarded |
+| `include/configs/esp32.h` | XTENSA_SYS_TEXT_ADDR → 0x40800400 |
+| `arch/xtensa/cpu/start.S` | Moved `_start` to `.text`, stack in DRAM |
+| `tools/mk_esp_mcuboot_image.py` | **New**: ELF→MCUboot image wrapper with 64KB alignment |
+| `board/espressif/esp32/Kconfig` | Added CONFIG_2ND_STAGE_BOOT symbol |
+| `arch/xtensa/config.mk` | Restored to original |
+
+## Flash Layout (4MB ESP32)
+
+| Offset | Size | Content |
+|--------|------|---------|
+| 0x1000 | ~30KB | MCUboot bootloader |
+| 0x20000 | ~213KB | U-Boot (primary slot) |
+| 0x170000 | 1.5MB | Secondary slot (OTA) |
+
+## Verified Boot Flow
+
+```
+ROM (mask ROM @ 0x40000000)
+ │ Loads MCUboot segments from flash @ 0x1000, jumps to entry
+ │
+ └─ MCUboot
+    │ bootloader_init() → reset MMU, init SPI flash
+    │ main() → do_boot()
+    │   esp_app_image_load():
+    │     1. Load DRAM via bootloader_mmap + memcpy → 0x3FFAF000
+    │     2. Cache_Read_Disable(0) + Cache_Flush(0)
+    │     3. cache_flash_mmu_set(0, 0, 0x40800000, 0x30000, 64KB, 2)
+    │     4. DPORT unmask IROM0/IROM1 buses
+    │     5. Cache_Read_Enable(0) + isync
+    │     6. Jump to 0x40801854 (_start)
+    │
+    └─ U-Boot 2025.04
+       │ _start: 'X' → UART, VECBASE → 0x40800000, DRAM zero, reloc
+       │ board_init_f → board_init_r → U-Boot prompt
+       │
+       └─ U-Boot prompt: loads UniversalisOS
+```
+
+## Reference
+
+- `docs/esp32-boot-chain-comparison.md` — Full IDF vs our chain comparison
+- `docs/esp32-mcuboot-uboot-strategy.md` — Original strategy document
+- `esp-hal-3rdparty-rel-master.c/components/soc/esp32/include/soc/ext_mem_defs.h` — Official memory map