From 3961c6c42a64d7cd76e157f7da08745bf9856169 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?F=C3=A1bio=20Coutada?= Date: Thu, 16 Jul 2026 16:44:25 +0100 Subject: [PATCH] docs: ESP32 boot chain comparison and bring-up report - esp32-boot-chain-comparison.md: IDF vs MCUboot->U-Boot comparison covering memory regions, cache/MMU sequences, image formats, and full feature matrix. - esp32-mcuboot-uboot-bringup-report.md: complete bring-up log with root cause analysis (IROM at 0x40800000, I-bus/D-bus split, cache bus masks, 64KB alignment requirement). --- docs/esp32-boot-chain-comparison.md | 286 +++++++++++++++++++++ docs/esp32-mcuboot-uboot-bringup-report.md | 138 ++++++++++ 2 files changed, 424 insertions(+) create mode 100644 docs/esp32-boot-chain-comparison.md create mode 100644 docs/esp32-mcuboot-uboot-bringup-report.md diff --git a/docs/esp32-boot-chain-comparison.md b/docs/esp32-boot-chain-comparison.md new file mode 100644 index 000000000..f2aac53c0 --- /dev/null +++ b/docs/esp32-boot-chain-comparison.md @@ -0,0 +1,286 @@ +# ESP32 Boot Chain: ESP-IDF vs UniversalisOS (MCUboot→U-Boot→UOS) + +## 1. Complete Boot Sequence Comparison + +### 1.1 ESP-IDF Flow + +``` +ROM (1st stage, mask ROM @ 0x40000000) + │ - Reads ESP image header (magic 0xE9) from flash @ 0x1000 + │ - Parses segment headers, loads segments per Memory Types: + │ · BYTE_ACCESSIBLE/DRAM → memcpy to DRAM + │ · CACHE_APP → flash-MMU maps flash pages to virtual address + │ · IRAM → memcpy to IRAM + │ - Sets up SPI flash, initial cache/MMU config + │ - Jumps to entry point (bootloader) + │ + └─ 2nd Stage Bootloader (IDF bootloader @ flash 0x1000) + │ bootloader_init(): + │ - bootloader_init_mem() → clear BSS, check stack + │ - bootloader_clock_configure() + │ - bootloader_console_init() + │ - bootloader_reset_mmu(): + │ Cache_Read_Disable(0) + │ Cache_Flush(0) + │ mmu_init(0) ← HARD reset of MMU controller + │ DPORT unmask DROM0 + │ - bootloader_flash_update_id() + │ - bootloader_init_spi_flash() → SPI flash init + │ - bootloader_enable_random() + │ + │ bootloader_utility_load_image(): + │ - Parse app partition, read ESP image header + │ - For each segment: memcpy IRAM/DRAM segments to RAM + │ - Record DROM/IROM segment addrs for MMU mapping + │ - load_image() → unpack_load_app() + │ + │ set_cache_and_start_app(): + │ Cache_Read_Disable(0) ← disable cache + │ Cache_Flush(0) ← flush D-cache to SRAM + │ mmu_hal_unmap_all() ← SOFT reset: clear page table only + │ cache_flash_mmu_set(0, DROM) ← map DROM flash pages + │ cache_flash_mmu_set(1, DROM) ← map DROM for APP CPU too + │ cache_flash_mmu_set(0, IROM) ← map IROM flash pages + │ cache_flash_mmu_set(1, IROM) ← map IROM for APP CPU too + │ cache_ll_l1_get_bus() / enable ← enable cache bus masks + │ Cache_Read_Enable(0) ← re-enable cache + │ isync + │ bootloader_atexit() ← clean up (WDT, uart flush) + │ (*entry)() ← jump to app IROM entry + │ + └─ Application (FreeRTOS app) + - Runs from IROM (flash-cached @ 0x400D0000+) + - data/bss in DRAM +``` + +### 1.2 UniversalisOS Flow (our chain) + +``` +ROM (1st stage, mask ROM) + │ - Reads ESP image header from flash @ 0x1000 + │ - Loads MCUboot segments (CACHE_APP + DRAM + IRAM) + │ - Jumps to MCUboot entry + │ + └─ MCUboot (2nd stage bootloader @ flash 0x1000) + │ bootloader_init(): [same ESP-IDF HAL, no major diffs] + │ - bootloader_reset_mmu() (via bootloader_init → init_ext_mem) + │ + │ main() → do_boot(): + │ - Validate image, check swap type + │ - start_cpu0_image(): + │ esp_app_image_load(): + │ - Parse MCUboot's esp_image_load_header_t (magic 0xACE637D3) + │ - Load DRAM via memcpy (→ 0x3FFAF000+) + │ - **IROM path**: cache_flash_mmu_set to map flash→IROM + │ - **IRAM path**: memcpy (BROKEN on ESP32 — D-bus can't write I-bus) + │ jump to entry_addr + │ + └─ U-Boot (loaded by MCUboot @ IROM 0x400D0000+) + │ _start: + │ - Print 'X', set VECBASE, unpack reloc, init UART + │ - board_init_f → board_init_r → U-Boot prompt + │ + └─ UniversalisOS (loaded by U-Boot) +``` + +### 1.3 Critical Difference: IRAM vs IROM + +| Aspect | ESP-IDF | UniversalisOS | +|--------|---------|---------------| +| Bootloader code location | IRAM (0x40078000) + CACHE_APP segment | IRAM (0x40078000 loader) + IRAM (0x40090000 main) | +| App code location | **IROM** (flash-cached, 0x400D0000+) | Was **IRAM** (SRAM, 0x40080000) — **BROKEN** | +| App code loading | flash-MMU maps flash pages to virtual IROM | Was memcpy to SRAM — **ESP32 D-bus can't write I-bus** | +| Handoff sequence | Disable cache → unmap_all → map DROM → map IROM → bus enable → Enable cache → isync → jump | Same pattern (current code) | +| MMU reset | `mmu_hal_unmap_all()` (soft, entries only) | Was `mmu_init(0)` (hard, corrupts controller) — **fixed** | + +## 2. Memory Region Comparison + +### 2.1 ESP32 Physical Memory Map + +``` +Bus Address Size Content +──── ──────── ──── ─────── + 0x4000_0000-0x4005_FFFF 384KB ROM (mask ROM, read-only) +I-bus 0x4007_0000-0x4007_7FFF 32KB SRAM0 — ROM cache (reserved) +I-bus 0x4007_8000-0x4007_A800 10KB MCUboot LOADER (CACHE_APP segment) +I-bus 0x4007_A800-0x4008_FFFF 86KB GAP — free +I-bus 0x4008_0000-0x4008_003F — UserExceptionVector (Xtensa hw) +I-bus 0x4008_0100-0x4008_01xx — KernelExceptionVector +I-bus 0x4008_03C0-0x4008_xxxx — DoubleExceptionVector +I-bus 0x4009_0000-0x4009_E800 58KB MCUboot IRAM MAIN (RECLAIMABLE by app) +I-bus 0x400A_0000-0x400B_FFFF 128KB SRAM1 (free, dual-bus) +I-bus 0x400C_0000-0x40FF_FFFF 4MB IROM — flash-cached execution +D-bus 0x3F40_0000-0x3F80_0000 4MB DROM — flash-cached data +D-bus 0x3FFA_E000-0x3FFC_FFFF 200KB SRAM2 — DRAM (data/bss/stack) +D-bus 0x3FFE_0000-0x3FFF_FFFF 128KB SRAM1 — DRAM (dual-bus mirror) +``` + +### 2.2 IDF App Memory Layout + +``` +Code: 0x400D_0000+ (IROM, flash-cached via MMU) +RO: 0x3F40_0000+ (DROM, flash-cached via MMU) +Data: 0x3FFB_0000+ (DRAM, SRAM2/SRAM1) +BSS: 0x3FFx_xxxx (DRAM, after data) +Stack: 0x3FFF_xxxx (DRAM, top of SRAM1) +``` + +### 2.3 U-Boot Memory Layout (new IROM design) + +``` +Code: 0x400D_0000+ (IROM, flash-cached — same as IDF app) +RO: 0x3FFB_0000+ (DRAM, same as IDF) +Data: 0x3FFB_xxxx (DRAM, after RO) +BSS: 0x3FFB_xxxx (DRAM, after data) +Stack: 0x4000_0000- (DRAM, top of address space) +``` + +### 2.4 The IRAM Problem (why memcpy never worked) + +ESP32 has **split bus architecture**: +- I-bus (instruction bus): fetches code from 0x40000000+ +- D-bus (data bus): reads/writes data from 0x3FFxxxxx+ + +A `memcpy((void*)0x40080000, ...)` uses D-bus to write to I-bus address. +The D-bus does NOT route to SRAM0 at that address. The write goes through +the flash cache controller, which requires MMU entries for the target address. +If no MMU entry exists, the write either goes nowhere or faults. + +**The only safe way to execute code on ESP32 is from IROM (flash-cached).** +Direct IRAM loading via memcpy is architecturally impossible from software. + +## 3. Cache / MMU Handling Comparison + +### 3.1 ROM Initial State + +After ROM boots the 2nd stage: +- VECBASE = 0x4000_0000 (ROM exception vectors) +- DROM0 bus: unmasked (ROM clears mask) +- IROM0: mapped by ROM to the bootloader's CACHE_APP segments +- D-cache: enabled, contains bootloader code cache lines +- I-cache: enabled, contains bootloader IROM cache lines +- MMU table: populated with bootloader's flash page mappings + +### 3.2 IDF Bootloader Init + +`bootloader_reset_mmu()` (bootloader_esp32.c:43-69): +``` +Cache_Read_Disable(0); // stop cache reads +Cache_Read_Disable(1); // APP CPU too +Cache_Flush(0); // write back D-cache to SRAM +Cache_Flush(1); +mmu_init(0); // HARD RESET: clear ALL MMU entries + controller state +mmu_init(1); // APP CPU too +DPORT unmask DROM0; // restore DROM0 bus so flash reads work +``` +After this, the bootloader must re-initialize SPI flash and re-read +its own header from flash. ALL previous MMU mappings are destroyed. + +### 3.3 IDF App Load + +`set_cache_and_start_app()` (bootloader_utility.c:1036-1158): +``` +Cache_Read_Disable(0); // stop cache reads +Cache_Flush(0); // write back D-cache +mmu_hal_unmap_all(); // SOFT RESET: clear page table, keep controller state +cache_flash_mmu_set(0, DROM); // map app's data from flash +cache_flash_mmu_set(1, DROM); // for APP CPU +cache_flash_mmu_set(0, IROM); // map app's code from flash +cache_flash_mmu_set(1, IROM); // for APP CPU +cache_ll_l1_enable_bus(); // enable bus masks +Cache_Read_Enable(0); // re-enable cache +isync; // flush prefetch pipeline +bootloader_atexit(); // clean up WDT, flush UART +(*entry)(); // JUMP +``` + +Key: uses `mmu_hal_unmap_all()` NOT `mmu_init()`. The soft reset only +clears the page table entries, leaving the MMU controller configured. +`mmu_init()` hard-resets the controller which requires full re-init. + +### 3.4 Our MCUboot Flow + +``` +Cache_Read_Disable(0); // ✓ +Cache_Flush(0); // ✓ +// (no mmu_hal_unmap_all — we don't have this HAL function) +cache_flash_mmu_set(0, IROM); // ✓ rc=0 +Cache_Read_Enable(0); // ✓ +isync; // ✓ +(*entry)(); // JUMP +``` + +**Missing vs IDF:** +1. No `mmu_hal_unmap_all()` — but we don't need it since we're adding + entries on top of MCUboot's existing mappings +2. No DROM mapping for U-Boot — U-Boot's RO data is in DRAM, not DROM +3. No bus mask enable — MCUboot's existing bus masks should still be active +4. No APP CPU mapping — ESP32 single-core during boot + +## 4. Image Formats + +### 4.1 ESP Image Format (ROM loader) +``` +[ESP image header 24B] magic=0xE9, segment_count, entry_addr +[Segment header 8B] × N load_addr, data_len +[Segment data] raw bytes +[Checksum 1B] +[SHA256 32B] (optional) +``` +Segment types determined by esptool from ELF segment VMA: +- 0x3FFxxxxx → BYTE_ACCESSIBLE, DRAM +- 0x4007xxxx → CACHE_APP (flash-mapped, no memcpy) +- 0x4008xxxx → IRAM (memcpy to SRAM) +- 0x400Dxxxx → CACHE_APP (flash-mapped, no memcpy) + +### 4.2 MCUboot Image Format (our chain) +``` +[MCUboot header 32B] magic=0x96f3b83d, image size, version +[Load header 96B] magic=0xACE637D3, entry_addr, iram_*, irom_*, dram_* +[IRAM data] code segment (legacy, memcpy) +[DRAM data] data segment +[MCUboot TLV] hash, signatures +``` +Our wrapper uses: +- irom_map_addr, irom_flash_offset, irom_size → flash-MMU mapped +- dram_dest_addr, dram_flash_offset, dram_size → memcpy to DRAM + +## 5. Feature Comparison Matrix + +| Feature | ESP-IDF Bootloader | MCUboot (our chain) | U-Boot (our chain) | +|---------|-------------------|---------------------|-------------------| +| **ROM loading** | ESP image (0xE9) | ESP image (0xE9) | N/A (loaded by MCUboot) | +| **Code execution** | IROM (flash-cached) | IRAM (SRAM) | IROM (flash-cached) | +| **Data storage** | DRAM (memcpy) | DRAM (memcpy) | DRAM (memcpy) | +| **Flash MMU** | `mmu_hal_unmap_all` + `cache_flash_mmu_set` × 4 | `cache_flash_mmu_set` × 1 | N/A (runs from MCUboot's MMU) | +| **Cache handling** | Disable→Flush→Unmap→Map→Enable→isync | Disable→Flush→Map→Enable→isync | Cache already enabled by MCUboot | +| **VECBASE** | App sets it (IRAM or IROM) | Set to 0x40080000 (old IRAM) | Sets VECBASE to IROM 0x400D0000 | +| **Exception vectors** | App provides, at VECBASE offsets | None (ROM vectors) | Built-in, at IROM offsets | +| **Secure boot** | Yes (RSA/ECDSA) | Yes (MCUboot signature validation) | N/A (trusts MCUboot) | +| **OTA updates** | Yes (factory/OTA partitions) | Yes (primary/secondary slots) | N/A (loaded by MCUboot) | +| **Flash encryption** | Yes (AES-XTS) | Yes (MCUboot supports) | N/A | +| **Image validation** | SHA256 checksum | SHA256 via imgtool TLV | N/A | +| **Multi-image** | No (single app) | Yes (multiple slots) | N/A | +| **Stack location** | DRAM (top of SRAM1) | DRAM (MCUboot's loader segment) | DRAM (0x40000000 - 16) | +| **Boot time** | ~50ms | ~100ms (MCUboot validation) | ~200ms (U-Boot init) | + +## 6. Root Cause Summary + +The fundamental issue was: **ESP32 D-bus cannot write to I-bus addresses.** + +Our original approach loaded U-Boot code to IRAM (0x40080000+) via memcpy, +which uses D-bus stores. ESP32's bus architecture routes D-bus writes to the +flash cache controller, not directly to SRAM0. The MMU must have valid entries +for the target address to route writes to physical SRAM. + +The fix: **Run U-Boot from IROM (flash-cached at 0x400D0000+).** This matches +how ESP-IDF runs applications. The flash MMU maps flash pages to IROM virtual +addresses. The CPU fetches instructions via I-cache → MMU → flash SPI controller. +No memcpy needed — the code executes directly from flash through the cache. + +This required: +1. U-Boot linker script migrated from IRAM (0x40080000) to IROM (0x400D0000) +2. MCUboot `esp_loader.c` updated to map IROM via `cache_flash_mmu_set` +3. Wrapper tool updated to emit IROM-segment load headers +4. Flash offset aligned to 64KB (ESP32 MMU page size) +5. VECBASE moved from IRAM to IROM address space diff --git a/docs/esp32-mcuboot-uboot-bringup-report.md b/docs/esp32-mcuboot-uboot-bringup-report.md new file mode 100644 index 000000000..3e30db300 --- /dev/null +++ b/docs/esp32-mcuboot-uboot-bringup-report.md @@ -0,0 +1,138 @@ +# ESP32 MCUboot → U-Boot Boot Chain — Bring-Up Report + +**Date:** 2026-07-16 +**Target:** ESP32-D0WDQ6-V3 (Xtensa LX6) +**Status:** ✅ Working + +## Overview + +UniversalisOS boot chain on ESP32: ROM (1st stage) → MCUboot (2nd stage) → U-Boot. + +MCUboot validates and loads U-Boot from the primary flash slot. U-Boot runs +from IROM (flash-cached at 0x40800000) with data in DRAM at 0x3FFAF000. + +## The Fundamental Discovery + +### ESP32 IROM is at 0x40800000, not 0x400D0000 + +The ESP32 memory map (from `esp-hal-3rdparty/components/soc/esp32/include/soc/ext_mem_defs.h`): + +| Region | Bus | Address Range | Purpose | +|--------|-----|---------------|---------| +| IRAM0 | IBUS0 | 0x400D0000-0x40400000 | SRAM / configurable cache | +| IRAM1 | IBUS1 | 0x40400000-0x40800000 | SRAM / configurable cache | +| **IROM0** | **IBUS2** | **0x40800000-0x40C00000** | **Flash-cached execution (dedicated)** | +| DROM0 | DBUS0 | 0x3F400000-0x3F800000 | Flash-cached data | +| DRAM1 | DBUS1 | 0x3F800000-0x3FC00000 | PSRAM / configurable data cache | + +The ESP32's IROM (where flash-MMU-mapped code executes) is at **0x40800000**. +This differs from newer ESP32 variants (S3, C3, etc.) where IROM starts at +0x400C0000 or 0x400D0000. + +### D-Bus Cannot Write to I-Bus Addresses + +The ESP32 has a split bus architecture: +- **I-Bus** (0x40000000+): instruction fetch, ROM, IRAM, IROM +- **D-Bus** (0x3FFxxxxx+): data access, DRAM, DROM + +A `memcpy((void*)0x40080000, ...)` uses D-bus to write to an I-bus address. +The D-bus routes such writes through the flash cache controller, which requires +MMU page table entries for the target address. Without valid entries, the write +either goes nowhere or faults. **Direct IRAM loading via memcpy is impossible +on ESP32 from software.** + +### The Solution: IROM Execution via Flash MMU + +Espressif's own bootloader ("CACHE_APP" segments) and all IDF applications +run from IROM — the flash MMU maps flash pages to IROM virtual addresses. +The CPU fetches instructions via I-cache → MMU → SPI flash controller. +No memcpy needed. + +Our MCUboot uses `cache_flash_mmu_set()` to map U-Boot's flash pages to +IROM0 at 0x40800000, then jumps to the entry point there. + +## The Cache Bus Mask Problem + +The ESP32 DPORT register `DPORT_PRO_CACHE_CTRL1_REG` (0x3FF00044) controls +which cache buses are enabled. After `bootloader_reset_mmu()` (called during +MCUboot init), only DROM0 bus (bit 0) is unmasked. The IROM0 bus (bit 2) +remains **masked** — meaning instruction fetches from IROM never reach the +flash cache. + +The fix: clear bits 2 and 3 (IROM0 and IROM1 masks) before re-enabling cache. + +## Image Format + +U-Boot is wrapped with an `esp_image_load_header_t` (magic 0xACE637D3) that +MCUboot's Espressif port understands: + +``` +[ MCUboot header 32B ] magic=0x96f3b83d +[ Load header 96B ] magic=0xACE637D3 entry=0x40801854 irom_map=0x40800000 +[ Padding to 64KB ] ESP32 MMU requires 64KB-aligned flash addresses +[ IROM data ] U-Boot .text + .ResetVector.text +[ DRAM data ] U-Boot .rodata + .data +[ MCUboot TLV ] SHA256 signature +``` + +The `mk_esp_mcuboot_image.py` tool wraps U-Boot's ELF into this format. + +## Files Changed + +### MCUboot (`portugalfuturista/mcuboot`) + +| File | Change | +|------|--------| +| `boot/espressif/port/esp_loader.c` | IROM flash-MMU mapping, cache bus unmask, DRAM offset fix | +| `boot/espressif/CMakeLists.txt` | Relaxed IDF HAL version check (6.0.0 → 6.1.0) | +| `boot/espressif/port/esp32/bootloader.conf` | Added CONFIG_ESPRESSIF_BOOTLOADER_MCUBOOT | + +### U-Boot (`portugalfuturista/universalis-uboot`) + +| File | Change | +|------|--------| +| `arch/xtensa/include/asm/arch-esp32/core.h` | Rewritten: IROM base 0x40800000, VECBASE, vectors | +| `board/espressif/esp32/u-boot.lds` | Rewritten: IROM execution, vectors discarded | +| `include/configs/esp32.h` | XTENSA_SYS_TEXT_ADDR → 0x40800400 | +| `arch/xtensa/cpu/start.S` | Moved `_start` to `.text`, stack in DRAM | +| `tools/mk_esp_mcuboot_image.py` | **New**: ELF→MCUboot image wrapper with 64KB alignment | +| `board/espressif/esp32/Kconfig` | Added CONFIG_2ND_STAGE_BOOT symbol | +| `arch/xtensa/config.mk` | Restored to original | + +## Flash Layout (4MB ESP32) + +| Offset | Size | Content | +|--------|------|---------| +| 0x1000 | ~30KB | MCUboot bootloader | +| 0x20000 | ~213KB | U-Boot (primary slot) | +| 0x170000 | 1.5MB | Secondary slot (OTA) | + +## Verified Boot Flow + +``` +ROM (mask ROM @ 0x40000000) + │ Loads MCUboot segments from flash @ 0x1000, jumps to entry + │ + └─ MCUboot + │ bootloader_init() → reset MMU, init SPI flash + │ main() → do_boot() + │ esp_app_image_load(): + │ 1. Load DRAM via bootloader_mmap + memcpy → 0x3FFAF000 + │ 2. Cache_Read_Disable(0) + Cache_Flush(0) + │ 3. cache_flash_mmu_set(0, 0, 0x40800000, 0x30000, 64KB, 2) + │ 4. DPORT unmask IROM0/IROM1 buses + │ 5. Cache_Read_Enable(0) + isync + │ 6. Jump to 0x40801854 (_start) + │ + └─ U-Boot 2025.04 + │ _start: 'X' → UART, VECBASE → 0x40800000, DRAM zero, reloc + │ board_init_f → board_init_r → U-Boot prompt + │ + └─ U-Boot prompt: loads UniversalisOS +``` + +## Reference + +- `docs/esp32-boot-chain-comparison.md` — Full IDF vs our chain comparison +- `docs/esp32-mcuboot-uboot-strategy.md` — Original strategy document +- `esp-hal-3rdparty-rel-master.c/components/soc/esp32/include/soc/ext_mem_defs.h` — Official memory map