diff --git a/sessions/2026-07-10_replan-mycelium-universalisos/PARITY.md b/sessions/2026-07-10_replan-mycelium-universalisos/PARITY.md new file mode 100644 index 00000000..4503549d --- /dev/null +++ b/sessions/2026-07-10_replan-mycelium-universalisos/PARITY.md @@ -0,0 +1,230 @@ +# Universalisos ↔ PikeOS 5.0 / Bao — Parity Backlog (verified) + +> **Created**: 2026-07-10 (synthesis of in-repo analyses, **reconciled with verified ground truth**) +> **Author**: Kimi (root agent, `replica-omnisciente`) +> **Scope**: `/home/fabiorafaelcoutada/portugalfuturista/universalisos` +> **Supersedes**: the per-doc verdicts where they are stale or mutually contradictory (see §6). + +--- + +## 0. How to read this + +- Two reference targets, two different parity axes: + - **PikeOS 5.0** — the *RTOS/partition/API/safety* axis (ARINC 653 partitions, IPC, HM, + personalities, syscalls, system-software layer). + - **Bao** — the *ARMv8/RISC-V hardware-virtualization* axis (Stage-2 MMU, GICv3 VGIC, vCPU + world-switch, static partitioning). Bao is the right model for the **AArch64** track, **not** + the x86 VMX/EPT/VMCS vocabulary that some older analyses use. +- Status key: **✅ done (verified)** · **🟡 partial / framework** · **❌ missing** · **— not applicable**. +- Rows marked **(V)** are verified by build+QEMU this session; others are doc-derived and flagged. + +--- + +## 1. Verified ground-truth snapshot (2026-07-10) — supersedes stale docs + +| Track | Verified state | Supersedes | +|---|---|---| +| **ARMv7 / qemu-arm-virt** | Boots, full demo end-to-end (scheduler, ARINC sampling/queuing/shmem IPC, 3-tier HM, PV ABI, preempt A/B). **0 linker warnings.** BSS 20.5 MB → 3.6 MB. | `status_report` "MATURE" (still accurate) | +| **AArch64 / qemu-aarch64-virt** | **MP0 COMPLETE.** Builds (0 warnings), boots at EL2 (`virtualization=on`), LPAE S1, GICv3, CNTHP @100 Hz, PikeOS scheduler, preemptive ctx switch → interleaved A/B. | `status_report` "build broken / MP0 ~15%" (stale) | +| **RISC-V / polarfire** | Functional parallel track (own scheduler/VMM/hypercalls/Sv39). Not exercised this session. | `status_report` "FUNCTIONAL" (assumed current) | +| **Tooling (mycelium/CODEO)** | `mycelium` XSD codegen exists & compiles; one round-trip bug + documented gaps. | `VS_PIKEOS_5.0` "Tooling 0%" (stale) | + +> The older analyses (`UNIVERSALISOS_VS_PIKEOS_5.0.md`, `UNIVERSALISOS_PIKEOS_ANALYSIS.md`) use +> **x86 virtualization terms** (VT-x, VMX, EPT, VPID, VMCS) and pre-date mycelium + the AArch64 MP0 +> work. Treat their percentages as directional only; this matrix re-maps everything to ARM. + +--- + +## 2. Parity gap matrix + +### A. CPU / virtualization core (axis: **Bao** on AArch64; PikeOS VM layer on ARMv7) + +| Feature | PikeOS | Bao | Universalisos ARMv7 | Universalisos AArch64 | Effort | +|---|---|---|---|---|---| +| Boot + EL/priv bring-up | ✅ | ✅ | ✅ (V) | ✅ EL2 (V) | — | +| Stage-1 MMU (flat identity) | ✅ | ✅ | ✅ (V) | ✅ LPAE (V) | — | +| Exception/trap vectors | ✅ | ✅ | ✅ (V) | ✅ VBAR_EL2 (V) | — | +| Context switch (kernel tasks) | ✅ | ✅ | ✅ (V) | ✅ + preempt fix (V) | — | +| Preemptive scheduling (tick) | ✅ | ✅ | ✅ (V) | ✅ CNTHP @100 Hz (V) | — | +| **Stage-2 MMU (VTTBR_EL2 / IPA→PA)** | ✅ | ✅ | — (no S2 on v7) | ❌ | **L** | +| **vCPU struct + world switch (ERET to guest EL1)** | ✅ | ✅ | 🟡 (vm.cpp frame) | ❌ | **L** | +| **VGIC (GICv3 ICH_* virtual interrupts)** | ✅ | ✅ | — (GICv2) | ❌ | **M** | +| **Virtual timer to guest (CNTVOFF_EL2)** | ✅ | ✅ | — | ❌ | **S** | +| **HCR_EL2 trap config (sysreg/IO/WFI trapping)** | ✅ | ✅ | — | 🟡 (IMO/FMO/AMO only) (V) | **M** | +| Hypercall ABI (guest→hyp) | ✅ | ✅ | ✅ PV ABI (V) | ❌ | **M** | +| SMP (real multi-CPU scheduler) | ✅ | ✅ | 🟡 framework | ❌ | **L** | + +### B. Partitioning, scheduling, IPC (axis: **PikeOS / ARINC 653**) + +| Feature | PikeOS | Bao | ARMv7 | AArch64 | Effort | +|---|---|---|---|---|---| +| Static partitions + windows | ✅ | ✅ | ✅ (V) | 🟡 schema only (V) | **M** | +| ARINC 653 sampling ports | ✅ | — | ✅ (V) | ❌ | **M** | +| ARINC 653 queuing ports | ✅ | — | ✅ (V) | ❌ | **M** | +| Shared memory + events | ✅ | 🟡 | ✅ (V) | ❌ | **M** | +| Time-partition scheduler | ✅ | ✅ | ✅ (V) | ✅ (V) | — | +| Rate/Deadline-monotonic, PIP | ✅ | — | ❌ | ❌ | **M** | +| Deadline-miss detection | ✅ | — | 🟡 (HM) | ❌ | **S** | +| Capability-based access (MDB) | 🟡 | — | ✅ seL4-style (V) | ❌ | **M** | + +### C. Health, safety, system software (axis: **PikeOS**) + +| Feature | PikeOS | Bao | ARMv7 | AArch64 | Effort | +|---|---|---|---|---|---| +| 3-tier Health Monitoring | ✅ | 🟡 | ✅ (V) | ❌ | **M** | +| ASIL classification / fault containment | ✅ | — | 🟡 framework | ❌ | **L** | +| MISRA C++ / ISO 26262 / DAL | ✅ | — | ❌ | ❌ | **XL** (cert) | +| Personalities (APEX/POSIX/Linux/Ada) | ✅ | — | ❌ | ❌ | **XL** | +| Network stack (lwIP/TCP-IP) | ✅ | — | ❌ (driver only) | ❌ | **L** | +| Certified filesystem (FAT/JFFS2) | ✅ | — | ❌ | ❌ | **L** | +| Real Linux guest boot | ✅ | ✅ | ❌ (PV ABI only) | ❌ | **L** (needs MP1) | + +### D. Device & I/O (axis: **PikeOS + Bao**) + +| Feature | PikeOS | Bao | ARMv7 | AArch64 | Effort | +|---|---|---|---|---|---| +| UART / timer / GIC drivers | ✅ | ✅ | ✅ (V) | ✅ (V) | — | +| Block / net / GPIO / I2C / SPI / USB / PCI | ✅ | 🟡 | ✅ present (V) | ❌ | **L** (port) | +| Virtio emulation (blk/net/console) | 🟡 | ✅ | 🟡 (states exist) | ❌ | **M** | +| Device passthrough + IOMMU (SMMU) | ✅ | ✅ | ❌ | ❌ | **XL** | +| MMIO trap-emulate for guests | ✅ | ✅ | — | ❌ | **M** (needs S2) | + +> **Driver caveat:** `UNIVERSALISOS_PIKEOS_ANALYSIS.md` claims "drivers 95–100% stubs", while +> `status_report.md` marks ARMv7 drivers "✅ Complete". The ARMv7 demo does exercise block/network +> (e1000/virtio/rtl8139 BSS states exist; block demo reports 0 errors). Treat drivers as +> **present-but-largely-emulated** on QEMU, not hardware-validated. (V) for "present", not "proven on HW". + +### E. Tooling & config (axis: **PikeOS CODEO**) + +| Feature | PikeOS | Universalisos | Effort | +|---|---|---|---| +| XSD → C/binary codegen (`pikeos-configconv`) | ✅ | 🟡 `mycelium` (V) — works, 1 round-trip bug + gaps | **S–M** | +| VMIT XML config → kernel | ✅ | ✅ `cfg_parser.cpp` (V) | — | +| IDE / Theia (CODEO) integration | ✅ | 🟡 scaffold (mycelium `ide/`) | **M** | +| Host test harness (`configmore --test`) | ✅ | 🟡 `mycelium --test` (WIP) | **S** | + +--- + +## 3. Prioritized, ordered backlog + +> Order principle: **foundation before features**, **ARMv7 is the reference of record** (don't regress it), +> **AArch64 is the growth track** (virtualization lives here), **RISC-V stays the parallel HW track**. +> Each item: `ID — title — [MPx] — effort — verify`. + +### Horizon H0 — done / keep green (this session) +- **DONE-1** ARMv7 linker `ENTRY` + RX/RW PHDRS → 0 warnings (V). Keep as the clean baseline. +- **DONE-2** BSS 20.5 MB → 3.6 MB (guest image 16→1 MiB, ramdisk 256→128 KiB) (V). +- **DONE-3** AArch64 MP0: EL2 boot (`virtualization=on`), LPAE S1, GICv3, CNTHP timer, preemptive + scheduler → interleaved A/B, 0 traps (V). **This is the MP0 gate; it now passes.** + +### Horizon H1 — virtualization foundation (the PikeOS/Bao core gap) ← **next track** +- **MP1.1 — Stage-2 MMU (AArch64)** — `[MP1]` — **L** + Build `VTTBR_EL2` LPAE Stage-2 tables (IPA→PA), `VTCR_EL2`, `HCR_EL2.VM=1`, per-vCPU `VPIDR_EL2`/`VMPIDR_EL2`. + *Verify*: identity-mapped guest IPA region; a Stage-2 fault routes to EL2 with `ESR.EC=0x20/0x24`. +- **MP1.2 — vCPU struct + world switch** — `[MP1]` — **L** + `vCPU` {GPRs, ELR_EL2, SPSR_EL2, SP_EL1, SCTLR_EL1, TTBR0_EL1, VBAR_EL1, …}; `vcpu_run()` saves hyp, loads guest, `ERET`; exit traps back to EL2. Reuse `context_switch.S` frame shape. + *Verify*: enter a tiny guest at EL1, execute a few instructions, trap on `hvc`/WFI, return to EL2. +- **MP1.3 — VGIC (GICv3 list registers)** — `[MP1]` — **M** + Program `ICH_*_EL2` (HCR/VSEIR/LR/VMCR), inject a vIRQ to the guest, EOI/maintenance IRQ path. + *Verify*: guest receives a virtual PPI and EOIs it without a raw trap storm. +- **MP1.4 — Virtual timer to guest** — `[MP1]` — **S** + `CNTVOFF_EL2`, `CNTV_CVAL_EL0`/`CNTV_CTL_EL0` routing; guest sees a vtimer tick. + *Verify*: guest EL1 reads `CNTVCT` and takes a vtimer interrupt. +- **MP1.5 — Trap config matrix** — `[MP1]` — **M** + Decide/encode `HCR_EL2.{TWI,TWE,TSC,TVM,…}` and CPACR/TIDCP traps; document the policy. + *Verify*: table-driven test of each trap class. + +### Horizon H2 — core PikeOS parity (RTOS axis) +- **CORE-1 — Unify AArch64 with `src/core/` IPC + HM + cap** — `[MP1/MP3]` — **M** + Port the ARMv7 core subsystems (sampling/queuing/shmem IPC, 3-tier HM, cap MDB) to AArch64 so both ARM + tracks share one core (resolves open Q1: shared `src/core/` over standalone). + *Verify*: ARMv7 IPC/HM demo runs unchanged on AArch64. +- **CORE-2 — Real SMP** — `[MP5]` — **L** + Secondary CPU bring-up, IPI/SGI, per-CPU runqueues, TLB shootdown. ARMv7 first (core exists), then AArch64. + *Verify*: two tasks pinned to two CPUs both preempt via tick. +- **CORE-3 — ARINC 653 scheduling hardening** — `[—]` — **M** + RMS/DMS, priority inheritance, deadline-miss → HM action, temporal-isolation guarantees. + *Verify*: injected overrun triggers HM; windows enforced under load. +- **CORE-4 — PikeOS syscall surface** — `[—]` — **M** + Extend beyond the present task/TLS/ExRegs set toward partition/thread/memory/sync APIs (libpikeos-shape). + *Verify*: a conformance test list (name→implemented?) tracked in-tree. + +### Horizon H3 — system-software & platform reach (the "5–10%" layer) +- **SYS-1 — Linux guest boot** — `[MP3]` — **L** *(blocked on MP1)* + Load a real kernel + DTB + initramfs via Stage-2, boot to a shell under the hypervisor. + *Verify*: Linux boots as a guest vCPU on AArch64. +- **SYS-2 — Personalities (APEX first, then POSIX)** — `[—]` — **XL** + APEX ARINC 653 process API surface over the existing IPC; POSIX later. + *Verify*: APEX sample app links & runs in a partition. +- **SYS-3 — Network stack (lwIP)** — `[—]` — **L** + lwIP as a partition/personality; virtio-net or e1000 backend. + *Verify*: ping/echo from a guest/partition. +- **SYS-4 — Filesystem (FAT/JFFS2)** — `[—]` — **L** + Block-backed FS; certified subset documented. + *Verify*: mount/read/write in a partition. +- **PLAT-1 — Hardware ports** — `[MP2]` — **L** + i.MX 8M Plus (AArch64), STM32MP257 (AArch64/CA35), i.MX6 (ARMv7) platform dirs + BSP. + *Verify*: boots on real HW (or documented bring-up checklist). +- **SAFE-1 — MISRA / fault containment / ASIL enforcement** — `[—]` — **XL** *(certification horizon)* + Coding-standard gate, WCET, audit trails. Long-horizon; track, don't front-load. + +### Horizon H4 — tooling (mycelium/CODEO) — Track A (separate agent; **non-blocking**) +- **MYC-1 — Fix `header.rs` emitter** — `[M3]` — **S** (highest-leverage mycelium fix) + Restore child field names + array bounds; green the `offline_test_passes` round-trip. +- **MYC-2 — Finish `xs:extension`** — `[M3]` — **S** (wire `extension_attributes` through emitter). +- **MYC-3 — Clean 2 warnings** — `[M3]` — **S** (unused import, unread field). +- **MYC-4 — Theia/CODEO view + `configmore --test`** — `[—]` — **M**. + +--- + +## 4. Recommended next execution track: **MP1 (AArch64 hardware virtualization)** + +**Why now:** H0 is green on both ARM tracks; the single biggest PikeOS/Bao parity gap is *hardware +virtualization*, and it only exists on AArch64. Everything in H2 (guest boot, real partitions with +isolation, MMIO trap-emulate) is gated by MP1. Starting here unblocks the most downstream work and +matches the in-repo MP0→MP5 plan. + +**First slice (de-risked, observable):** Stage-2 identity map + a *single-instruction guest*. +1. Build Stage-2 L0→L1 (1 GiB block) tables mapping IPA==PA for the QEMU DRAM window; set `VTCR_EL2`, + `VTTBR_EL2`, `HCR_EL2.VM=1`, `TGE=0`, `RW=1`. +2. Allocate one `vCPU`, seed `ELR_EL2=guest_entry`, `SPSR_EL2=EL1h`, `SP_EL1`, minimal `SCTLR_EL1`. +3. Guest entry = a 3-instruction blob: `mov x0,#0xAB; hvc #0; b .` (the `hvc` traps to EL2). +4. `vcpu_run()` → `ERET` into EL1 → trap on `hvc` → EL2 handler prints `x0==0xAB` → park. +*Verify criterion:* one clean EL1 entry and one `hvc` round-trip to EL2 with the guest GPR intact. +No VGIC, no timer, no MMIO yet — those are MP1.3/1.4 once the world-switch is proven. + +**Guardrails (from this session's scars):** always boot QEMU with `virtualization=on`; add an early +`CurrentEL` print; for the world-switch debug, temporarily force the non-reschedule path to bisect +"timer vs switch" exactly as we did for MP0. + +--- + +## 5. Mycelium tie-in (Track A, parallel — does not block MP1) + +Mycelium already **builds** (`cargo check` clean); its real blocker is the `header.rs` emitter +(field-name loss + array-bound misplacement → `gcc` rejects the generated header), codified by a unit +test. It is the *PikeOS CODEO `pikeos-configconv`* parity axis (Tooling, §E) — valuable but independent of +the hypervisor virtualization track. Run it on the **mycelium agent** (it never edits universalisos). + +--- + +## 6. Caveats on the source analyses (why this matrix supersedes them) + +- **Stale**: `UNIVERSALISOS_VS_PIKEOS_5.0.md` & `UNIVERSALISOS_PIKEOS_ANALYSIS.md` pre-date mycelium and + the AArch64 MP0 work; e.g. "Tooling 0%", "AArch64 boot ✅" (was false until this session), "drivers + 95–100% stubs" vs `status_report` "drivers ✅ Complete". +- **x86-biased**: both use VT-x/VMX/EPT/VPID/VMCS vocabulary. Universalisos is an ARM/RISC-V hypervisor; + parity is measured against **Stage-2/VTTBR/GICv3-VGIC/vCPU** (Bao), re-mapped here. +- **Now-current**: `status_report.md`'s EXISTS/MISSING + MP0–MP5 scaffold is the closest to truth, but its + AArch64 row ("build broken / MP0 ~15%") is outdated by this session's MP0 pass. +- **Percentages** in all docs are directional. The matrix uses ✅/🟡/❌ + **(V)** verified flags instead. + +--- + +## 7. Sources + +- `universalisos/UNIVERSALISOS_VS_PIKEOS_5.0.md` — 11-category feature % (x86-biased, directional). +- `universalisos/UNIVERSALISOS_PIKEOS_ANALYSIS.md` — P1/P2/P3 critical gaps + 12/18-month roadmap. +- `universalisos/universalisos_status_report.md` — per-arch maturity, EXISTS/MISSING, MP0–MP5. +- `universalisos/RISCV_PIKEOS_PARITY_PLAN.md` — RISC-V S-mode paravirt map (parallel track). +- Verified this session (build + QEMU): ARMv7 demo, AArch64 MP0, BSS cut, linker hygiene. diff --git a/sessions/2026-07-10_replan-mycelium-universalisos/PLAN.md b/sessions/2026-07-10_replan-mycelium-universalisos/PLAN.md new file mode 100644 index 00000000..57fb8783 --- /dev/null +++ b/sessions/2026-07-10_replan-mycelium-universalisos/PLAN.md @@ -0,0 +1,411 @@ +# Replanned Sprint — Mycelium & Universalisos (evidence-based) + +> **Created**: 2026-07-10 (re-analysis) +> **Supersedes**: the antigravity multi-agent plan of the same date +> **Analyst**: Kimi (root agent, `replica-omnisciente`) +> **Repos** (siblings of this brain, **not** the in-tree `mycelium-rd/`): +> - `/home/fabiorafaelcoutada/portugalfuturista/mycelium` +> - `/home/fabiorafaelcoutada/portugalfuturista/universalisos` + +--- + +## 0. TL;DR — the old plan is built on stale premises + +The antigravity plan frames both projects as *early-construction* efforts with a few +compile errors. **That is not reality.** I built, tested, booted and `nm`-inspected +both trees today. The corrected picture: + +| Project | Old-plan assumption | Verified reality (2026-07-10) | +|---|---|---| +| **Mycelium** | "3 parser compile errors block `cargo check`; pipeline (lowering/emitters/CLI) not yet built" | `cargo check --workspace` **passes**. The whole pipeline (parser → IR lowering → C/asm/binary/CRC emitters → drop-in CLI) **already exists and compiles**, with 11 tests, golden VMIT fixtures and a round-trip harness. The only failing gate is **one test** (`offline_test_passes`): the **header emitter generates malformed C**. | +| **Universalisos ARMv7** | "fix `_start` warning; shrink `vm_page_tables[16][4096]` to fix 20 MB BSS" | `_start` warning is real but the cause is inverted (entry symbol is `reset_handler`, not `_start`); the 20 MB BSS is real but **does not come from `vm_page_tables` (256 KB)** — it comes from `g_guest_manager` (**16 MiB**) + `ramdisk_memory` (2 MiB). | +| **Universalisos AArch64** | "builds, healthy (17 KB text / 79 KB BSS); just remove the `wfi` isolation loop to reach MP0 A/B" | Build numbers are correct, **but `qemu-system-aarch64` produces zero UART output** — the kernel does not even print its banner. MP0 is gated by **boot/UART bring-up**, upstream of the scheduler handoff. | + +Net: less "build from scratch", more "fix specific, root-caused defects and finish in-progress work". + +--- + +## 1. Method & evidence + +Every claim below was reproduced, not inferred: + +```bash +# Mycelium +cargo check --workspace # exit 0 (2 warnings) +cargo test --workspace # 1 failure: offline_test_passes +# manual replay of the round-trip steps → captured real gcc diagnostics + +# Universalisos (from kernel/) +make ARCH=armv7 PLATFORM=qemu-arm-virt # exit 0, 2 ld warnings +make ARCH=aarch64 PLATFORM=qemu-aarch64-virt # exit 0, 1 ld warning +arm-none-eabi-nm --print-size --size-sort … # located the 20 MB BSS owner +timeout 6s qemu-system-aarch64 … -kernel … # NO output +``` + +Tooling present: `cargo`/`rustc`, `arm-none-eabi-gcc`, `aarch64-linux-gnu-{gcc,g++,ld,size,nm}`, +`qemu-system-arm`, `qemu-system-aarch64`. (Note: AArch64 uses the **linux-gnu** toolchain in +`-ffreestanding -nostdlib` mode — see `arch/aarch64/arch.mk:10`; there is no bare-metal +`aarch64-none-elf-` on this box.) + +--- + +## 2. Verified reality vs. the antigravity claims + +### Mycelium +- ❌ **WS-M1 (fix 3 parser compile errors)** — `cargo check` already passes. The line numbers + cited (L507–512, L711, L103 `Attribute`) do not correspond to current compile errors. The + file was edited since (WIP on `xs:extension`); the only parser-side diagnostic is a dead-field + warning (`extension_attributes`, parser.rs:111). +- ✅ The pipeline is implemented: + - `mycelium-model/src/lower.rs` (543 lines) — XSD→IR lowering exists. + - `mycelium-codegen/src/{header,binary,crc,test_inc}.rs` — emitters exist. + - `mycelium-cli/src/main.rs` — CLI is a **superset** of the old WS-M4 flag list + (`--enum/--struct`, `--enum-asm`, `--bin`, `--crc`, `--ifndef/--comment/--include`, + `--wordsize/--endian/--align-tail`, `--catalog`, `--xml`, `--out`, **plus `--test`**). +- ✅ **Real defect**: the round-trip test (`crates/mycelium-codegen/tests/runner_roundtrip.rs`) + drives the CLI then `gcc`-compiles a harness — **gcc rejects the generated header**. Root + cause is in `header.rs` struct-body emission (see §4 M0). + +### Universalisos +- ✅ AArch64 build is healthy: `text 16912 / data 0 / bss 79440`. Only an **RWX LOAD-segment** + warning (`/usr/bin/aarch64-linux-gnu-ld: aviso: … segmento LOAD com permissões RWX`). +- ✅ ARMv7 builds but with **two** `ld` warnings and **20,540,000 bytes of BSS**: + ``` + arm-none-eabi-ld: warning: cannot find entry symbol _start; defaulting to 40000000 + arm-none-eabi-ld: warning: … has a LOAD segment with RWX permissions + text data bss dec hex + 172288 179280 20540000 20891568 13ec7b0 + ``` +- 🔧 The `_start` warning is **misdiagnosed** in the old plan. `armv7/linker.ld:8` already + declares `ENTRY(_start)`, but `armv7/boot.S` defines the entry as `reset_handler` + (boot.S:54–55) — there is no `_start` symbol. **Fix is `ENTRY(reset_handler)` (or add a + `_start` alias), not "add ENTRY(_start)".** +- 🔧 The 20 MB BSS is **mis-aimed** in the old plan. `nm --size-sort` shows the owners: + ``` + 16777292 b g_guest_manager ← guest.cpp:15 (image_buffer[16 MiB]) + 2097152 b ramdisk_memory ← block.cpp:820 ([MAX_BLOCK_DEVICES][RAMDISK_MAX_BYTES]) + 262144 b g_uos_vm_pgdir_pool + 262144 b vm_page_tables ← only 256 KB, NOT the problem + 206976 b e1000_states + 173176 d g_gic_state + ... + ``` + `guest.h:42` sets `MAX_GUEST_IMAGE_SIZE = 16*1024*1024`, embedded as + `uint8_t image_buffer[…]` inside `guest_manager_t` (guest.h:230). Shrinking + `vm_page_tables` (old U1.2) saves ~240 KB of a 20 MB problem — **pointless**. +- ❌ **AArch64 MP0 is further away than stated.** A 6 s `qemu-system-aarch64` run printed + **nothing** — no banner, no `CurrentEL`, nothing. So the "remove the `wfi` loop and observe + interleaved A/B" task (old U2.1) is blocked by an **earlier** bring-up gap: the kernel does + not reach (or does not drive) the PL011. (The `wfi` isolation loop at + `kernel_aarch64.cpp:126–132` is confirmed present and the real `eret` path at L135–138 is + unreachable — that part of the old plan is accurate, just not the next blocker.) + +--- + +## 3. Open questions — answered with evidence + +- **Q1 (AArch64 IPC strategy).** Premature. AArch64 does not boot to a banner yet; IPC is two + milestones out. **Defer the decision.** When reached, prefer the RISC-V pattern (standalone, + arch-local) over refactoring `src/core/ipc/` — AGENTS.md is explicit that AArch64/RISC-V are + self-contained and ARMv7 owns `core/`; pulling core IPC into AArch64 risks cross-arch header + coupling for no near-term payoff. +- **Q2 (Mycelium XML data parsing).** **Already implemented.** The `--bin` and `--test` paths + read an XML instance (`--xml=` or positional) — `main.rs:112–153` — and the round-trip test + exercises `configtest.xml`. The gap is *not* parsing; it is **binary cross-reference / IDREF + tables** (documented in `CONFIGCONV_CONTRACT.md#open-gaps`). Treat Q2 as closed. +- **Q3 (QEMU availability).** Present: `qemu-system-aarch64` and `qemu-system-arm` are in + `/usr/bin`. No install needed. + +--- +## 4. Project 1 — Mycelium (re-scoped) + +**Goal unchanged**: drop-in `pikeos-configconv`. **Status**: ~90% done; one correctness bug +blocks the round-trip gate, plus documented open gaps and in-progress `xs:extension` work. + +### M0 — Fix the C-header emitter (the keystone) **[highest priority]** + +**File**: `crates/mycelium-codegen/src/header.rs` (struct-body loop, **L148–160**). + +Current (broken) emission drops the **field name** for every child and mis-places array bounds: + +```rust +// header.rs:148-160 (today) +for c in &rs.children { + if c.app_info.annotation.ignore { continue; } + if c.max > 1 && c.max < u32::MAX { + out.push_str(&format!(" {}[{}];\n", c.c_type, c.max)); // no name; T[N]; is ill-formed + } else { + out.push_str(&format!(" {};\n", c.c_type)); // no name at all + } +} +``` + +Captured `gcc` diagnostics prove this exactly: +``` +configtest.h:160: error: expected identifier … before '[' → struct teebag_t[10]; +configtest.h:173: error: expected specifier-qualifier-list → vehicle_t; +configtest.h:202: error: … before '[' → char[32] str; +configtest.h:197: error: unknown type name 'uw_t' → uw_t uw; +configtest.h:200/201: unknown type name 'stdIPv4_t'/'stdMAC_t' +``` + +Three concrete fixes: +1. **Emit the child field name**: `T name;` and `T name[N];` (use the child's resolved name; the + `ResolvedChild`/field carries it — confirm field name in `lower.rs`). +2. **Inline strings**: the IR stores `c_type = "char[32]"`, so the attribute path + (`header.rs:146`, `{c_type} {fname}`) yields `char[32] str`. Split into base + bound and emit + `char str[32];` (either carry `{base, size}` in the IR or parse the trailing `[N]` in the + emitter). Prefer carrying structured size from lowering. +3. **Missing typedefs** (`uw_t`, `stdIPv4_t`, `stdMAC_t`, `vehicle_t`): named simple types are + used as field `c_type` but never declared. In the first pass (today only enums are emitted), + also emit `typedef _t;` for non-enum, non-`cx:external` simple types that lower + to a named alias; honour `cx:external` by *assuming* the typedef is provided by the + `--include`d header and emitting nothing. + +> ⚠️ The unit test `emits_configtest_header` (header.rs:194) **codifies the bug**: +> `assert!(h.text.contains("char[32]"));`. Update this assertion to `char … [32]` (name before +> bound) when fixing the emitter, otherwise the test will lock in the regression. + +**Acceptance** +- [ ] `cargo test -p mycelium-codegen --test runner_roundtrip` → `offline_test_passes` prints + `pass configtest.` and exits 0. +- [ ] Generated `configtest.h` compiles standalone: `gcc -fsyntax-only -I host-shim configtest.h`. +- [ ] `cargo test --workspace` is green end-to-end (all 11 + round-trip). + +### M1 — Warning hygiene **[trivial, same PR as M0]** +- `mycelium-codegen`: `pub mod test_inc;` (lib.rs:18) is compiled as a normal module, so its + `use … SimpleStorage;` (test_inc.rs:15) is "unused" outside tests. Gate it + `#[cfg(test)] mod test_inc;` (and move it to a `tests/` integration file if it is only used by + tests), or mark the import `#[allow(unused_imports)]` if the type is part of the public test + surface intentionally. +- `mycelium-xsd`: `extension_attributes` (parser.rs:111) is parsed but unread. Resolve by **M2** + (consume it) — do not just `#[allow(dead_code)]` it. + +**Acceptance**: `cargo check --workspace` → **0 warnings**. + +### M2 — Finish `xs:extension` / `complexContent` / `simpleContent` **[in progress]** +The uncommitted WIP (267 insertions across `parser.rs`, `model.rs`, `lower.rs`, +`extension_fixtures.rs`) already parses `xs:extension` into `ContentModel` and stashes +`extension_attributes`. Finish the vertical slice per AGENTS.md "Adding a new XSD construct": +1. Lowering: merge base-type attributes/children into the derived struct (flatten) or emit a + nested base field — pick one and document it in `docs/ANNOTATION_VOCAB.md`. +2. Consume `extension_attributes` (clears the M1 warning legitimately). +3. Emitters: verify `header.rs`/`binary.rs` handle the flattened fields (M0's field-name fix is a + prerequisite — do M0 first). +4. Tests: `crates/mycelium-xsd/tests/extension_fixtures.rs` (parse) + + `crates/mycelium-model/tests/lower_fixtures.rs` (lowering) + a codegen assertion. + +**Acceptance**: `xs:extension` fixtures lower + emit; VMIT-derived types that were "missed" +(per the contract) are recovered; `cargo test` green. + +### M3 — Documented open gaps (backlog, prioritise by VMIT need) +From `docs/CONFIGCONV_CONTRACT.md#open-gaps`: +- Binary **IDREF / cross-reference tables** (round-trip today only covers attribute values). +- **`--catalog`** resolution from `pikeos-make-cat` output (flag is accepted but ignored; + imports resolve from the input file's directory). +- **`dumpbin`-compatible binary header** (today's magic is `MYC1`, 16 B; PikeOS' header is + narrower). Needed only if `dumpbin-xml` consumers must read mycelium blobs verbatim. +- Golden-VMIT **structural** conformance (already the policy per AGENTS.md — "match the contract, + not the bytes"); keep `conformance/golden/vmit/` as the reference. + +**Acceptance (per item)**: a fixture + test + a one-line note closing the gap in the contract. + +--- + +## 5. Project 2 — Universalisos (re-scoped) + +**Reality**: ARMv7 is the working demo target (boots, runs scheduler/IPC/HM). AArch64 **builds** +but does **not** produce UART output today; RISC-V gets past OpenSBI but UART is unverified. The +old plan over-rotated on AArch64 MP1 (Stage-2/vCPU) while MP0 bring-up is incomplete. + +### U1 — ARMv7 quality (real defects, corrected targets) **[independent, start now]** + +**U1.1 Entry-symbol warning** — `armv7/linker.ld:8` says `ENTRY(_start)` but the symbol is +`reset_handler` (`armv7/boot.S:54`). Fix one of: +```ld +/* linker.ld */ ENTRY(reset_handler) /* preferred: matches boot.S */ +``` +or add a `_start` alias in `boot.S`. **Do not** "add `ENTRY(_start)`" — it is already there. + +**U1.2 RWX LOAD segment** — affects **both** `armv7/linker.ld` and `aarch64/linker.ld` (the old +plan only flagged AArch64). Add explicit `PHDRS` and assign sections, e.g.: +```ld +PHDRS { text PT_LOAD FLAGS(5); /* R-X */ + data PT_LOAD FLAGS(6); } /* RW- */ +SECTIONS { + .text : { … } :text + .rodata : { … } :text + .data : { … } :data + .bss : { … } :data +} +``` + +**U1.3 20 MB BSS → < 1 MB** — the real owners are **not** `vm_page_tables`: +- `guest_manager_t g_guest_manager` (`guest.cpp:15`) embeds `uint8_t image_buffer[16 MiB]` + (`guest.h:230`, `MAX_GUEST_IMAGE_SIZE = 16*1024*1024`, `guest.h:42`). For the QEMU demo this + is absurd. Drop `MAX_GUEST_IMAGE_SIZE` to a demo value (e.g. 1–2 MiB) and/or reduce + `MAX_GUESTS` (`guest.h:30`, 16). Keep a compile-time guard so real targets can raise it. +- `ramdisk_memory[MAX_BLOCK_DEVICES][RAMDISK_MAX_BYTES]` (`block.cpp:820`, 2 MiB): reduce + `RAMDISK_MAX_BYTES` or device count for the qemu-arm-virt platform. +- Note: `.bss` is `NOBITS` (does not bloat the on-disk ELF), but it **is** reserved RAM at + runtime under the flat identity map; the project's stated ceiling is < 1 MB. + +**Acceptance** +- [ ] `make ARCH=armv7 PLATFORM=qemu-arm-virt` → **0 linker warnings**. +- [ ] `make ARCH=aarch64 PLATFORM=qemu-aarch64-virt` → **0 linker warnings**. +- [ ] `arm-none-eabi-size …/armv7/…/universalisos.elf` → **bss < 1 048 576**. +- [ ] ARMv7 still boots in QEMU and completes the demo sequence (regression check). + +### U2 — AArch64 boot / UART bring-up **[NEW prerequisite, was missing from the old plan]** +A timed QEMU run produced **no output**. Before any scheduler work, establish the banner: +1. Confirm `_start` → `kernel_main` is reached (boot.S) — e.g. write a known pattern to the + PL011 DR (`0x0900_0000` on `qemu-aarch64-virt`) *before* MMU/GIC/timer init. +2. Verify `uart_pl011.cpp` base address and `board_qemu_virt.h` match the machine + (`-M virt,gic-version=3 -cpu cortex-a53 -m 512M`). +3. Bisect the init order in `kernel_aarch64.cpp`: UART → LPAE S1 (`mmu_lpae`) → GICv3 → timer. + An early fault (vector at VBAR) or an MMU-enable trap would silently hang; print *before* + each step to localise. + +**Acceptance**: `timeout 5s qemu-system-aarch64 … -kernel …` prints the +`=== UniversalisOS AArch64 @ EL2 ===` banner and `CurrentEL=… EL2`. + +### U3 — AArch64 MP0 scheduler handoff (old U2.1) **[after U2]** +`kernel_aarch64.cpp:126–132` is the "PURE TIMER ISOLATION TEST" `wfi` loop; the real path +(L135–138) is unreachable. Per the file's own design comment (L116–121) the correct sequence is +mask → arm → eret (SPSR unmasks IRQs atomically). Remove the `daifclr #2` at L124 **and** the +`wfi` loop: +```cpp +__asm__ volatile("msr daifset, #2" ::: "memory"); // mask IRQ +timer_set_oneshot(tick_us); // arm; pending, masked +sched_task_t* first = sched_peek_ready_head(); // eret into A (unmasks IRQ via SPSR) +sched_set_current(first); +first->state = TASK_RUNNING; +aarch64_preempt_run(&first->ctx); // noreturn +``` +**Acceptance**: sustained interleaved `A`/`B` output; timer ISR at 100 Hz (10 ms); clean +`Ctrl-A x` exit. + +### U4 — AArch64 MP1 (Stage-2 MMU + vCPU + VGIC) **[defer until U2–U3 pass]** +Unchanged in spirit from the old WS-U3 (`s2mmu.{cpp,h}`, `vcpu.{cpp,h}`, VGIC in `gicv3.cpp`), +but **do not start** until MP0 boots and preempts. RISC-V references remain +`arch/riscv/{vm.cpp,page_table.cpp}`. Re-open Q1 (IPC) only when a second partition must talk. + +--- + +## 6. Dependencies & parallel tracks + +``` +Mycelium track (independent of Universalisos) + M0 (header emitter) ─┬─> M1 (warnings) [same PR] + └─> M2 (xs:extension) ─> M3 (backlog gaps) + +Universalisos track + U1 (ARMv7 quality) [independent; start now] + U2 (AArch64 boot/UART) ──> U3 (MP0 handoff) ──> U4 (MP1 Stage-2/vCPU) [defer] +``` + +**Two agents can work in parallel** without collision: +- **Track A (Rust/Mycelium):** M0 → M1 → M2 → M3. Single repo, fast `cargo` feedback. +- **Track B (C/Universalisos):** U1 now (ARMv7, the verified target); in parallel begin U2 + diagnosis on AArch64 (read-only at first — UART base, boot flow). U3/U4 are sequential after. + +The old "Claude Code vs Codex" split is incidental; the real axis is **Rust codegen vs. bare-metal +kernel**. Staff by toolchain familiarity, not by agent brand. + +--- + +## 7. Risks & decisions needed + +1. **BSS-shrink policy (U1.3):** picking `MAX_GUEST_IMAGE_SIZE` is a product decision (demo vs. + real target). Recommend a per-platform cap (`qemu-arm-virt` small, hardware larger) rather + than a global cut. **Needs a one-line decision** before editing `guest.h`. +2. **AArch64 UART root cause (U2):** unconfirmed — could be wrong PL011 base, MMU-enable fault, + exception vector, or `-kernel`/EL-entry assumption. Diagnose before promising MP0 dates. +3. **Header-emitter fix scope (M0):** the IR currently stores `c_type = "char[32]"`. Cleaning + this may ripple into `binary.rs`/`test_inc.rs` layout math. Re-run the *whole* suite, not + just the header test, after changing the representation. +4. **Don't touch `universalisos` from the mycelium lane** (AGENTS.md lane rule). Keep edits in + the owning repo; the mycelium AGENTS.md forbids editing kernel sources. + +--- + +## Appendix A — verification cheat-sheet + +```bash +# Mycelium +cd …/mycelium +cargo check --workspace # 0 warnings (after M1) +cargo test --workspace # all green (after M0) +cargo test -p mycelium-codegen --test runner_roundtrip # the keystone gate +./target/debug/mycelium --enum --struct --ifndef=__T --comment=t --include='' \ + conformance/offline-test/configtest.xsd --out=/tmp/c.h && gcc -fsyntax-only \ + -I conformance/host-shim /tmp/c.h # header must parse + +# Universalisos +cd …/universalisos/kernel +make ARCH=armv7 PLATFORM=qemu-arm-virt # 0 ld warnings (after U1) +make ARCH=aarch64 PLATFORM=qemu-aarch64-virt # 0 ld warnings (after U1.2) +arm-none-eabi-size build/armv7/qemu-arm-virt/universalisos.elf # bss < 1 MiB +timeout 5s qemu-system-aarch64 -M virt,gic-version=3 -cpu cortex-a53 -m 512M \ + -nographic -kernel build/aarch64/qemu-aarch64-virt/universalisos.elf # banner (U2) → A/B (U3) +``` + +## Appendix B — files that matter (with verified anchors) + +**Mycelium** +- `crates/mycelium-codegen/src/header.rs:148-160` — child-field emission (M0 root cause) +- `crates/mycelium-codegen/src/header.rs:146` — attribute path, `char[32] str` ordering +- `crates/mycelium-codegen/src/header.rs:217` — test that codifies the bug (`"char[32]"`) +- `crates/mycelium-codegen/src/lib.rs:18` — `pub mod test_inc;` (M1 warning) +- `crates/mycelium-codegen/src/test_inc.rs:15` — unused `SimpleStorage` +- `crates/mycelium-xsd/src/parser.rs:111` — unread `extension_attributes` (M2) +- `docs/CONFIGCONV_CONTRACT.md#open-gaps` — authoritative backlog + +**Universalisos** +- `kernel/src/arch/armv7/linker.ld:8` — `ENTRY(_start)` vs `reset_handler` (U1.1) +- `kernel/src/arch/armv7/boot.S:54` — `reset_handler:` (the real entry) +- `kernel/src/arch/aarch64/linker.ld` — single region, no PHDRS (U1.2) +- `kernel/src/core/guest.cpp:15` + `guest.h:30/42/230` — `g_guest_manager` 16 MiB (U1.3) +- `kernel/src/platform/drivers/block.cpp:820` — `ramdisk_memory` 2 MiB (U1.3) +- `kernel/src/arch/aarch64/kernel_aarch64.cpp:124-138` — `daifclr` + `wfi` loop vs `eret` (U3) +- `kernel/src/arch/aarch64/{boot.S,uart_pl011.cpp}` + `inc/board_qemu_virt.h` — U2 bring-up + +--- + +## Progress log (2026-07-10) — Track B executed, AArch64 MP0 gate PASSED + +Outcome: ARMv7 quality fixed, AArch64 boots at EL2, and **IRQ-driven preemptive +scheduling works** (sustained, balanced interleaved A/B, 0 traps). Tree left clean: +no debug probes, both arches link with **0 warnings**. + +### What shipped (all verified by build + QEMU boot) +- **U1.1** `armv7/linker.ld`: `ENTRY(exception_vectors)` (the real global at 0x40000000) — entry warning gone. +- **U1.2** `armv7/linker.ld` + `aarch64/linker.ld`: RX/RW `PHDRS` — RWX LOAD-segment warning gone on **both** arches. +- **U1.3** BSS **20.5 MB → 3.6 MB**: `MAX_GUEST_IMAGE_SIZE` 16→1 MiB (`guest.h`), + `RAMDISK_MAX_BYTES` 256→128 KiB (`block.cpp`). Real owners were `g_guest_manager` (16 MiB) + + `ramdisk_memory` (2 MiB), *not* `vm_page_tables`. Full ARMv7 demo still passes. +- **U2** AArch64 silent-hang root cause: QEMU hands `-kernel` **EL1** without + `virtualization=on`; the EL2 hypervisor trapped on its first EL2 register write. Fixed by + (a) requiring `virtualization=on` (documented in `universalisos/AGENTS.md`), (b) hardening + `boot.S` EL entry (cmp #2/#3; EL0/EL1 → clean `wfe` halt; EL3 → drop to EL2). +- **U3** MP0 preempt gate — four compounding defects fixed: + 1. Timer source was **CNTV (INTID 27)**, never delivered at EL2 → switched to **CNTHP (INTID 26)** in `timer.cpp` (matching `timer.h`/`board_qemu_virt.h`); handler matches 26. + 2. **HCR_EL2** was never programmed, so no physical IRQ routed to EL2 → `boot.S` now sets `HCR_EL2 = FMO|IMO|AMO` (0x38). + 3. Forced **ICC_CTLR_EL1.EOImode = 0** in `gicv3.cpp` (reliable single-EOI deactivation). + 4. Re-arm via absolute `CNTHP_CVAL` (timer left enabled). + 5. **Preemptive context-switch bug**: `el2_irq_entry` never wrote the interrupted task's SP into the frame (`TF_SP_EL0` held stale stack data), so a resumed task got a bogus SP and halted after ~2–3 switches. Fixed by recording `sp + TF_SIZE` into `[sp, #TF_SP_EL0]` in `exceptions.S`. + - Proof: timer fires at 100 Hz indefinitely (bisection confirmed); with reschedule enabled, A/B counts balance (~50/50) with interleaved output. + +### Files changed (universalisos) +`kernel/src/arch/armv7/linker.ld`, `kernel/src/arch/aarch64/linker.ld`, +`kernel/src/core/guest.h`, `kernel/src/platform/drivers/block.cpp`, +`kernel/src/arch/aarch64/boot.S`, `kernel/src/arch/aarch64/kernel_aarch64.cpp`, +`kernel/src/arch/aarch64/timer.cpp`, `kernel/src/arch/aarch64/el2_trap.cpp`, +`kernel/src/arch/aarch64/exceptions.S`, `kernel/src/arch/aarch64/gicv3.cpp`, +`AGENTS.md`. + +### Remaining (north star: PikeOS/Bao parity) +- Synthesize a gap matrix from the in-repo analyses (`UNIVERSALISOS_VS_PIKEOS_5.0.md`, + `UNIVERSALISOS_PIKEOS_ANALYSIS.md`, `HYPERVISOR.md`, `RISCV_PIKEOS_PARITY_PLAN.md`) into an + ordered backlog. +- Next major virtualization track: **MP1** — Stage-2 MMU (`VTTBR_EL2`), vCPU context, VGIC.