docs(sessions): 2026-07-10 replan mycelium-universalisos

This commit is contained in:
fabiorafaelcoutada 2026-07-12 20:53:09 +01:00
parent 5a6296eaa4
commit 1867673ce7
2 changed files with 641 additions and 0 deletions

View file

@ -0,0 +1,230 @@
# Universalisos ↔ PikeOS 5.0 / Bao — Parity Backlog (verified)
> **Created**: 2026-07-10 (synthesis of in-repo analyses, **reconciled with verified ground truth**)
> **Author**: Kimi (root agent, `replica-omnisciente`)
> **Scope**: `/home/fabiorafaelcoutada/portugalfuturista/universalisos`
> **Supersedes**: the per-doc verdicts where they are stale or mutually contradictory (see §6).
---
## 0. How to read this
- Two reference targets, two different parity axes:
- **PikeOS 5.0** — the *RTOS/partition/API/safety* axis (ARINC 653 partitions, IPC, HM,
personalities, syscalls, system-software layer).
- **Bao** — the *ARMv8/RISC-V hardware-virtualization* axis (Stage-2 MMU, GICv3 VGIC, vCPU
world-switch, static partitioning). Bao is the right model for the **AArch64** track, **not**
the x86 VMX/EPT/VMCS vocabulary that some older analyses use.
- Status key: **✅ done (verified)** · **🟡 partial / framework** · **❌ missing** · **— not applicable**.
- Rows marked **(V)** are verified by build+QEMU this session; others are doc-derived and flagged.
---
## 1. Verified ground-truth snapshot (2026-07-10) — supersedes stale docs
| Track | Verified state | Supersedes |
|---|---|---|
| **ARMv7 / qemu-arm-virt** | Boots, full demo end-to-end (scheduler, ARINC sampling/queuing/shmem IPC, 3-tier HM, PV ABI, preempt A/B). **0 linker warnings.** BSS 20.5 MB → 3.6 MB. | `status_report` "MATURE" (still accurate) |
| **AArch64 / qemu-aarch64-virt** | **MP0 COMPLETE.** Builds (0 warnings), boots at EL2 (`virtualization=on`), LPAE S1, GICv3, CNTHP @100 Hz, PikeOS scheduler, preemptive ctx switch → interleaved A/B. | `status_report` "build broken / MP0 ~15%" (stale) |
| **RISC-V / polarfire** | Functional parallel track (own scheduler/VMM/hypercalls/Sv39). Not exercised this session. | `status_report` "FUNCTIONAL" (assumed current) |
| **Tooling (mycelium/CODEO)** | `mycelium` XSD codegen exists & compiles; one round-trip bug + documented gaps. | `VS_PIKEOS_5.0` "Tooling 0%" (stale) |
> The older analyses (`UNIVERSALISOS_VS_PIKEOS_5.0.md`, `UNIVERSALISOS_PIKEOS_ANALYSIS.md`) use
> **x86 virtualization terms** (VT-x, VMX, EPT, VPID, VMCS) and pre-date mycelium + the AArch64 MP0
> work. Treat their percentages as directional only; this matrix re-maps everything to ARM.
---
## 2. Parity gap matrix
### A. CPU / virtualization core (axis: **Bao** on AArch64; PikeOS VM layer on ARMv7)
| Feature | PikeOS | Bao | Universalisos ARMv7 | Universalisos AArch64 | Effort |
|---|---|---|---|---|---|
| Boot + EL/priv bring-up | ✅ | ✅ | ✅ (V) | ✅ EL2 (V) | — |
| Stage-1 MMU (flat identity) | ✅ | ✅ | ✅ (V) | ✅ LPAE (V) | — |
| Exception/trap vectors | ✅ | ✅ | ✅ (V) | ✅ VBAR_EL2 (V) | — |
| Context switch (kernel tasks) | ✅ | ✅ | ✅ (V) | ✅ + preempt fix (V) | — |
| Preemptive scheduling (tick) | ✅ | ✅ | ✅ (V) | ✅ CNTHP @100 Hz (V) | — |
| **Stage-2 MMU (VTTBR_EL2 / IPA→PA)** | ✅ | ✅ | — (no S2 on v7) | ❌ | **L** |
| **vCPU struct + world switch (ERET to guest EL1)** | ✅ | ✅ | 🟡 (vm.cpp frame) | ❌ | **L** |
| **VGIC (GICv3 ICH_* virtual interrupts)** | ✅ | ✅ | — (GICv2) | ❌ | **M** |
| **Virtual timer to guest (CNTVOFF_EL2)** | ✅ | ✅ | — | ❌ | **S** |
| **HCR_EL2 trap config (sysreg/IO/WFI trapping)** | ✅ | ✅ | — | 🟡 (IMO/FMO/AMO only) (V) | **M** |
| Hypercall ABI (guest→hyp) | ✅ | ✅ | ✅ PV ABI (V) | ❌ | **M** |
| SMP (real multi-CPU scheduler) | ✅ | ✅ | 🟡 framework | ❌ | **L** |
### B. Partitioning, scheduling, IPC (axis: **PikeOS / ARINC 653**)
| Feature | PikeOS | Bao | ARMv7 | AArch64 | Effort |
|---|---|---|---|---|---|
| Static partitions + windows | ✅ | ✅ | ✅ (V) | 🟡 schema only (V) | **M** |
| ARINC 653 sampling ports | ✅ | — | ✅ (V) | ❌ | **M** |
| ARINC 653 queuing ports | ✅ | — | ✅ (V) | ❌ | **M** |
| Shared memory + events | ✅ | 🟡 | ✅ (V) | ❌ | **M** |
| Time-partition scheduler | ✅ | ✅ | ✅ (V) | ✅ (V) | — |
| Rate/Deadline-monotonic, PIP | ✅ | — | ❌ | ❌ | **M** |
| Deadline-miss detection | ✅ | — | 🟡 (HM) | ❌ | **S** |
| Capability-based access (MDB) | 🟡 | — | ✅ seL4-style (V) | ❌ | **M** |
### C. Health, safety, system software (axis: **PikeOS**)
| Feature | PikeOS | Bao | ARMv7 | AArch64 | Effort |
|---|---|---|---|---|---|
| 3-tier Health Monitoring | ✅ | 🟡 | ✅ (V) | ❌ | **M** |
| ASIL classification / fault containment | ✅ | — | 🟡 framework | ❌ | **L** |
| MISRA C++ / ISO 26262 / DAL | ✅ | — | ❌ | ❌ | **XL** (cert) |
| Personalities (APEX/POSIX/Linux/Ada) | ✅ | — | ❌ | ❌ | **XL** |
| Network stack (lwIP/TCP-IP) | ✅ | — | ❌ (driver only) | ❌ | **L** |
| Certified filesystem (FAT/JFFS2) | ✅ | — | ❌ | ❌ | **L** |
| Real Linux guest boot | ✅ | ✅ | ❌ (PV ABI only) | ❌ | **L** (needs MP1) |
### D. Device & I/O (axis: **PikeOS + Bao**)
| Feature | PikeOS | Bao | ARMv7 | AArch64 | Effort |
|---|---|---|---|---|---|
| UART / timer / GIC drivers | ✅ | ✅ | ✅ (V) | ✅ (V) | — |
| Block / net / GPIO / I2C / SPI / USB / PCI | ✅ | 🟡 | ✅ present (V) | ❌ | **L** (port) |
| Virtio emulation (blk/net/console) | 🟡 | ✅ | 🟡 (states exist) | ❌ | **M** |
| Device passthrough + IOMMU (SMMU) | ✅ | ✅ | ❌ | ❌ | **XL** |
| MMIO trap-emulate for guests | ✅ | ✅ | — | ❌ | **M** (needs S2) |
> **Driver caveat:** `UNIVERSALISOS_PIKEOS_ANALYSIS.md` claims "drivers 95100% stubs", while
> `status_report.md` marks ARMv7 drivers "✅ Complete". The ARMv7 demo does exercise block/network
> (e1000/virtio/rtl8139 BSS states exist; block demo reports 0 errors). Treat drivers as
> **present-but-largely-emulated** on QEMU, not hardware-validated. (V) for "present", not "proven on HW".
### E. Tooling & config (axis: **PikeOS CODEO**)
| Feature | PikeOS | Universalisos | Effort |
|---|---|---|---|
| XSD → C/binary codegen (`pikeos-configconv`) | ✅ | 🟡 `mycelium` (V) — works, 1 round-trip bug + gaps | **SM** |
| VMIT XML config → kernel | ✅ | ✅ `cfg_parser.cpp` (V) | — |
| IDE / Theia (CODEO) integration | ✅ | 🟡 scaffold (mycelium `ide/`) | **M** |
| Host test harness (`configmore --test`) | ✅ | 🟡 `mycelium --test` (WIP) | **S** |
---
## 3. Prioritized, ordered backlog
> Order principle: **foundation before features**, **ARMv7 is the reference of record** (don't regress it),
> **AArch64 is the growth track** (virtualization lives here), **RISC-V stays the parallel HW track**.
> Each item: `ID — title — [MPx] — effort — verify`.
### Horizon H0 — done / keep green (this session)
- **DONE-1** ARMv7 linker `ENTRY` + RX/RW PHDRS → 0 warnings (V). Keep as the clean baseline.
- **DONE-2** BSS 20.5 MB → 3.6 MB (guest image 16→1 MiB, ramdisk 256→128 KiB) (V).
- **DONE-3** AArch64 MP0: EL2 boot (`virtualization=on`), LPAE S1, GICv3, CNTHP timer, preemptive
scheduler → interleaved A/B, 0 traps (V). **This is the MP0 gate; it now passes.**
### Horizon H1 — virtualization foundation (the PikeOS/Bao core gap) ← **next track**
- **MP1.1 — Stage-2 MMU (AArch64)**`[MP1]` — **L**
Build `VTTBR_EL2` LPAE Stage-2 tables (IPA→PA), `VTCR_EL2`, `HCR_EL2.VM=1`, per-vCPU `VPIDR_EL2`/`VMPIDR_EL2`.
*Verify*: identity-mapped guest IPA region; a Stage-2 fault routes to EL2 with `ESR.EC=0x20/0x24`.
- **MP1.2 — vCPU struct + world switch**`[MP1]` — **L**
`vCPU` {GPRs, ELR_EL2, SPSR_EL2, SP_EL1, SCTLR_EL1, TTBR0_EL1, VBAR_EL1, …}; `vcpu_run()` saves hyp, loads guest, `ERET`; exit traps back to EL2. Reuse `context_switch.S` frame shape.
*Verify*: enter a tiny guest at EL1, execute a few instructions, trap on `hvc`/WFI, return to EL2.
- **MP1.3 — VGIC (GICv3 list registers)**`[MP1]` — **M**
Program `ICH_*_EL2` (HCR/VSEIR/LR/VMCR), inject a vIRQ to the guest, EOI/maintenance IRQ path.
*Verify*: guest receives a virtual PPI and EOIs it without a raw trap storm.
- **MP1.4 — Virtual timer to guest**`[MP1]` — **S**
`CNTVOFF_EL2`, `CNTV_CVAL_EL0`/`CNTV_CTL_EL0` routing; guest sees a vtimer tick.
*Verify*: guest EL1 reads `CNTVCT` and takes a vtimer interrupt.
- **MP1.5 — Trap config matrix**`[MP1]` — **M**
Decide/encode `HCR_EL2.{TWI,TWE,TSC,TVM,…}` and CPACR/TIDCP traps; document the policy.
*Verify*: table-driven test of each trap class.
### Horizon H2 — core PikeOS parity (RTOS axis)
- **CORE-1 — Unify AArch64 with `src/core/` IPC + HM + cap**`[MP1/MP3]` — **M**
Port the ARMv7 core subsystems (sampling/queuing/shmem IPC, 3-tier HM, cap MDB) to AArch64 so both ARM
tracks share one core (resolves open Q1: shared `src/core/` over standalone).
*Verify*: ARMv7 IPC/HM demo runs unchanged on AArch64.
- **CORE-2 — Real SMP**`[MP5]` — **L**
Secondary CPU bring-up, IPI/SGI, per-CPU runqueues, TLB shootdown. ARMv7 first (core exists), then AArch64.
*Verify*: two tasks pinned to two CPUs both preempt via tick.
- **CORE-3 — ARINC 653 scheduling hardening**`[—]` — **M**
RMS/DMS, priority inheritance, deadline-miss → HM action, temporal-isolation guarantees.
*Verify*: injected overrun triggers HM; windows enforced under load.
- **CORE-4 — PikeOS syscall surface**`[—]` — **M**
Extend beyond the present task/TLS/ExRegs set toward partition/thread/memory/sync APIs (libpikeos-shape).
*Verify*: a conformance test list (name→implemented?) tracked in-tree.
### Horizon H3 — system-software & platform reach (the "510%" layer)
- **SYS-1 — Linux guest boot**`[MP3]`**L** *(blocked on MP1)*
Load a real kernel + DTB + initramfs via Stage-2, boot to a shell under the hypervisor.
*Verify*: Linux boots as a guest vCPU on AArch64.
- **SYS-2 — Personalities (APEX first, then POSIX)**`[—]` — **XL**
APEX ARINC 653 process API surface over the existing IPC; POSIX later.
*Verify*: APEX sample app links & runs in a partition.
- **SYS-3 — Network stack (lwIP)**`[—]` — **L**
lwIP as a partition/personality; virtio-net or e1000 backend.
*Verify*: ping/echo from a guest/partition.
- **SYS-4 — Filesystem (FAT/JFFS2)**`[—]` — **L**
Block-backed FS; certified subset documented.
*Verify*: mount/read/write in a partition.
- **PLAT-1 — Hardware ports**`[MP2]` — **L**
i.MX 8M Plus (AArch64), STM32MP257 (AArch64/CA35), i.MX6 (ARMv7) platform dirs + BSP.
*Verify*: boots on real HW (or documented bring-up checklist).
- **SAFE-1 — MISRA / fault containment / ASIL enforcement**`[—]`**XL** *(certification horizon)*
Coding-standard gate, WCET, audit trails. Long-horizon; track, don't front-load.
### Horizon H4 — tooling (mycelium/CODEO) — Track A (separate agent; **non-blocking**)
- **MYC-1 — Fix `header.rs` emitter**`[M3]`**S** (highest-leverage mycelium fix)
Restore child field names + array bounds; green the `offline_test_passes` round-trip.
- **MYC-2 — Finish `xs:extension`**`[M3]`**S** (wire `extension_attributes` through emitter).
- **MYC-3 — Clean 2 warnings**`[M3]`**S** (unused import, unread field).
- **MYC-4 — Theia/CODEO view + `configmore --test`**`[—]`**M**.
---
## 4. Recommended next execution track: **MP1 (AArch64 hardware virtualization)**
**Why now:** H0 is green on both ARM tracks; the single biggest PikeOS/Bao parity gap is *hardware
virtualization*, and it only exists on AArch64. Everything in H2 (guest boot, real partitions with
isolation, MMIO trap-emulate) is gated by MP1. Starting here unblocks the most downstream work and
matches the in-repo MP0→MP5 plan.
**First slice (de-risked, observable):** Stage-2 identity map + a *single-instruction guest*.
1. Build Stage-2 L0→L1 (1 GiB block) tables mapping IPA==PA for the QEMU DRAM window; set `VTCR_EL2`,
`VTTBR_EL2`, `HCR_EL2.VM=1`, `TGE=0`, `RW=1`.
2. Allocate one `vCPU`, seed `ELR_EL2=guest_entry`, `SPSR_EL2=EL1h`, `SP_EL1`, minimal `SCTLR_EL1`.
3. Guest entry = a 3-instruction blob: `mov x0,#0xAB; hvc #0; b .` (the `hvc` traps to EL2).
4. `vcpu_run()``ERET` into EL1 → trap on `hvc` → EL2 handler prints `x0==0xAB` → park.
*Verify criterion:* one clean EL1 entry and one `hvc` round-trip to EL2 with the guest GPR intact.
No VGIC, no timer, no MMIO yet — those are MP1.3/1.4 once the world-switch is proven.
**Guardrails (from this session's scars):** always boot QEMU with `virtualization=on`; add an early
`CurrentEL` print; for the world-switch debug, temporarily force the non-reschedule path to bisect
"timer vs switch" exactly as we did for MP0.
---
## 5. Mycelium tie-in (Track A, parallel — does not block MP1)
Mycelium already **builds** (`cargo check` clean); its real blocker is the `header.rs` emitter
(field-name loss + array-bound misplacement → `gcc` rejects the generated header), codified by a unit
test. It is the *PikeOS CODEO `pikeos-configconv`* parity axis (Tooling, §E) — valuable but independent of
the hypervisor virtualization track. Run it on the **mycelium agent** (it never edits universalisos).
---
## 6. Caveats on the source analyses (why this matrix supersedes them)
- **Stale**: `UNIVERSALISOS_VS_PIKEOS_5.0.md` & `UNIVERSALISOS_PIKEOS_ANALYSIS.md` pre-date mycelium and
the AArch64 MP0 work; e.g. "Tooling 0%", "AArch64 boot ✅" (was false until this session), "drivers
95100% stubs" vs `status_report` "drivers ✅ Complete".
- **x86-biased**: both use VT-x/VMX/EPT/VPID/VMCS vocabulary. Universalisos is an ARM/RISC-V hypervisor;
parity is measured against **Stage-2/VTTBR/GICv3-VGIC/vCPU** (Bao), re-mapped here.
- **Now-current**: `status_report.md`'s EXISTS/MISSING + MP0MP5 scaffold is the closest to truth, but its
AArch64 row ("build broken / MP0 ~15%") is outdated by this session's MP0 pass.
- **Percentages** in all docs are directional. The matrix uses ✅/🟡/❌ + **(V)** verified flags instead.
---
## 7. Sources
- `universalisos/UNIVERSALISOS_VS_PIKEOS_5.0.md` — 11-category feature % (x86-biased, directional).
- `universalisos/UNIVERSALISOS_PIKEOS_ANALYSIS.md` — P1/P2/P3 critical gaps + 12/18-month roadmap.
- `universalisos/universalisos_status_report.md` — per-arch maturity, EXISTS/MISSING, MP0MP5.
- `universalisos/RISCV_PIKEOS_PARITY_PLAN.md` — RISC-V S-mode paravirt map (parallel track).
- Verified this session (build + QEMU): ARMv7 demo, AArch64 MP0, BSS cut, linker hygiene.

View file

@ -0,0 +1,411 @@
# Replanned Sprint — Mycelium & Universalisos (evidence-based)
> **Created**: 2026-07-10 (re-analysis)
> **Supersedes**: the antigravity multi-agent plan of the same date
> **Analyst**: Kimi (root agent, `replica-omnisciente`)
> **Repos** (siblings of this brain, **not** the in-tree `mycelium-rd/`):
> - `/home/fabiorafaelcoutada/portugalfuturista/mycelium`
> - `/home/fabiorafaelcoutada/portugalfuturista/universalisos`
---
## 0. TL;DR — the old plan is built on stale premises
The antigravity plan frames both projects as *early-construction* efforts with a few
compile errors. **That is not reality.** I built, tested, booted and `nm`-inspected
both trees today. The corrected picture:
| Project | Old-plan assumption | Verified reality (2026-07-10) |
|---|---|---|
| **Mycelium** | "3 parser compile errors block `cargo check`; pipeline (lowering/emitters/CLI) not yet built" | `cargo check --workspace` **passes**. The whole pipeline (parser → IR lowering → C/asm/binary/CRC emitters → drop-in CLI) **already exists and compiles**, with 11 tests, golden VMIT fixtures and a round-trip harness. The only failing gate is **one test** (`offline_test_passes`): the **header emitter generates malformed C**. |
| **Universalisos ARMv7** | "fix `_start` warning; shrink `vm_page_tables[16][4096]` to fix 20 MB BSS" | `_start` warning is real but the cause is inverted (entry symbol is `reset_handler`, not `_start`); the 20 MB BSS is real but **does not come from `vm_page_tables` (256 KB)** — it comes from `g_guest_manager` (**16 MiB**) + `ramdisk_memory` (2 MiB). |
| **Universalisos AArch64** | "builds, healthy (17 KB text / 79 KB BSS); just remove the `wfi` isolation loop to reach MP0 A/B" | Build numbers are correct, **but `qemu-system-aarch64` produces zero UART output** — the kernel does not even print its banner. MP0 is gated by **boot/UART bring-up**, upstream of the scheduler handoff. |
Net: less "build from scratch", more "fix specific, root-caused defects and finish in-progress work".
---
## 1. Method & evidence
Every claim below was reproduced, not inferred:
```bash
# Mycelium
cargo check --workspace # exit 0 (2 warnings)
cargo test --workspace # 1 failure: offline_test_passes
# manual replay of the round-trip steps → captured real gcc diagnostics
# Universalisos (from kernel/)
make ARCH=armv7 PLATFORM=qemu-arm-virt # exit 0, 2 ld warnings
make ARCH=aarch64 PLATFORM=qemu-aarch64-virt # exit 0, 1 ld warning
arm-none-eabi-nm --print-size --size-sort … # located the 20 MB BSS owner
timeout 6s qemu-system-aarch64 … -kernel … # NO output
```
Tooling present: `cargo`/`rustc`, `arm-none-eabi-gcc`, `aarch64-linux-gnu-{gcc,g++,ld,size,nm}`,
`qemu-system-arm`, `qemu-system-aarch64`. (Note: AArch64 uses the **linux-gnu** toolchain in
`-ffreestanding -nostdlib` mode — see `arch/aarch64/arch.mk:10`; there is no bare-metal
`aarch64-none-elf-` on this box.)
---
## 2. Verified reality vs. the antigravity claims
### Mycelium
- ❌ **WS-M1 (fix 3 parser compile errors)**`cargo check` already passes. The line numbers
cited (L507512, L711, L103 `Attribute`) do not correspond to current compile errors. The
file was edited since (WIP on `xs:extension`); the only parser-side diagnostic is a dead-field
warning (`extension_attributes`, parser.rs:111).
- ✅ The pipeline is implemented:
- `mycelium-model/src/lower.rs` (543 lines) — XSD→IR lowering exists.
- `mycelium-codegen/src/{header,binary,crc,test_inc}.rs` — emitters exist.
- `mycelium-cli/src/main.rs` — CLI is a **superset** of the old WS-M4 flag list
(`--enum/--struct`, `--enum-asm`, `--bin`, `--crc`, `--ifndef/--comment/--include`,
`--wordsize/--endian/--align-tail`, `--catalog`, `--xml`, `--out`, **plus `--test`**).
- ✅ **Real defect**: the round-trip test (`crates/mycelium-codegen/tests/runner_roundtrip.rs`)
drives the CLI then `gcc`-compiles a harness — **gcc rejects the generated header**. Root
cause is in `header.rs` struct-body emission (see §4 M0).
### Universalisos
- ✅ AArch64 build is healthy: `text 16912 / data 0 / bss 79440`. Only an **RWX LOAD-segment**
warning (`/usr/bin/aarch64-linux-gnu-ld: aviso: … segmento LOAD com permissões RWX`).
- ✅ ARMv7 builds but with **two** `ld` warnings and **20,540,000 bytes of BSS**:
```
arm-none-eabi-ld: warning: cannot find entry symbol _start; defaulting to 40000000
arm-none-eabi-ld: warning: … has a LOAD segment with RWX permissions
text data bss dec hex
172288 179280 20540000 20891568 13ec7b0
```
- 🔧 The `_start` warning is **misdiagnosed** in the old plan. `armv7/linker.ld:8` already
declares `ENTRY(_start)`, but `armv7/boot.S` defines the entry as `reset_handler`
(boot.S:5455) — there is no `_start` symbol. **Fix is `ENTRY(reset_handler)` (or add a
`_start` alias), not "add ENTRY(_start)".**
- 🔧 The 20 MB BSS is **mis-aimed** in the old plan. `nm --size-sort` shows the owners:
```
16777292 b g_guest_manager ← guest.cpp:15 (image_buffer[16 MiB])
2097152 b ramdisk_memory ← block.cpp:820 ([MAX_BLOCK_DEVICES][RAMDISK_MAX_BYTES])
262144 b g_uos_vm_pgdir_pool
262144 b vm_page_tables ← only 256 KB, NOT the problem
206976 b e1000_states
173176 d g_gic_state
...
```
`guest.h:42` sets `MAX_GUEST_IMAGE_SIZE = 16*1024*1024`, embedded as
`uint8_t image_buffer[…]` inside `guest_manager_t` (guest.h:230). Shrinking
`vm_page_tables` (old U1.2) saves ~240 KB of a 20 MB problem — **pointless**.
- ❌ **AArch64 MP0 is further away than stated.** A 6 s `qemu-system-aarch64` run printed
**nothing** — no banner, no `CurrentEL`, nothing. So the "remove the `wfi` loop and observe
interleaved A/B" task (old U2.1) is blocked by an **earlier** bring-up gap: the kernel does
not reach (or does not drive) the PL011. (The `wfi` isolation loop at
`kernel_aarch64.cpp:126132` is confirmed present and the real `eret` path at L135138 is
unreachable — that part of the old plan is accurate, just not the next blocker.)
---
## 3. Open questions — answered with evidence
- **Q1 (AArch64 IPC strategy).** Premature. AArch64 does not boot to a banner yet; IPC is two
milestones out. **Defer the decision.** When reached, prefer the RISC-V pattern (standalone,
arch-local) over refactoring `src/core/ipc/` — AGENTS.md is explicit that AArch64/RISC-V are
self-contained and ARMv7 owns `core/`; pulling core IPC into AArch64 risks cross-arch header
coupling for no near-term payoff.
- **Q2 (Mycelium XML data parsing).** **Already implemented.** The `--bin` and `--test` paths
read an XML instance (`--xml=` or positional) — `main.rs:112153` — and the round-trip test
exercises `configtest.xml`. The gap is *not* parsing; it is **binary cross-reference / IDREF
tables** (documented in `CONFIGCONV_CONTRACT.md#open-gaps`). Treat Q2 as closed.
- **Q3 (QEMU availability).** Present: `qemu-system-aarch64` and `qemu-system-arm` are in
`/usr/bin`. No install needed.
---
## 4. Project 1 — Mycelium (re-scoped)
**Goal unchanged**: drop-in `pikeos-configconv`. **Status**: ~90% done; one correctness bug
blocks the round-trip gate, plus documented open gaps and in-progress `xs:extension` work.
### M0 — Fix the C-header emitter (the keystone) **[highest priority]**
**File**: `crates/mycelium-codegen/src/header.rs` (struct-body loop, **L148160**).
Current (broken) emission drops the **field name** for every child and mis-places array bounds:
```rust
// header.rs:148-160 (today)
for c in &rs.children {
if c.app_info.annotation.ignore { continue; }
if c.max > 1 && c.max < u32::MAX {
out.push_str(&format!(" {}[{}];\n", c.c_type, c.max)); // no name; T[N]; is ill-formed
} else {
out.push_str(&format!(" {};\n", c.c_type)); // no name at all
}
}
```
Captured `gcc` diagnostics prove this exactly:
```
configtest.h:160: error: expected identifier … before '[' → struct teebag_t[10];
configtest.h:173: error: expected specifier-qualifier-list → vehicle_t;
configtest.h:202: error: … before '[' → char[32] str;
configtest.h:197: error: unknown type name 'uw_t' → uw_t uw;
configtest.h:200/201: unknown type name 'stdIPv4_t'/'stdMAC_t'
```
Three concrete fixes:
1. **Emit the child field name**: `T name;` and `T name[N];` (use the child's resolved name; the
`ResolvedChild`/field carries it — confirm field name in `lower.rs`).
2. **Inline strings**: the IR stores `c_type = "char[32]"`, so the attribute path
(`header.rs:146`, `{c_type} {fname}`) yields `char[32] str`. Split into base + bound and emit
`char str[32];` (either carry `{base, size}` in the IR or parse the trailing `[N]` in the
emitter). Prefer carrying structured size from lowering.
3. **Missing typedefs** (`uw_t`, `stdIPv4_t`, `stdMAC_t`, `vehicle_t`): named simple types are
used as field `c_type` but never declared. In the first pass (today only enums are emitted),
also emit `typedef <base> <Name>_t;` for non-enum, non-`cx:external` simple types that lower
to a named alias; honour `cx:external` by *assuming* the typedef is provided by the
`--include`d header and emitting nothing.
> ⚠️ The unit test `emits_configtest_header` (header.rs:194) **codifies the bug**:
> `assert!(h.text.contains("char[32]"));`. Update this assertion to `char … [32]` (name before
> bound) when fixing the emitter, otherwise the test will lock in the regression.
**Acceptance**
- [ ] `cargo test -p mycelium-codegen --test runner_roundtrip``offline_test_passes` prints
`pass configtest.` and exits 0.
- [ ] Generated `configtest.h` compiles standalone: `gcc -fsyntax-only -I host-shim configtest.h`.
- [ ] `cargo test --workspace` is green end-to-end (all 11 + round-trip).
### M1 — Warning hygiene **[trivial, same PR as M0]**
- `mycelium-codegen`: `pub mod test_inc;` (lib.rs:18) is compiled as a normal module, so its
`use … SimpleStorage;` (test_inc.rs:15) is "unused" outside tests. Gate it
`#[cfg(test)] mod test_inc;` (and move it to a `tests/` integration file if it is only used by
tests), or mark the import `#[allow(unused_imports)]` if the type is part of the public test
surface intentionally.
- `mycelium-xsd`: `extension_attributes` (parser.rs:111) is parsed but unread. Resolve by **M2**
(consume it) — do not just `#[allow(dead_code)]` it.
**Acceptance**: `cargo check --workspace`**0 warnings**.
### M2 — Finish `xs:extension` / `complexContent` / `simpleContent` **[in progress]**
The uncommitted WIP (267 insertions across `parser.rs`, `model.rs`, `lower.rs`,
`extension_fixtures.rs`) already parses `xs:extension` into `ContentModel` and stashes
`extension_attributes`. Finish the vertical slice per AGENTS.md "Adding a new XSD construct":
1. Lowering: merge base-type attributes/children into the derived struct (flatten) or emit a
nested base field — pick one and document it in `docs/ANNOTATION_VOCAB.md`.
2. Consume `extension_attributes` (clears the M1 warning legitimately).
3. Emitters: verify `header.rs`/`binary.rs` handle the flattened fields (M0's field-name fix is a
prerequisite — do M0 first).
4. Tests: `crates/mycelium-xsd/tests/extension_fixtures.rs` (parse) +
`crates/mycelium-model/tests/lower_fixtures.rs` (lowering) + a codegen assertion.
**Acceptance**: `xs:extension` fixtures lower + emit; VMIT-derived types that were "missed"
(per the contract) are recovered; `cargo test` green.
### M3 — Documented open gaps (backlog, prioritise by VMIT need)
From `docs/CONFIGCONV_CONTRACT.md#open-gaps`:
- Binary **IDREF / cross-reference tables** (round-trip today only covers attribute values).
- **`--catalog`** resolution from `pikeos-make-cat` output (flag is accepted but ignored;
imports resolve from the input file's directory).
- **`dumpbin`-compatible binary header** (today's magic is `MYC1`, 16 B; PikeOS' header is
narrower). Needed only if `dumpbin-xml` consumers must read mycelium blobs verbatim.
- Golden-VMIT **structural** conformance (already the policy per AGENTS.md — "match the contract,
not the bytes"); keep `conformance/golden/vmit/` as the reference.
**Acceptance (per item)**: a fixture + test + a one-line note closing the gap in the contract.
---
## 5. Project 2 — Universalisos (re-scoped)
**Reality**: ARMv7 is the working demo target (boots, runs scheduler/IPC/HM). AArch64 **builds**
but does **not** produce UART output today; RISC-V gets past OpenSBI but UART is unverified. The
old plan over-rotated on AArch64 MP1 (Stage-2/vCPU) while MP0 bring-up is incomplete.
### U1 — ARMv7 quality (real defects, corrected targets) **[independent, start now]**
**U1.1 Entry-symbol warning** — `armv7/linker.ld:8` says `ENTRY(_start)` but the symbol is
`reset_handler` (`armv7/boot.S:54`). Fix one of:
```ld
/* linker.ld */ ENTRY(reset_handler) /* preferred: matches boot.S */
```
or add a `_start` alias in `boot.S`. **Do not** "add `ENTRY(_start)`" — it is already there.
**U1.2 RWX LOAD segment** — affects **both** `armv7/linker.ld` and `aarch64/linker.ld` (the old
plan only flagged AArch64). Add explicit `PHDRS` and assign sections, e.g.:
```ld
PHDRS { text PT_LOAD FLAGS(5); /* R-X */
data PT_LOAD FLAGS(6); } /* RW- */
SECTIONS {
.text : { … } :text
.rodata : { … } :text
.data : { … } :data
.bss : { … } :data
}
```
**U1.3 20 MB BSS → < 1 MB** the real owners are **not** `vm_page_tables`:
- `guest_manager_t g_guest_manager` (`guest.cpp:15`) embeds `uint8_t image_buffer[16 MiB]`
(`guest.h:230`, `MAX_GUEST_IMAGE_SIZE = 16*1024*1024`, `guest.h:42`). For the QEMU demo this
is absurd. Drop `MAX_GUEST_IMAGE_SIZE` to a demo value (e.g. 12 MiB) and/or reduce
`MAX_GUESTS` (`guest.h:30`, 16). Keep a compile-time guard so real targets can raise it.
- `ramdisk_memory[MAX_BLOCK_DEVICES][RAMDISK_MAX_BYTES]` (`block.cpp:820`, 2 MiB): reduce
`RAMDISK_MAX_BYTES` or device count for the qemu-arm-virt platform.
- Note: `.bss` is `NOBITS` (does not bloat the on-disk ELF), but it **is** reserved RAM at
runtime under the flat identity map; the project's stated ceiling is < 1 MB.
**Acceptance**
- [ ] `make ARCH=armv7 PLATFORM=qemu-arm-virt`**0 linker warnings**.
- [ ] `make ARCH=aarch64 PLATFORM=qemu-aarch64-virt`**0 linker warnings**.
- [ ] `arm-none-eabi-size …/armv7/…/universalisos.elf`**bss < 1 048 576**.
- [ ] ARMv7 still boots in QEMU and completes the demo sequence (regression check).
### U2 — AArch64 boot / UART bring-up **[NEW prerequisite, was missing from the old plan]**
A timed QEMU run produced **no output**. Before any scheduler work, establish the banner:
1. Confirm `_start``kernel_main` is reached (boot.S) — e.g. write a known pattern to the
PL011 DR (`0x0900_0000` on `qemu-aarch64-virt`) *before* MMU/GIC/timer init.
2. Verify `uart_pl011.cpp` base address and `board_qemu_virt.h` match the machine
(`-M virt,gic-version=3 -cpu cortex-a53 -m 512M`).
3. Bisect the init order in `kernel_aarch64.cpp`: UART → LPAE S1 (`mmu_lpae`) → GICv3 → timer.
An early fault (vector at VBAR) or an MMU-enable trap would silently hang; print *before*
each step to localise.
**Acceptance**: `timeout 5s qemu-system-aarch64 … -kernel …` prints the
`=== UniversalisOS AArch64 @ EL2 ===` banner and `CurrentEL=… EL2`.
### U3 — AArch64 MP0 scheduler handoff (old U2.1) **[after U2]**
`kernel_aarch64.cpp:126132` is the "PURE TIMER ISOLATION TEST" `wfi` loop; the real path
(L135138) is unreachable. Per the file's own design comment (L116121) the correct sequence is
mask → arm → eret (SPSR unmasks IRQs atomically). Remove the `daifclr #2` at L124 **and** the
`wfi` loop:
```cpp
__asm__ volatile("msr daifset, #2" ::: "memory"); // mask IRQ
timer_set_oneshot(tick_us); // arm; pending, masked
sched_task_t* first = sched_peek_ready_head(); // eret into A (unmasks IRQ via SPSR)
sched_set_current(first);
first->state = TASK_RUNNING;
aarch64_preempt_run(&first->ctx); // noreturn
```
**Acceptance**: sustained interleaved `A`/`B` output; timer ISR at 100 Hz (10 ms); clean
`Ctrl-A x` exit.
### U4 — AArch64 MP1 (Stage-2 MMU + vCPU + VGIC) **[defer until U2U3 pass]**
Unchanged in spirit from the old WS-U3 (`s2mmu.{cpp,h}`, `vcpu.{cpp,h}`, VGIC in `gicv3.cpp`),
but **do not start** until MP0 boots and preempts. RISC-V references remain
`arch/riscv/{vm.cpp,page_table.cpp}`. Re-open Q1 (IPC) only when a second partition must talk.
---
## 6. Dependencies & parallel tracks
```
Mycelium track (independent of Universalisos)
M0 (header emitter) ─┬─> M1 (warnings) [same PR]
└─> M2 (xs:extension) ─> M3 (backlog gaps)
Universalisos track
U1 (ARMv7 quality) [independent; start now]
U2 (AArch64 boot/UART) ──> U3 (MP0 handoff) ──> U4 (MP1 Stage-2/vCPU) [defer]
```
**Two agents can work in parallel** without collision:
- **Track A (Rust/Mycelium):** M0 → M1 → M2 → M3. Single repo, fast `cargo` feedback.
- **Track B (C/Universalisos):** U1 now (ARMv7, the verified target); in parallel begin U2
diagnosis on AArch64 (read-only at first — UART base, boot flow). U3/U4 are sequential after.
The old "Claude Code vs Codex" split is incidental; the real axis is **Rust codegen vs. bare-metal
kernel**. Staff by toolchain familiarity, not by agent brand.
---
## 7. Risks & decisions needed
1. **BSS-shrink policy (U1.3):** picking `MAX_GUEST_IMAGE_SIZE` is a product decision (demo vs.
real target). Recommend a per-platform cap (`qemu-arm-virt` small, hardware larger) rather
than a global cut. **Needs a one-line decision** before editing `guest.h`.
2. **AArch64 UART root cause (U2):** unconfirmed — could be wrong PL011 base, MMU-enable fault,
exception vector, or `-kernel`/EL-entry assumption. Diagnose before promising MP0 dates.
3. **Header-emitter fix scope (M0):** the IR currently stores `c_type = "char[32]"`. Cleaning
this may ripple into `binary.rs`/`test_inc.rs` layout math. Re-run the *whole* suite, not
just the header test, after changing the representation.
4. **Don't touch `universalisos` from the mycelium lane** (AGENTS.md lane rule). Keep edits in
the owning repo; the mycelium AGENTS.md forbids editing kernel sources.
---
## Appendix A — verification cheat-sheet
```bash
# Mycelium
cd …/mycelium
cargo check --workspace # 0 warnings (after M1)
cargo test --workspace # all green (after M0)
cargo test -p mycelium-codegen --test runner_roundtrip # the keystone gate
./target/debug/mycelium --enum --struct --ifndef=__T --comment=t --include='<x.h>' \
conformance/offline-test/configtest.xsd --out=/tmp/c.h && gcc -fsyntax-only \
-I conformance/host-shim /tmp/c.h # header must parse
# Universalisos
cd …/universalisos/kernel
make ARCH=armv7 PLATFORM=qemu-arm-virt # 0 ld warnings (after U1)
make ARCH=aarch64 PLATFORM=qemu-aarch64-virt # 0 ld warnings (after U1.2)
arm-none-eabi-size build/armv7/qemu-arm-virt/universalisos.elf # bss < 1 MiB
timeout 5s qemu-system-aarch64 -M virt,gic-version=3 -cpu cortex-a53 -m 512M \
-nographic -kernel build/aarch64/qemu-aarch64-virt/universalisos.elf # banner (U2) → A/B (U3)
```
## Appendix B — files that matter (with verified anchors)
**Mycelium**
- `crates/mycelium-codegen/src/header.rs:148-160` — child-field emission (M0 root cause)
- `crates/mycelium-codegen/src/header.rs:146` — attribute path, `char[32] str` ordering
- `crates/mycelium-codegen/src/header.rs:217` — test that codifies the bug (`"char[32]"`)
- `crates/mycelium-codegen/src/lib.rs:18``pub mod test_inc;` (M1 warning)
- `crates/mycelium-codegen/src/test_inc.rs:15` — unused `SimpleStorage`
- `crates/mycelium-xsd/src/parser.rs:111` — unread `extension_attributes` (M2)
- `docs/CONFIGCONV_CONTRACT.md#open-gaps` — authoritative backlog
**Universalisos**
- `kernel/src/arch/armv7/linker.ld:8``ENTRY(_start)` vs `reset_handler` (U1.1)
- `kernel/src/arch/armv7/boot.S:54``reset_handler:` (the real entry)
- `kernel/src/arch/aarch64/linker.ld` — single region, no PHDRS (U1.2)
- `kernel/src/core/guest.cpp:15` + `guest.h:30/42/230``g_guest_manager` 16 MiB (U1.3)
- `kernel/src/platform/drivers/block.cpp:820``ramdisk_memory` 2 MiB (U1.3)
- `kernel/src/arch/aarch64/kernel_aarch64.cpp:124-138``daifclr` + `wfi` loop vs `eret` (U3)
- `kernel/src/arch/aarch64/{boot.S,uart_pl011.cpp}` + `inc/board_qemu_virt.h` — U2 bring-up
---
## Progress log (2026-07-10) — Track B executed, AArch64 MP0 gate PASSED
Outcome: ARMv7 quality fixed, AArch64 boots at EL2, and **IRQ-driven preemptive
scheduling works** (sustained, balanced interleaved A/B, 0 traps). Tree left clean:
no debug probes, both arches link with **0 warnings**.
### What shipped (all verified by build + QEMU boot)
- **U1.1** `armv7/linker.ld`: `ENTRY(exception_vectors)` (the real global at 0x40000000) — entry warning gone.
- **U1.2** `armv7/linker.ld` + `aarch64/linker.ld`: RX/RW `PHDRS` — RWX LOAD-segment warning gone on **both** arches.
- **U1.3** BSS **20.5 MB → 3.6 MB**: `MAX_GUEST_IMAGE_SIZE` 16→1 MiB (`guest.h`),
`RAMDISK_MAX_BYTES` 256→128 KiB (`block.cpp`). Real owners were `g_guest_manager` (16 MiB) +
`ramdisk_memory` (2 MiB), *not* `vm_page_tables`. Full ARMv7 demo still passes.
- **U2** AArch64 silent-hang root cause: QEMU hands `-kernel` **EL1** without
`virtualization=on`; the EL2 hypervisor trapped on its first EL2 register write. Fixed by
(a) requiring `virtualization=on` (documented in `universalisos/AGENTS.md`), (b) hardening
`boot.S` EL entry (cmp #2/#3; EL0/EL1 → clean `wfe` halt; EL3 → drop to EL2).
- **U3** MP0 preempt gate — four compounding defects fixed:
1. Timer source was **CNTV (INTID 27)**, never delivered at EL2 → switched to **CNTHP (INTID 26)** in `timer.cpp` (matching `timer.h`/`board_qemu_virt.h`); handler matches 26.
2. **HCR_EL2** was never programmed, so no physical IRQ routed to EL2 → `boot.S` now sets `HCR_EL2 = FMO|IMO|AMO` (0x38).
3. Forced **ICC_CTLR_EL1.EOImode = 0** in `gicv3.cpp` (reliable single-EOI deactivation).
4. Re-arm via absolute `CNTHP_CVAL` (timer left enabled).
5. **Preemptive context-switch bug**: `el2_irq_entry` never wrote the interrupted task's SP into the frame (`TF_SP_EL0` held stale stack data), so a resumed task got a bogus SP and halted after ~23 switches. Fixed by recording `sp + TF_SIZE` into `[sp, #TF_SP_EL0]` in `exceptions.S`.
- Proof: timer fires at 100 Hz indefinitely (bisection confirmed); with reschedule enabled, A/B counts balance (~50/50) with interleaved output.
### Files changed (universalisos)
`kernel/src/arch/armv7/linker.ld`, `kernel/src/arch/aarch64/linker.ld`,
`kernel/src/core/guest.h`, `kernel/src/platform/drivers/block.cpp`,
`kernel/src/arch/aarch64/boot.S`, `kernel/src/arch/aarch64/kernel_aarch64.cpp`,
`kernel/src/arch/aarch64/timer.cpp`, `kernel/src/arch/aarch64/el2_trap.cpp`,
`kernel/src/arch/aarch64/exceptions.S`, `kernel/src/arch/aarch64/gicv3.cpp`,
`AGENTS.md`.
### Remaining (north star: PikeOS/Bao parity)
- Synthesize a gap matrix from the in-repo analyses (`UNIVERSALISOS_VS_PIKEOS_5.0.md`,
`UNIVERSALISOS_PIKEOS_ANALYSIS.md`, `HYPERVISOR.md`, `RISCV_PIKEOS_PARITY_PLAN.md`) into an
ordered backlog.
- Next major virtualization track: **MP1** — Stage-2 MMU (`VTTBR_EL2`), vCPU context, VGIC.