universalisos/docs/PIKEOS_X86_AUDIT.md
Fábio Coutada 47f5885da6 docs: add RTOS audit, bibliography, PikeOS audit
- RTOS audit: ChibiOS, RT-Thread, Contiki, RODOS comparison
- PikeOS x86 audit report
- Bibliography for safety-critical hypervisor research
2026-07-15 15:32:04 +01:00

482 lines
19 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# PikeOS x86_64 Audit Report — UniversalisOS Windows Guest Path
**Date:** 2026-07-12
**Purpose:** Complete recursive audit of PikeOS 5.0 x86 kernel source, boot infrastructure,
and platform documentation to enable x86_64 hypervisor development in UniversalisOS.
---
## 1. Executive Summary
PikeOS on x86_64 is a **paravirtualized microkernel hypervisor** (not a bare-metal VMX/VMM
hypervisor). It boots via **Multiboot2/GRUB2** or **UEFI**, runs at **Ring 0** (kernel mode),
and partitions run at **Ring 3** (user mode). There is **no VT-x/VMX usage** in the kernel
source — guest isolation is achieved through **page table partitioning** (separate CR3 per
partition), **I/O port bitmap filtering** (IOPL=0 + TSS I/O bitmap), and **syscall-based IPC**.
This means **Windows guest support requires a fundamentally different approach** than what
PikeOS provides natively — PikeOS is a partitioning hypervisor, not a hardware virtualization
hypervisor. UniversalisOS would need to add **VT-x/VMX support** on top of the PikeOS
partitioning model to run unmodified Windows guests.
---
## 2. File Inventory
### 2.1 Architecture Files (arch/x86/) — 11 files
| File | Size | Purpose |
|------|------|---------|
| `aexcpt.S` | 37KB | IDT vector table + assembly exception entry/exit (256 vectors) |
| `cexcpt.c` | 49KB | C exception dispatch — page faults, GPF, breakpoint, debug, NM, DF |
| `ccontext.c` | 38KB | Context switch: P4_regs_t save/restore, FPU lazy/eager switching |
| `mmu.c` | 34KB | 4-level page tables (PML4→P3→PD→PT), map/unmap/protect, TLB flush |
| `ccontext_ext.c` | 17KB | XSAVE/XRSTOR for AVX/AVX-512, MPX, PKRU extended state |
| `cmm.c` | 15KB | Memory management: phys alloc, DMA, IOMMU integration |
| `adspace.c` | 10KB | Address space CRUD: create/destroy/switch AS (CR3 manipulation) |
| `ccopy.c` | 12KB | Safe user↔kernel memory copy with fault recovery |
| `acopy.S` | 6KB | Assembly optimized memcpy/memset with SMAP/SMEP awareness |
| `acontext.S` | 4KB | Assembly context switch stub (swapgs, iretq) |
| `ioports.c` | 7KB | I/O port access: inb/outb/inw/outw/inl/outl + TSS I/O bitmap |
### 2.2 Key Headers (arch/x86/include/)
| Header | Purpose |
|--------|---------|
| `p4const.h` | Page size (4KB), user space (0→0x7ffffffff000), kernel (0xffff800000000000), 512 interrupts |
| `p4regs.h` | P4_regs_t: 15 GPRs + RIP/RFLAGS/CS/SS + FS/GS base + FPU/SSE/AVX state (64-byte aligned) |
| `p4feature.h` | CPU feature detection (SMEP, SMAP, UMIP, PCID, FSGSBASE, XSAVE) |
| `p4kinfoarch.h` | Architecture kernel info: CR0/CR4 values, EFER, TSC frequency |
| `x86cpu.h` | CPUID wrappers, MSR read/write, CR register manipulation |
| `x86mmu.h` | PML4/P3/PD/PT entry format, PTE flags (P=1, R/W, U/S, NX, PAT, XD) |
| `x86vectors.h` | IDT vector numbers (0-255), IST stack indices |
| `syscall.h` | SYSCALL/SYSRET instruction wrappers, STAR/LSTAR/FMASK MSR setup |
| `p4regoffset.h` | Offsets into P4_regs_t for assembly access |
| `arch_per_cpu.h` | Per-CPU data: current thread, kernel stack, TSC offset, CPUID |
### 2.3 Core Kernel (src/) — 97 files
**Syscall table** (`syscall_table.S`): Maps syscall numbers to C handlers via `sys_*` functions.
**Key subsystems:**
- **Scheduler** (`sched.c`, `sched_readyq.c`, `sched_deadline.c`, `sched_timeout.c`): RMS/EDF/priority scheduling
- **IPC** (`ipc.c`, `sys_ipc.c`, `comm.c`): Sampling + queuing ports, shared memory
- **Memory** (`mm.c`, `mm_balloc.c`, `mm_kmem.c`, `mm_list.c`, `map.c`): Physical allocator, kernel heap, VM map
- **Task/Thread** (`task.c`, `task_attr.c`, `thread.c`, `thread_create.c`, `thread_attr.c`): Task lifecycle
- **Health Monitor** (`hm.c`, `hm_dump.c`, `hm_lookup.c`): 3-tier HM with action injection
- **Interrupts** (`int.c`, `sys_int.c`): IRQ routing, interrupt attachment
- **KDEV** (30+ files): Device abstraction framework (gates, providers, descriptors, I/O)
- **Trace** (`trace.c`, `spider.c`): Instrumentation and tracing
- **Time** (`time.c`, `tps.c`): Time partitioning, timer management
### 2.4 User Library (lib/) — 80+ files
- **Syscall stubs** (`lib/stubs/p4_*.S`): Assembly wrappers for every syscall (SYSCALL instruction)
- **TLS** (`lib/p4_tls_*.c`): Thread-local storage via FS segment base
- **Mutex/Cond/Sem/Barrier** (`lib/p4_mutex_*.c`, etc.): POSIX-like synchronization primitives
- **FPU control** (`lib/p4_thread_fpu_on.c`, `p4_thread_fpu_off.c`): Enable/disable FPU per thread
### 2.5 Kernel Config (.cmp files) — 10 files
| File | Purpose |
|------|---------|
| `kernel.cmp` | Main kernel component (SMP) |
| `kernel-up.cmp` | Uniprocessor kernel variant |
| `kernel-smp.cmp` | SMP-specific config |
| `kernel-cert.cmp` | Certification build config |
| `barekernel.cmp` | Bare kernel (no fusion) |
| `kerneldriver.cmp` | Kernel-space driver support |
| `psp.cmp` | Platform Support Package config |
| `kernel_tags.cmp`, `psp_tags.cmp`, `tracing_tags.cmp` | Trace tag definitions |
---
## 3. Boot Protocol
### 3.1 Multiboot2 (Primary for QEMU/GRUB)
PikeOS boots via **Multiboot2** on x86. The flow:
1. **GRUB2** loads the PikeOS ELF at `PIKEOS_START_ADDRESS`
2. PikeOS preboot code (`multiboot1.bin` for MB1, or direct for MB2) sets up:
- GDT (flat 4GB segments, 64-bit long mode)
- Page tables (PML4 for long mode)
- Stack
3. Transitions to 64-bit long mode
4. Calls kernel `main()`
**Multiboot2 ELF format** (`bs.multiboot`):
```
OUTPUT_FORMAT("elf32-i386") ; Multiboot2 header is 32-bit
ENTRY (_start)
SECTIONS { .text $PIKEOS_START_ADDRESS : { _start = .; *(.text .data) } }
```
**QEMU x86 command** (`bs.qemu`):
```
qemu-system-x86_64 -boot d -cdrom <diskimage> -m <mem> -smp <cpus>
```
PikeOS boots x86 from a **CD-ROM/ISO image** via QEMU.
### 3.2 UEFI Boot
The UEFI boot path (`bs.uefi`) creates an EFI application:
1. PikeOS kernel binary is wrapped with a preboot object (`uefi-x86_amd64.o`)
2. Linked as a shared object with a UEFI-compatible linker script
3. `objcopy` converts to `efi-app-x86-64` PE/COFF format
4. Result is a standalone `.efi` binary for UEFI boot
**This is the path to Windows guest support** — UEFI firmware is required.
### 3.3 GRUB2 Configuration
PikeOS ships a full GRUB2 distribution (`share/grub2/`) with:
- `i386-pc` modules (BIOS boot)
- `x86_64-efi` modules (UEFI boot)
- Custom GRUB configuration for PikeOS Multiboot2
**GRUB2 menu entry:**
```
menuentry "PikeOS <version>" {
multiboot2 /<output_file>
set gfxpayload=text ; or auto for EFI
boot
}
```
### 3.4 Disk Image Boot
For x86 QEMU, PikeOS creates a disk image (ISO) using `prepare_diskimage`:
- The kernel is packaged into a bootable ISO with GRUB2
- QEMU boots from the ISO via `-boot d -cdrom`
---
## 4. Memory Layout (x86_64)
```
0x0000000000000000 - 0x00007ffffffff000 User space (128 TB)
0x00007ffffffff000 - 0xffff7fffffffffff Non-canonical (guard)
0xffff800000000000 - 0xffffffffffdfffff Kernel space (PikeOS kernel + PSP)
0xffffffffffe00000 - 0xffffffffffffffff Kernel info (KINFO_BASE, 2MB)
```
**Page table structure:** 4-level (PML4 → P3 → PD → PT), 4KB pages
- PML4: 512 entries × 512GB = 256 TB
- P3: 512 entries × 1GB = 512GB
- PD: 512 entries × 2MB = 1GB
- PT: 512 entries × 4KB = 2MB
**Large pages:** 2MB (PD level) and 1GB (P3 level) supported
**PTE flags:**
- Bit 0: Present (P)
- Bit 1: Read/Write (R/W)
- Bit 2: User/Supervisor (U/S)
- Bit 3: Page-level Write-Through (PWT)
- Bit 4: Page-level Cache Disable (PCD)
- Bit 7: Page Size (PS) — 2MB/1GB large page
- Bit 8: Global (G)
- Bit 63: Execute Disable (XD/NX)
---
## 5. Context Switch (P4_regs_t)
```c
typedef struct P4_regs_str {
// GPRs (pushed by assembly on syscall/exception entry)
P4_cpureg_t rdi, rsi, rdx, r10, r8, r9, rcx, r11;
P4_cpureg_t rax, rbx, rbp, r12, r13, r14, r15;
// Exception frame (pushed by CPU + assembly)
P4_cpureg_t vector; // Exception vector number
P4_cpureg_t error; // Error code
P4_cpureg_t rip; // Instruction pointer
P4_cpureg_t cs; // Code segment
P4_cpureg_t rflags; // CPU flags
P4_cpureg_t rsp; // Stack pointer
P4_cpureg_t ss; // Stack segment
// Segment bases (for TLS)
P4_cpureg_t fs_base;
P4_cpureg_t gs_base;
// PikeOS internal
P4_cpureg_t reserved[6];
P4_cpureg_t ex_code; // Exception status/reply
P4_cpureg_t usedfpu; // FPU enable flag
// FPU/SSE/AVX state (64-byte aligned)
struct {
struct { /* FXSAVE area: x87 + SSE */ } fxsave;
struct { /* XSAVE header */ } xsave_header;
struct { /* AVX YMM registers */ } avx;
} fpu;
} P4_regs_t __attribute__((aligned(64)));
```
**Syscall entry** (via SYSCALL instruction):
- RCX → RIP (saved return address)
- R11 → RFLAGS
- RAX → syscall number
- RDI, RSI, RDX, R10, R8, R9 → arguments 1-6
---
## 6. Interrupt/Exception Handling
**IDT:** 256 vectors, 64-bit IDT entries (16 bytes each)
**Exception flow:**
1. CPU pushes SS, RSP, RFLAGS, CS, RIP (+ error code for some)
2. Assembly (`aexcpt.S`) saves all GPRs → builds P4_regs_t on stack
3. Loads kernel CR3 (page table switch for Meltdown mitigation)
4. Calls C handler (`cexcpt.c`) with vector number + P4_regs_t
5. C handler dispatches: page fault → mmu.c, GPF → panic, syscall → sys_*
**Key exceptions handled:**
- #PF (14): Page fault → memory management, demand paging
- #GP (13): General protection → I/O port violation, segment violation
- #UD (6): Undefined instruction → FPU/SSE trap (lazy FPU switching)
- #NM (7): Device not available → FPU/SSE/AVX context save/restore
- #DB (1): Debug → breakpoint handling
- #DF (8): Double fault → critical error
**Meltdown mitigation:** Kernel/user page table switching via trampoline code
(`p4x86_int_vectors_meltdown`, `p4x86_set_both_cr3_meltdown`)
---
## 7. x86-Specific Features Used
| Feature | Usage | Status |
|---------|-------|--------|
| **CR3** | Per-partition page tables | ✅ Core isolation mechanism |
| **TSS I/O Bitmap** | I/O port filtering per partition | ✅ Used for device passthrough |
| **SYSCALL/SYSRET** | Fast system call interface | ✅ Primary syscall mechanism |
| **FXSAVE/XSAVE** | FPU/SSE/AVX state save/restore | ✅ Full support |
| **PCID** | Process-context IDs for TLB | ✅ Performance optimization |
| **SMEP/SMAP** | Supervisor mode execution/access prevention | ✅ Security hardening |
| **UMIP** | User-mode instruction prevention | ✅ Security hardening |
| **FSGSBASE** | Fast FS/GS base access | ✅ TLS optimization |
| **TSC** | Time stamp counter for timing | ✅ Primary time source |
| **IOPL** | I/O privilege level (set to 0 for user) | ✅ I/O isolation |
| **VT-x/VMX** | Hardware virtualization | ❌ NOT USED |
---
## 8. Boot Infrastructure (share/boot/)
### Boot Strategies Available
| Strategy | File | Target |
|----------|------|--------|
| `bs.qemu` | QEMU boot (all arches) | x86: cdrom image; arm/aarch64: kernel |
| `bs.grub` | GRUB2 Multiboot2 | x86 (requires GRUB2) |
| `bs.multiboot` | Multiboot1/2 ELF | x86 (generic) |
| `bs.uefi` | UEFI EFI application | x86_64 (requires UEFI firmware) |
| `bs.elf` | Raw ELF boot | All arches |
| `bs.raw` | Raw binary boot | All arches |
| `bs.uboot` | U-Boot boot | ARM/PPC |
| `bs.diskimage` | Disk image (ISO) | x86 (for QEMU cdrom) |
| `bs.fastboot` | Android fastboot | ARM |
| `bs.fastmodel` | ARM Fast Model | ARM |
### QEMU x86 Specifics
```bash
# Minimal QEMU x86 command
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512
# With SMP
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512 -smp 4
# With networking (virtio)
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512 \
-device virtio-net-pci,vlan=0 -net tap,ifname=tap0
# With AHCI storage
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512 \
-device ich9-ahci,id=ahci0 \
-device ide-drive,bus=ahci0.0,drive=ahcidrive0 \
-drive file=disk.img,if=none,id=ahcidrive0,format=raw
# With USB
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512 \
-drive if=none,id=usbstick,file=usb.img \
-usb -device nec-usb-xhci,id=xhci \
-device usb-storage,bus=xhci.0,drive=usbstick,port=2
# No graphics (serial console)
qemu-system-x86_64 -nographic -fw_cfg etc/sercon-port,string=0 \
-boot d -cdrom <image.iso>
```
---
## 9. Target/x86 BSP Structure
```
target/x86/amd64/
├── apex/ APEX ARINC-653 personality configs
├── board/ Board-specific .cmp files
├── boot-images/ Boot image configs
├── cenv/ C environment configs
├── cppenv/ C++ environment configs
├── ddk-kerneldriver/ Kernel driver DDK
├── ddk-user-level/ User-level driver DDK
├── driver/ Device driver .cmp files
├── fusion-kernel/ Fusion kernel configs
├── fusion-pssw/ Fusion PSSW configs
├── fusion-volume-provider/ Volume provider configs
├── health-monitoring/ HM configs
├── include/ BSP-specific headers
├── integration/ Integration project configs
├── integration-partition/ Integration partition configs
├── integration-preconf/ Pre-configured integration
├── kernel/ Kernel build configs
├── kerneldriver/ Kernel driver configs
├── ldscript/ Linker scripts
├── lib/ BSP libraries
├── linux/ Linux personality configs
├── makeinc/ Make include files
├── network/ Network configs
├── object/ Object file configs
├── objects/ Object configs
├── partition/ Partition configs
├── pikeos/ PikeOS native personality
├── pikeos-native/ PikeOS native configs
├── posix/ POSIX personality configs
├── preboot/ Preboot objects (multiboot1.bin, uefi-x86_amd64.o)
├── psp/ Platform Support Package
├── pssw/ PSSW configs
├── scov/ Source code coverage configs
├── scov-output/ Coverage output configs
├── scripts/ Build scripts
├── share/ Shared configs
├── systemextension/ System extension configs
└── volume-provider/ Volume provider configs
```
**Total BSP files:** 2,566 files
---
## 10. What's Needed for Windows Guest Support
### 10.1 Current PikeOS Architecture (NOT sufficient for Windows)
PikeOS is a **partitioning hypervisor**, not a hardware virtualization hypervisor:
- Isolation via **page tables** (separate CR3 per partition) + **I/O port bitmap** + **syscall IPC**
- No VT-x/VMX usage — cannot run unmodified OS guests
- Guests must be **PikeOS-aware** (use PikeOS syscalls, not hardware interrupts)
- Cannot trap hardware exceptions into a guest — they go to the kernel
### 10.2 Required Additions for Windows Guest
To run Windows as a guest, UniversalisOS needs **VT-x/VMX hardware virtualization**:
| Component | Effort | Description |
|-----------|--------|-------------|
| **VMX init** | Large | Enable VT-x, set up VMCS, configure VM-exit controls |
| **VMCS management** | Large | VM-entry/exit fields, host/guest state save/restore |
| **EPT (Extended Page Tables)** | Large | Stage-2 translation: guest physical → host physical |
| **VM-exit handler** | Large | Handle CPUID, MSR, I/O, HLT, CR access, EPT violations |
| **APIC virtualization** | Large | Virtual APIC, posted interrupts, TPR virtualization |
| **I/O emulation** | Very Large | Emulate PIT, PIC, PS/2 keyboard/mouse, VGA, serial |
| **UEFI firmware** | Very Large | Embed OVMF/EDK2 as guest firmware for Windows boot |
| **PCI passthrough** | Large | VT-d/IOMMU for device assignment |
| **virtio devices** | Large | virtio-blk, virtio-net, virtio-gpu for paravirtualized I/O |
| **ACPI tables** | Medium | Generate DSDT/SSDT/FADT/MADT for Windows |
| **SMBIOS** | Small | System management BIOS tables |
### 10.3 Recommended Implementation Path
**Phase 1: x86_64 kernel boot** (2-3 months)
- Port PikeOS x86 kernel structure to UniversalisOS
- Multiboot2 boot via GRUB2
- GDT/IDT/TSS setup
- Paging (PML4→PT)
- SYSCALL/SYSRET
- Serial console (UART 16550)
**Phase 2: Partitioning base** (2-3 months)
- Per-partition page tables (CR3 switching)
- Context switch (P4_regs_t equivalent)
- I/O port bitmap (TSS I/O bitmap)
- Basic scheduler
**Phase 3: VT-x/VMX hypervisor** (4-6 months)
- VMX initialization and VMCS setup
- EPT (Extended Page Tables)
- VM-exit handling (CPUID, MSR, I/O, HLT, CR)
- APIC virtualization
- Guest boot (start in real mode, transition through protected to long mode)
**Phase 4: Device emulation** (4-6 months)
- Serial UART (16550)
- PS/2 keyboard/mouse
- VGA/Bochs VGA
- PIT/RTC timer
- PCI/PCIe configuration space
- AHCI/NVMe storage
- virtio-blk, virtio-net, virtio-gpu
**Phase 5: UEFI firmware** (6-12 months)
- Embed OVMF (open-source UEFI firmware) as guest firmware
- Or implement minimal UEFI services for Windows boot
- ACPI table generation
- SMBIOS tables
**Phase 6: Windows boot** (3-6 months)
- Windows installer boot (WinPE)
- Driver integration (virtio-win drivers for paravirtualized I/O)
- GPU passthrough or virtio-gpu
- Network (virtio-net or e1000 emulation)
**Total estimated time to Windows guest: 21-36 months**
---
## 11. Key Differences: PikeOS vs UniversalisOS x86
| Aspect | PikeOS | UniversalisOS (needed) |
|--------|--------|----------------------|
| Boot protocol | Multiboot2/UEFI | Same (replicate) |
| Isolation | Page tables + I/O bitmap | Same + VT-x/EPT |
| Guest awareness | PikeOS syscalls | Unmodified OS (VT-x traps) |
| Interrupts | Kernel handles all | VM-exit → hypervisor → inject to guest |
| Memory | Flat 4GB sections | EPT for nested translation |
| I/O | Direct port access (filtered) | Trapped and emulated |
| Timer | TSC + PIT | Virtual APIC timer + TSC offset |
| SMP | IPI via APIC | Virtual IPI via virtual APIC |
---
## 12. Documentation References
- **PSP Development Guide**: BSP creation workflow, .cmp/.bsp.dom model, linker scripts
- **x86 Platform Manual**: Memory layout, boot protocol, interrupt routing, PCI configuration
- **GCC Compiler Annex**: x86_64 cross-compilation flags, ABI conventions
- **Visual Studio Annex**: Windows host compilation (future: Windows development tooling)
- **TASKING VxToolset Annex**: Alternative compiler support
- **GHS Multi Annex**: Green Hills compiler support
---
## 13. Next Actions
1. **Write x86_64 architecture backend** (`kernel/src/arch/x86_64/`) replicating PikeOS structure
2. **Implement Multiboot2 boot** with GRUB2
3. **Set up GDT/IDT/TSS** for 64-bit long mode
4. **Implement 4-level paging** (PML4→PT)
5. **Add SYSCALL/SYSRET** support
6. **Implement context switch** (P4_regs_t equivalent)
7. **Add VT-x/VMX** initialization (for hardware virtualization)
8. **Implement VMCS** management and VM-exit handling
9. **Add EPT** (Extended Page Tables)
10. **Implement device emulation** (serial, keyboard, VGA, storage)
11. **Embed OVMF** or implement UEFI services
12. **Boot Windows** as guest