portugalfuturista/universalisos
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
universalisos/docs/PIKEOS_X86_AUDIT.md
Fábio Coutada 47f5885da6 docs: add RTOS audit, bibliography, PikeOS audit 
- RTOS audit: ChibiOS, RT-Thread, Contiki, RODOS comparison
- PikeOS x86 audit report
- Bibliography for safety-critical hypervisor research
2026-07-15 15:32:04 +01:00

19 KiB
Raw Blame History

PikeOS x86_64 Audit Report — UniversalisOS Windows Guest Path

Date: 2026-07-12 Purpose: Complete recursive audit of PikeOS 5.0 x86 kernel source, boot infrastructure, and platform documentation to enable x86_64 hypervisor development in UniversalisOS.

1. Executive Summary

PikeOS on x86_64 is a paravirtualized microkernel hypervisor (not a bare-metal VMX/VMM hypervisor). It boots via Multiboot2/GRUB2 or UEFI, runs at Ring 0 (kernel mode), and partitions run at Ring 3 (user mode). There is no VT-x/VMX usage in the kernel source — guest isolation is achieved through page table partitioning (separate CR3 per partition), I/O port bitmap filtering (IOPL=0 + TSS I/O bitmap), and syscall-based IPC.

This means Windows guest support requires a fundamentally different approach than what PikeOS provides natively — PikeOS is a partitioning hypervisor, not a hardware virtualization hypervisor. UniversalisOS would need to add VT-x/VMX support on top of the PikeOS partitioning model to run unmodified Windows guests.

2. File Inventory

2.1 Architecture Files (arch/x86/) — 11 files

File Size Purpose
aexcpt.S 37KB IDT vector table + assembly exception entry/exit (256 vectors)
cexcpt.c 49KB C exception dispatch — page faults, GPF, breakpoint, debug, NM, DF
ccontext.c 38KB Context switch: P4_regs_t save/restore, FPU lazy/eager switching
mmu.c 34KB 4-level page tables (PML4→P3→PD→PT), map/unmap/protect, TLB flush
ccontext_ext.c 17KB XSAVE/XRSTOR for AVX/AVX-512, MPX, PKRU extended state
cmm.c 15KB Memory management: phys alloc, DMA, IOMMU integration
adspace.c 10KB Address space CRUD: create/destroy/switch AS (CR3 manipulation)
ccopy.c 12KB Safe user↔kernel memory copy with fault recovery
acopy.S 6KB Assembly optimized memcpy/memset with SMAP/SMEP awareness
acontext.S 4KB Assembly context switch stub (swapgs, iretq)
ioports.c 7KB I/O port access: inb/outb/inw/outw/inl/outl + TSS I/O bitmap

2.2 Key Headers (arch/x86/include/)

Header Purpose
p4const.h Page size (4KB), user space (0→0x7ffffffff000), kernel (0xffff800000000000), 512 interrupts
p4regs.h P4_regs_t: 15 GPRs + RIP/RFLAGS/CS/SS + FS/GS base + FPU/SSE/AVX state (64-byte aligned)
p4feature.h CPU feature detection (SMEP, SMAP, UMIP, PCID, FSGSBASE, XSAVE)
p4kinfoarch.h Architecture kernel info: CR0/CR4 values, EFER, TSC frequency
x86cpu.h CPUID wrappers, MSR read/write, CR register manipulation
x86mmu.h PML4/P3/PD/PT entry format, PTE flags (P=1, R/W, U/S, NX, PAT, XD)
x86vectors.h IDT vector numbers (0-255), IST stack indices
syscall.h SYSCALL/SYSRET instruction wrappers, STAR/LSTAR/FMASK MSR setup
p4regoffset.h Offsets into P4_regs_t for assembly access
arch_per_cpu.h Per-CPU data: current thread, kernel stack, TSC offset, CPUID

2.3 Core Kernel (src/) — 97 files

Syscall table (syscall_table.S): Maps syscall numbers to C handlers via sys_* functions.

Key subsystems:

  • Scheduler (sched.c, sched_readyq.c, sched_deadline.c, sched_timeout.c): RMS/EDF/priority scheduling
  • IPC (ipc.c, sys_ipc.c, comm.c): Sampling + queuing ports, shared memory
  • Memory (mm.c, mm_balloc.c, mm_kmem.c, mm_list.c, map.c): Physical allocator, kernel heap, VM map
  • Task/Thread (task.c, task_attr.c, thread.c, thread_create.c, thread_attr.c): Task lifecycle
  • Health Monitor (hm.c, hm_dump.c, hm_lookup.c): 3-tier HM with action injection
  • Interrupts (int.c, sys_int.c): IRQ routing, interrupt attachment
  • KDEV (30+ files): Device abstraction framework (gates, providers, descriptors, I/O)
  • Trace (trace.c, spider.c): Instrumentation and tracing
  • Time (time.c, tps.c): Time partitioning, timer management

2.4 User Library (lib/) — 80+ files

  • Syscall stubs (lib/stubs/p4_*.S): Assembly wrappers for every syscall (SYSCALL instruction)
  • TLS (lib/p4_tls_*.c): Thread-local storage via FS segment base
  • Mutex/Cond/Sem/Barrier (lib/p4_mutex_*.c, etc.): POSIX-like synchronization primitives
  • FPU control (lib/p4_thread_fpu_on.c, p4_thread_fpu_off.c): Enable/disable FPU per thread

2.5 Kernel Config (.cmp files) — 10 files

File Purpose
kernel.cmp Main kernel component (SMP)
kernel-up.cmp Uniprocessor kernel variant
kernel-smp.cmp SMP-specific config
kernel-cert.cmp Certification build config
barekernel.cmp Bare kernel (no fusion)
kerneldriver.cmp Kernel-space driver support
psp.cmp Platform Support Package config
kernel_tags.cmp, psp_tags.cmp, tracing_tags.cmp Trace tag definitions

3. Boot Protocol

3.1 Multiboot2 (Primary for QEMU/GRUB)

PikeOS boots via Multiboot2 on x86. The flow:

  1. GRUB2 loads the PikeOS ELF at PIKEOS_START_ADDRESS
  2. PikeOS preboot code (multiboot1.bin for MB1, or direct for MB2) sets up:
    • GDT (flat 4GB segments, 64-bit long mode)
    • Page tables (PML4 for long mode)
    • Stack
  3. Transitions to 64-bit long mode
  4. Calls kernel main()

Multiboot2 ELF format (bs.multiboot):

OUTPUT_FORMAT("elf32-i386")  ; Multiboot2 header is 32-bit
ENTRY (_start)
SECTIONS { .text $PIKEOS_START_ADDRESS : { _start = .; *(.text .data) } }

QEMU x86 command (bs.qemu):

qemu-system-x86_64 -boot d -cdrom <diskimage> -m <mem> -smp <cpus>

PikeOS boots x86 from a CD-ROM/ISO image via QEMU.

3.2 UEFI Boot

The UEFI boot path (bs.uefi) creates an EFI application:

  1. PikeOS kernel binary is wrapped with a preboot object (uefi-x86_amd64.o)
  2. Linked as a shared object with a UEFI-compatible linker script
  3. objcopy converts to efi-app-x86-64 PE/COFF format
  4. Result is a standalone .efi binary for UEFI boot

This is the path to Windows guest support — UEFI firmware is required.

3.3 GRUB2 Configuration

PikeOS ships a full GRUB2 distribution (share/grub2/) with:

  • i386-pc modules (BIOS boot)
  • x86_64-efi modules (UEFI boot)
  • Custom GRUB configuration for PikeOS Multiboot2

GRUB2 menu entry:

menuentry "PikeOS <version>" {
    multiboot2 /<output_file>
    set gfxpayload=text  ; or auto for EFI
    boot
}

3.4 Disk Image Boot

For x86 QEMU, PikeOS creates a disk image (ISO) using prepare_diskimage:

  • The kernel is packaged into a bootable ISO with GRUB2
  • QEMU boots from the ISO via -boot d -cdrom

4. Memory Layout (x86_64)

0x0000000000000000 - 0x00007ffffffff000  User space (128 TB)
0x00007ffffffff000 - 0xffff7fffffffffff  Non-canonical (guard)
0xffff800000000000 - 0xffffffffffdfffff  Kernel space (PikeOS kernel + PSP)
0xffffffffffe00000 - 0xffffffffffffffff  Kernel info (KINFO_BASE, 2MB)

Page table structure: 4-level (PML4 → P3 → PD → PT), 4KB pages

  • PML4: 512 entries × 512GB = 256 TB
  • P3: 512 entries × 1GB = 512GB
  • PD: 512 entries × 2MB = 1GB
  • PT: 512 entries × 4KB = 2MB

Large pages: 2MB (PD level) and 1GB (P3 level) supported

PTE flags:

  • Bit 0: Present (P)
  • Bit 1: Read/Write (R/W)
  • Bit 2: User/Supervisor (U/S)
  • Bit 3: Page-level Write-Through (PWT)
  • Bit 4: Page-level Cache Disable (PCD)
  • Bit 7: Page Size (PS) — 2MB/1GB large page
  • Bit 8: Global (G)
  • Bit 63: Execute Disable (XD/NX)

5. Context Switch (P4_regs_t)

typedef struct P4_regs_str {
    // GPRs (pushed by assembly on syscall/exception entry)
    P4_cpureg_t rdi, rsi, rdx, r10, r8, r9, rcx, r11;
    P4_cpureg_t rax, rbx, rbp, r12, r13, r14, r15;
    
    // Exception frame (pushed by CPU + assembly)
    P4_cpureg_t vector;     // Exception vector number
    P4_cpureg_t error;      // Error code
    P4_cpureg_t rip;        // Instruction pointer
    P4_cpureg_t cs;         // Code segment
    P4_cpureg_t rflags;     // CPU flags
    P4_cpureg_t rsp;        // Stack pointer
    P4_cpureg_t ss;         // Stack segment
    
    // Segment bases (for TLS)
    P4_cpureg_t fs_base;
    P4_cpureg_t gs_base;
    
    // PikeOS internal
    P4_cpureg_t reserved[6];
    P4_cpureg_t ex_code;    // Exception status/reply
    P4_cpureg_t usedfpu;    // FPU enable flag
    
    // FPU/SSE/AVX state (64-byte aligned)
    struct {
        struct { /* FXSAVE area: x87 + SSE */ } fxsave;
        struct { /* XSAVE header */ } xsave_header;
        struct { /* AVX YMM registers */ } avx;
    } fpu;
} P4_regs_t __attribute__((aligned(64)));

Syscall entry (via SYSCALL instruction):

  • RCX → RIP (saved return address)
  • R11 → RFLAGS
  • RAX → syscall number
  • RDI, RSI, RDX, R10, R8, R9 → arguments 1-6

6. Interrupt/Exception Handling

IDT: 256 vectors, 64-bit IDT entries (16 bytes each)

Exception flow:

  1. CPU pushes SS, RSP, RFLAGS, CS, RIP (+ error code for some)
  2. Assembly (aexcpt.S) saves all GPRs → builds P4_regs_t on stack
  3. Loads kernel CR3 (page table switch for Meltdown mitigation)
  4. Calls C handler (cexcpt.c) with vector number + P4_regs_t
  5. C handler dispatches: page fault → mmu.c, GPF → panic, syscall → sys_*

Key exceptions handled:

  • #PF (14): Page fault → memory management, demand paging
  • #GP (13): General protection → I/O port violation, segment violation
  • #UD (6): Undefined instruction → FPU/SSE trap (lazy FPU switching)
  • #NM (7): Device not available → FPU/SSE/AVX context save/restore
  • #DB (1): Debug → breakpoint handling
  • #DF (8): Double fault → critical error

Meltdown mitigation: Kernel/user page table switching via trampoline code (p4x86_int_vectors_meltdown, p4x86_set_both_cr3_meltdown)

7. x86-Specific Features Used

Feature Usage Status
CR3 Per-partition page tables ✅ Core isolation mechanism
TSS I/O Bitmap I/O port filtering per partition ✅ Used for device passthrough
SYSCALL/SYSRET Fast system call interface ✅ Primary syscall mechanism
FXSAVE/XSAVE FPU/SSE/AVX state save/restore ✅ Full support
PCID Process-context IDs for TLB ✅ Performance optimization
SMEP/SMAP Supervisor mode execution/access prevention ✅ Security hardening
UMIP User-mode instruction prevention ✅ Security hardening
FSGSBASE Fast FS/GS base access ✅ TLS optimization
TSC Time stamp counter for timing ✅ Primary time source
IOPL I/O privilege level (set to 0 for user) ✅ I/O isolation
VT-x/VMX Hardware virtualization ❌ NOT USED

8. Boot Infrastructure (share/boot/)

Boot Strategies Available

Strategy File Target
bs.qemu QEMU boot (all arches) x86: cdrom image; arm/aarch64: kernel
bs.grub GRUB2 Multiboot2 x86 (requires GRUB2)
bs.multiboot Multiboot1/2 ELF x86 (generic)
bs.uefi UEFI EFI application x86_64 (requires UEFI firmware)
bs.elf Raw ELF boot All arches
bs.raw Raw binary boot All arches
bs.uboot U-Boot boot ARM/PPC
bs.diskimage Disk image (ISO) x86 (for QEMU cdrom)
bs.fastboot Android fastboot ARM
bs.fastmodel ARM Fast Model ARM

QEMU x86 Specifics

# Minimal QEMU x86 command
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512

# With SMP
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512 -smp 4

# With networking (virtio)
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512 \
    -device virtio-net-pci,vlan=0 -net tap,ifname=tap0

# With AHCI storage
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512 \
    -device ich9-ahci,id=ahci0 \
    -device ide-drive,bus=ahci0.0,drive=ahcidrive0 \
    -drive file=disk.img,if=none,id=ahcidrive0,format=raw

# With USB
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512 \
    -drive if=none,id=usbstick,file=usb.img \
    -usb -device nec-usb-xhci,id=xhci \
    -device usb-storage,bus=xhci.0,drive=usbstick,port=2

# No graphics (serial console)
qemu-system-x86_64 -nographic -fw_cfg etc/sercon-port,string=0 \
    -boot d -cdrom <image.iso>

9. Target/x86 BSP Structure

target/x86/amd64/
├── apex/              APEX ARINC-653 personality configs
├── board/             Board-specific .cmp files
├── boot-images/       Boot image configs
├── cenv/              C environment configs
├── cppenv/            C++ environment configs
├── ddk-kerneldriver/  Kernel driver DDK
├── ddk-user-level/    User-level driver DDK
├── driver/            Device driver .cmp files
├── fusion-kernel/     Fusion kernel configs
├── fusion-pssw/       Fusion PSSW configs
├── fusion-volume-provider/  Volume provider configs
├── health-monitoring/ HM configs
├── include/           BSP-specific headers
├── integration/       Integration project configs
├── integration-partition/  Integration partition configs
├── integration-preconf/    Pre-configured integration
├── kernel/            Kernel build configs
├── kerneldriver/      Kernel driver configs
├── ldscript/          Linker scripts
├── lib/               BSP libraries
├── linux/             Linux personality configs
├── makeinc/           Make include files
├── network/           Network configs
├── object/            Object file configs
├── objects/           Object configs
├── partition/         Partition configs
├── pikeos/            PikeOS native personality
├── pikeos-native/     PikeOS native configs
├── posix/             POSIX personality configs
├── preboot/           Preboot objects (multiboot1.bin, uefi-x86_amd64.o)
├── psp/               Platform Support Package
├── pssw/              PSSW configs
├── scov/              Source code coverage configs
├── scov-output/       Coverage output configs
├── scripts/           Build scripts
├── share/             Shared configs
├── systemextension/   System extension configs
└── volume-provider/   Volume provider configs

Total BSP files: 2,566 files

10. What's Needed for Windows Guest Support

10.1 Current PikeOS Architecture (NOT sufficient for Windows)

PikeOS is a partitioning hypervisor, not a hardware virtualization hypervisor:

  • Isolation via page tables (separate CR3 per partition) + I/O port bitmap + syscall IPC
  • No VT-x/VMX usage — cannot run unmodified OS guests
  • Guests must be PikeOS-aware (use PikeOS syscalls, not hardware interrupts)
  • Cannot trap hardware exceptions into a guest — they go to the kernel

10.2 Required Additions for Windows Guest

To run Windows as a guest, UniversalisOS needs VT-x/VMX hardware virtualization:

Component Effort Description
VMX init Large Enable VT-x, set up VMCS, configure VM-exit controls
VMCS management Large VM-entry/exit fields, host/guest state save/restore
EPT (Extended Page Tables) Large Stage-2 translation: guest physical → host physical
VM-exit handler Large Handle CPUID, MSR, I/O, HLT, CR access, EPT violations
APIC virtualization Large Virtual APIC, posted interrupts, TPR virtualization
I/O emulation Very Large Emulate PIT, PIC, PS/2 keyboard/mouse, VGA, serial
UEFI firmware Very Large Embed OVMF/EDK2 as guest firmware for Windows boot
PCI passthrough Large VT-d/IOMMU for device assignment
virtio devices Large virtio-blk, virtio-net, virtio-gpu for paravirtualized I/O
ACPI tables Medium Generate DSDT/SSDT/FADT/MADT for Windows
SMBIOS Small System management BIOS tables

10.3 Recommended Implementation Path

Phase 1: x86_64 kernel boot (2-3 months)

  • Port PikeOS x86 kernel structure to UniversalisOS
  • Multiboot2 boot via GRUB2
  • GDT/IDT/TSS setup
  • Paging (PML4→PT)
  • SYSCALL/SYSRET
  • Serial console (UART 16550)

Phase 2: Partitioning base (2-3 months)

  • Per-partition page tables (CR3 switching)
  • Context switch (P4_regs_t equivalent)
  • I/O port bitmap (TSS I/O bitmap)
  • Basic scheduler

Phase 3: VT-x/VMX hypervisor (4-6 months)

  • VMX initialization and VMCS setup
  • EPT (Extended Page Tables)
  • VM-exit handling (CPUID, MSR, I/O, HLT, CR)
  • APIC virtualization
  • Guest boot (start in real mode, transition through protected to long mode)

Phase 4: Device emulation (4-6 months)

  • Serial UART (16550)
  • PS/2 keyboard/mouse
  • VGA/Bochs VGA
  • PIT/RTC timer
  • PCI/PCIe configuration space
  • AHCI/NVMe storage
  • virtio-blk, virtio-net, virtio-gpu

Phase 5: UEFI firmware (6-12 months)

  • Embed OVMF (open-source UEFI firmware) as guest firmware
  • Or implement minimal UEFI services for Windows boot
  • ACPI table generation
  • SMBIOS tables

Phase 6: Windows boot (3-6 months)

  • Windows installer boot (WinPE)
  • Driver integration (virtio-win drivers for paravirtualized I/O)
  • GPU passthrough or virtio-gpu
  • Network (virtio-net or e1000 emulation)

Total estimated time to Windows guest: 21-36 months

11. Key Differences: PikeOS vs UniversalisOS x86

Aspect PikeOS UniversalisOS (needed)
Boot protocol Multiboot2/UEFI Same (replicate)
Isolation Page tables + I/O bitmap Same + VT-x/EPT
Guest awareness PikeOS syscalls Unmodified OS (VT-x traps)
Interrupts Kernel handles all VM-exit → hypervisor → inject to guest
Memory Flat 4GB sections EPT for nested translation
I/O Direct port access (filtered) Trapped and emulated
Timer TSC + PIT Virtual APIC timer + TSC offset
SMP IPI via APIC Virtual IPI via virtual APIC

12. Documentation References

  • PSP Development Guide: BSP creation workflow, .cmp/.bsp.dom model, linker scripts
  • x86 Platform Manual: Memory layout, boot protocol, interrupt routing, PCI configuration
  • GCC Compiler Annex: x86_64 cross-compilation flags, ABI conventions
  • Visual Studio Annex: Windows host compilation (future: Windows development tooling)
  • TASKING VxToolset Annex: Alternative compiler support
  • GHS Multi Annex: Green Hills compiler support

13. Next Actions

  1. Write x86_64 architecture backend (kernel/src/arch/x86_64/) replicating PikeOS structure
  2. Implement Multiboot2 boot with GRUB2
  3. Set up GDT/IDT/TSS for 64-bit long mode
  4. Implement 4-level paging (PML4→PT)
  5. Add SYSCALL/SYSRET support
  6. Implement context switch (P4_regs_t equivalent)
  7. Add VT-x/VMX initialization (for hardware virtualization)
  8. Implement VMCS management and VM-exit handling
  9. Add EPT (Extended Page Tables)
  10. Implement device emulation (serial, keyboard, VGA, storage)
  11. Embed OVMF or implement UEFI services
  12. Boot Windows as guest