# PikeOS x86_64 Audit Report — UniversalisOS Windows Guest Path

**Date:** 2026-07-12
**Purpose:** Complete recursive audit of PikeOS 5.0 x86 kernel source, boot infrastructure,
and platform documentation to enable x86_64 hypervisor development in UniversalisOS.

---

## 1. Executive Summary

PikeOS on x86_64 is a **paravirtualized microkernel hypervisor** (not a bare-metal VMX/VMM
hypervisor). It boots via **Multiboot2/GRUB2** or **UEFI**, runs at **Ring 0** (kernel mode),
and partitions run at **Ring 3** (user mode). There is **no VT-x/VMX usage** in the kernel
source — guest isolation is achieved through **page table partitioning** (separate CR3 per
partition), **I/O port bitmap filtering** (IOPL=0 + TSS I/O bitmap), and **syscall-based IPC**.

This means **Windows guest support requires a fundamentally different approach** than what
PikeOS provides natively — PikeOS is a partitioning hypervisor, not a hardware virtualization
hypervisor. UniversalisOS would need to add **VT-x/VMX support** on top of the PikeOS
partitioning model to run unmodified Windows guests.

---

## 2. File Inventory

### 2.1 Architecture Files (arch/x86/) — 11 files

| File | Size | Purpose |
|------|------|---------|
| `aexcpt.S` | 37KB | IDT vector table + assembly exception entry/exit (256 vectors) |
| `cexcpt.c` | 49KB | C exception dispatch — page faults, GPF, breakpoint, debug, NM, DF |
| `ccontext.c` | 38KB | Context switch: P4_regs_t save/restore, FPU lazy/eager switching |
| `mmu.c` | 34KB | 4-level page tables (PML4→P3→PD→PT), map/unmap/protect, TLB flush |
| `ccontext_ext.c` | 17KB | XSAVE/XRSTOR for AVX/AVX-512, MPX, PKRU extended state |
| `cmm.c` | 15KB | Memory management: phys alloc, DMA, IOMMU integration |
| `adspace.c` | 10KB | Address space CRUD: create/destroy/switch AS (CR3 manipulation) |
| `ccopy.c` | 12KB | Safe user↔kernel memory copy with fault recovery |
| `acopy.S` | 6KB | Assembly optimized memcpy/memset with SMAP/SMEP awareness |
| `acontext.S` | 4KB | Assembly context switch stub (swapgs, iretq) |
| `ioports.c` | 7KB | I/O port access: inb/outb/inw/outw/inl/outl + TSS I/O bitmap |

### 2.2 Key Headers (arch/x86/include/)

| Header | Purpose |
|--------|---------|
| `p4const.h` | Page size (4KB), user space (0→0x7ffffffff000), kernel (0xffff800000000000), 512 interrupts |
| `p4regs.h` | P4_regs_t: 15 GPRs + RIP/RFLAGS/CS/SS + FS/GS base + FPU/SSE/AVX state (64-byte aligned) |
| `p4feature.h` | CPU feature detection (SMEP, SMAP, UMIP, PCID, FSGSBASE, XSAVE) |
| `p4kinfoarch.h` | Architecture kernel info: CR0/CR4 values, EFER, TSC frequency |
| `x86cpu.h` | CPUID wrappers, MSR read/write, CR register manipulation |
| `x86mmu.h` | PML4/P3/PD/PT entry format, PTE flags (P=1, R/W, U/S, NX, PAT, XD) |
| `x86vectors.h` | IDT vector numbers (0-255), IST stack indices |
| `syscall.h` | SYSCALL/SYSRET instruction wrappers, STAR/LSTAR/FMASK MSR setup |
| `p4regoffset.h` | Offsets into P4_regs_t for assembly access |
| `arch_per_cpu.h` | Per-CPU data: current thread, kernel stack, TSC offset, CPUID |

### 2.3 Core Kernel (src/) — 97 files

**Syscall table** (`syscall_table.S`): Maps syscall numbers to C handlers via `sys_*` functions.

**Key subsystems:**
- **Scheduler** (`sched.c`, `sched_readyq.c`, `sched_deadline.c`, `sched_timeout.c`): RMS/EDF/priority scheduling
- **IPC** (`ipc.c`, `sys_ipc.c`, `comm.c`): Sampling + queuing ports, shared memory
- **Memory** (`mm.c`, `mm_balloc.c`, `mm_kmem.c`, `mm_list.c`, `map.c`): Physical allocator, kernel heap, VM map
- **Task/Thread** (`task.c`, `task_attr.c`, `thread.c`, `thread_create.c`, `thread_attr.c`): Task lifecycle
- **Health Monitor** (`hm.c`, `hm_dump.c`, `hm_lookup.c`): 3-tier HM with action injection
- **Interrupts** (`int.c`, `sys_int.c`): IRQ routing, interrupt attachment
- **KDEV** (30+ files): Device abstraction framework (gates, providers, descriptors, I/O)
- **Trace** (`trace.c`, `spider.c`): Instrumentation and tracing
- **Time** (`time.c`, `tps.c`): Time partitioning, timer management

### 2.4 User Library (lib/) — 80+ files

- **Syscall stubs** (`lib/stubs/p4_*.S`): Assembly wrappers for every syscall (SYSCALL instruction)
- **TLS** (`lib/p4_tls_*.c`): Thread-local storage via FS segment base
- **Mutex/Cond/Sem/Barrier** (`lib/p4_mutex_*.c`, etc.): POSIX-like synchronization primitives
- **FPU control** (`lib/p4_thread_fpu_on.c`, `p4_thread_fpu_off.c`): Enable/disable FPU per thread

### 2.5 Kernel Config (.cmp files) — 10 files

| File | Purpose |
|------|---------|
| `kernel.cmp` | Main kernel component (SMP) |
| `kernel-up.cmp` | Uniprocessor kernel variant |
| `kernel-smp.cmp` | SMP-specific config |
| `kernel-cert.cmp` | Certification build config |
| `barekernel.cmp` | Bare kernel (no fusion) |
| `kerneldriver.cmp` | Kernel-space driver support |
| `psp.cmp` | Platform Support Package config |
| `kernel_tags.cmp`, `psp_tags.cmp`, `tracing_tags.cmp` | Trace tag definitions |

---

## 3. Boot Protocol

### 3.1 Multiboot2 (Primary for QEMU/GRUB)

PikeOS boots via **Multiboot2** on x86. The flow:

1. **GRUB2** loads the PikeOS ELF at `PIKEOS_START_ADDRESS`
2. PikeOS preboot code (`multiboot1.bin` for MB1, or direct for MB2) sets up:
   - GDT (flat 4GB segments, 64-bit long mode)
   - Page tables (PML4 for long mode)
   - Stack
3. Transitions to 64-bit long mode
4. Calls kernel `main()`

**Multiboot2 ELF format** (`bs.multiboot`):
```
OUTPUT_FORMAT("elf32-i386")  ; Multiboot2 header is 32-bit
ENTRY (_start)
SECTIONS { .text $PIKEOS_START_ADDRESS : { _start = .; *(.text .data) } }
```

**QEMU x86 command** (`bs.qemu`):
```
qemu-system-x86_64 -boot d -cdrom <diskimage> -m <mem> -smp <cpus>
```
PikeOS boots x86 from a **CD-ROM/ISO image** via QEMU.

### 3.2 UEFI Boot

The UEFI boot path (`bs.uefi`) creates an EFI application:

1. PikeOS kernel binary is wrapped with a preboot object (`uefi-x86_amd64.o`)
2. Linked as a shared object with a UEFI-compatible linker script
3. `objcopy` converts to `efi-app-x86-64` PE/COFF format
4. Result is a standalone `.efi` binary for UEFI boot

**This is the path to Windows guest support** — UEFI firmware is required.

### 3.3 GRUB2 Configuration

PikeOS ships a full GRUB2 distribution (`share/grub2/`) with:
- `i386-pc` modules (BIOS boot)
- `x86_64-efi` modules (UEFI boot)
- Custom GRUB configuration for PikeOS Multiboot2

**GRUB2 menu entry:**
```
menuentry "PikeOS <version>" {
    multiboot2 /<output_file>
    set gfxpayload=text  ; or auto for EFI
    boot
}
```

### 3.4 Disk Image Boot

For x86 QEMU, PikeOS creates a disk image (ISO) using `prepare_diskimage`:
- The kernel is packaged into a bootable ISO with GRUB2
- QEMU boots from the ISO via `-boot d -cdrom`

---

## 4. Memory Layout (x86_64)

```
0x0000000000000000 - 0x00007ffffffff000  User space (128 TB)
0x00007ffffffff000 - 0xffff7fffffffffff  Non-canonical (guard)
0xffff800000000000 - 0xffffffffffdfffff  Kernel space (PikeOS kernel + PSP)
0xffffffffffe00000 - 0xffffffffffffffff  Kernel info (KINFO_BASE, 2MB)
```

**Page table structure:** 4-level (PML4 → P3 → PD → PT), 4KB pages
- PML4: 512 entries × 512GB = 256 TB
- P3: 512 entries × 1GB = 512GB  
- PD: 512 entries × 2MB = 1GB
- PT: 512 entries × 4KB = 2MB

**Large pages:** 2MB (PD level) and 1GB (P3 level) supported

**PTE flags:**
- Bit 0: Present (P)
- Bit 1: Read/Write (R/W)
- Bit 2: User/Supervisor (U/S)
- Bit 3: Page-level Write-Through (PWT)
- Bit 4: Page-level Cache Disable (PCD)
- Bit 7: Page Size (PS) — 2MB/1GB large page
- Bit 8: Global (G)
- Bit 63: Execute Disable (XD/NX)

---

## 5. Context Switch (P4_regs_t)

```c
typedef struct P4_regs_str {
    // GPRs (pushed by assembly on syscall/exception entry)
    P4_cpureg_t rdi, rsi, rdx, r10, r8, r9, rcx, r11;
    P4_cpureg_t rax, rbx, rbp, r12, r13, r14, r15;
    
    // Exception frame (pushed by CPU + assembly)
    P4_cpureg_t vector;     // Exception vector number
    P4_cpureg_t error;      // Error code
    P4_cpureg_t rip;        // Instruction pointer
    P4_cpureg_t cs;         // Code segment
    P4_cpureg_t rflags;     // CPU flags
    P4_cpureg_t rsp;        // Stack pointer
    P4_cpureg_t ss;         // Stack segment
    
    // Segment bases (for TLS)
    P4_cpureg_t fs_base;
    P4_cpureg_t gs_base;
    
    // PikeOS internal
    P4_cpureg_t reserved[6];
    P4_cpureg_t ex_code;    // Exception status/reply
    P4_cpureg_t usedfpu;    // FPU enable flag
    
    // FPU/SSE/AVX state (64-byte aligned)
    struct {
        struct { /* FXSAVE area: x87 + SSE */ } fxsave;
        struct { /* XSAVE header */ } xsave_header;
        struct { /* AVX YMM registers */ } avx;
    } fpu;
} P4_regs_t __attribute__((aligned(64)));
```

**Syscall entry** (via SYSCALL instruction):
- RCX → RIP (saved return address)
- R11 → RFLAGS
- RAX → syscall number
- RDI, RSI, RDX, R10, R8, R9 → arguments 1-6

---

## 6. Interrupt/Exception Handling

**IDT:** 256 vectors, 64-bit IDT entries (16 bytes each)

**Exception flow:**
1. CPU pushes SS, RSP, RFLAGS, CS, RIP (+ error code for some)
2. Assembly (`aexcpt.S`) saves all GPRs → builds P4_regs_t on stack
3. Loads kernel CR3 (page table switch for Meltdown mitigation)
4. Calls C handler (`cexcpt.c`) with vector number + P4_regs_t
5. C handler dispatches: page fault → mmu.c, GPF → panic, syscall → sys_*

**Key exceptions handled:**
- #PF (14): Page fault → memory management, demand paging
- #GP (13): General protection → I/O port violation, segment violation
- #UD (6): Undefined instruction → FPU/SSE trap (lazy FPU switching)
- #NM (7): Device not available → FPU/SSE/AVX context save/restore
- #DB (1): Debug → breakpoint handling
- #DF (8): Double fault → critical error

**Meltdown mitigation:** Kernel/user page table switching via trampoline code
(`p4x86_int_vectors_meltdown`, `p4x86_set_both_cr3_meltdown`)

---

## 7. x86-Specific Features Used

| Feature | Usage | Status |
|---------|-------|--------|
| **CR3** | Per-partition page tables | ✅ Core isolation mechanism |
| **TSS I/O Bitmap** | I/O port filtering per partition | ✅ Used for device passthrough |
| **SYSCALL/SYSRET** | Fast system call interface | ✅ Primary syscall mechanism |
| **FXSAVE/XSAVE** | FPU/SSE/AVX state save/restore | ✅ Full support |
| **PCID** | Process-context IDs for TLB | ✅ Performance optimization |
| **SMEP/SMAP** | Supervisor mode execution/access prevention | ✅ Security hardening |
| **UMIP** | User-mode instruction prevention | ✅ Security hardening |
| **FSGSBASE** | Fast FS/GS base access | ✅ TLS optimization |
| **TSC** | Time stamp counter for timing | ✅ Primary time source |
| **IOPL** | I/O privilege level (set to 0 for user) | ✅ I/O isolation |
| **VT-x/VMX** | Hardware virtualization | ❌ NOT USED |

---

## 8. Boot Infrastructure (share/boot/)

### Boot Strategies Available

| Strategy | File | Target |
|----------|------|--------|
| `bs.qemu` | QEMU boot (all arches) | x86: cdrom image; arm/aarch64: kernel |
| `bs.grub` | GRUB2 Multiboot2 | x86 (requires GRUB2) |
| `bs.multiboot` | Multiboot1/2 ELF | x86 (generic) |
| `bs.uefi` | UEFI EFI application | x86_64 (requires UEFI firmware) |
| `bs.elf` | Raw ELF boot | All arches |
| `bs.raw` | Raw binary boot | All arches |
| `bs.uboot` | U-Boot boot | ARM/PPC |
| `bs.diskimage` | Disk image (ISO) | x86 (for QEMU cdrom) |
| `bs.fastboot` | Android fastboot | ARM |
| `bs.fastmodel` | ARM Fast Model | ARM |

### QEMU x86 Specifics

```bash
# Minimal QEMU x86 command
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512

# With SMP
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512 -smp 4

# With networking (virtio)
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512 \
    -device virtio-net-pci,vlan=0 -net tap,ifname=tap0

# With AHCI storage
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512 \
    -device ich9-ahci,id=ahci0 \
    -device ide-drive,bus=ahci0.0,drive=ahcidrive0 \
    -drive file=disk.img,if=none,id=ahcidrive0,format=raw

# With USB
qemu-system-x86_64 -boot d -cdrom <image.iso> -m 512 \
    -drive if=none,id=usbstick,file=usb.img \
    -usb -device nec-usb-xhci,id=xhci \
    -device usb-storage,bus=xhci.0,drive=usbstick,port=2

# No graphics (serial console)
qemu-system-x86_64 -nographic -fw_cfg etc/sercon-port,string=0 \
    -boot d -cdrom <image.iso>
```

---

## 9. Target/x86 BSP Structure

```
target/x86/amd64/
├── apex/              APEX ARINC-653 personality configs
├── board/             Board-specific .cmp files
├── boot-images/       Boot image configs
├── cenv/              C environment configs
├── cppenv/            C++ environment configs
├── ddk-kerneldriver/  Kernel driver DDK
├── ddk-user-level/    User-level driver DDK
├── driver/            Device driver .cmp files
├── fusion-kernel/     Fusion kernel configs
├── fusion-pssw/       Fusion PSSW configs
├── fusion-volume-provider/  Volume provider configs
├── health-monitoring/ HM configs
├── include/           BSP-specific headers
├── integration/       Integration project configs
├── integration-partition/  Integration partition configs
├── integration-preconf/    Pre-configured integration
├── kernel/            Kernel build configs
├── kerneldriver/      Kernel driver configs
├── ldscript/          Linker scripts
├── lib/               BSP libraries
├── linux/             Linux personality configs
├── makeinc/           Make include files
├── network/           Network configs
├── object/            Object file configs
├── objects/           Object configs
├── partition/         Partition configs
├── pikeos/            PikeOS native personality
├── pikeos-native/     PikeOS native configs
├── posix/             POSIX personality configs
├── preboot/           Preboot objects (multiboot1.bin, uefi-x86_amd64.o)
├── psp/               Platform Support Package
├── pssw/              PSSW configs
├── scov/              Source code coverage configs
├── scov-output/       Coverage output configs
├── scripts/           Build scripts
├── share/             Shared configs
├── systemextension/   System extension configs
└── volume-provider/   Volume provider configs
```

**Total BSP files:** 2,566 files

---

## 10. What's Needed for Windows Guest Support

### 10.1 Current PikeOS Architecture (NOT sufficient for Windows)

PikeOS is a **partitioning hypervisor**, not a hardware virtualization hypervisor:
- Isolation via **page tables** (separate CR3 per partition) + **I/O port bitmap** + **syscall IPC**
- No VT-x/VMX usage — cannot run unmodified OS guests
- Guests must be **PikeOS-aware** (use PikeOS syscalls, not hardware interrupts)
- Cannot trap hardware exceptions into a guest — they go to the kernel

### 10.2 Required Additions for Windows Guest

To run Windows as a guest, UniversalisOS needs **VT-x/VMX hardware virtualization**:

| Component | Effort | Description |
|-----------|--------|-------------|
| **VMX init** | Large | Enable VT-x, set up VMCS, configure VM-exit controls |
| **VMCS management** | Large | VM-entry/exit fields, host/guest state save/restore |
| **EPT (Extended Page Tables)** | Large | Stage-2 translation: guest physical → host physical |
| **VM-exit handler** | Large | Handle CPUID, MSR, I/O, HLT, CR access, EPT violations |
| **APIC virtualization** | Large | Virtual APIC, posted interrupts, TPR virtualization |
| **I/O emulation** | Very Large | Emulate PIT, PIC, PS/2 keyboard/mouse, VGA, serial |
| **UEFI firmware** | Very Large | Embed OVMF/EDK2 as guest firmware for Windows boot |
| **PCI passthrough** | Large | VT-d/IOMMU for device assignment |
| **virtio devices** | Large | virtio-blk, virtio-net, virtio-gpu for paravirtualized I/O |
| **ACPI tables** | Medium | Generate DSDT/SSDT/FADT/MADT for Windows |
| **SMBIOS** | Small | System management BIOS tables |

### 10.3 Recommended Implementation Path

**Phase 1: x86_64 kernel boot** (2-3 months)
- Port PikeOS x86 kernel structure to UniversalisOS
- Multiboot2 boot via GRUB2
- GDT/IDT/TSS setup
- Paging (PML4→PT)
- SYSCALL/SYSRET
- Serial console (UART 16550)

**Phase 2: Partitioning base** (2-3 months)
- Per-partition page tables (CR3 switching)
- Context switch (P4_regs_t equivalent)
- I/O port bitmap (TSS I/O bitmap)
- Basic scheduler

**Phase 3: VT-x/VMX hypervisor** (4-6 months)
- VMX initialization and VMCS setup
- EPT (Extended Page Tables)
- VM-exit handling (CPUID, MSR, I/O, HLT, CR)
- APIC virtualization
- Guest boot (start in real mode, transition through protected to long mode)

**Phase 4: Device emulation** (4-6 months)
- Serial UART (16550)
- PS/2 keyboard/mouse
- VGA/Bochs VGA
- PIT/RTC timer
- PCI/PCIe configuration space
- AHCI/NVMe storage
- virtio-blk, virtio-net, virtio-gpu

**Phase 5: UEFI firmware** (6-12 months)
- Embed OVMF (open-source UEFI firmware) as guest firmware
- Or implement minimal UEFI services for Windows boot
- ACPI table generation
- SMBIOS tables

**Phase 6: Windows boot** (3-6 months)
- Windows installer boot (WinPE)
- Driver integration (virtio-win drivers for paravirtualized I/O)
- GPU passthrough or virtio-gpu
- Network (virtio-net or e1000 emulation)

**Total estimated time to Windows guest: 21-36 months**

---

## 11. Key Differences: PikeOS vs UniversalisOS x86

| Aspect | PikeOS | UniversalisOS (needed) |
|--------|--------|----------------------|
| Boot protocol | Multiboot2/UEFI | Same (replicate) |
| Isolation | Page tables + I/O bitmap | Same + VT-x/EPT |
| Guest awareness | PikeOS syscalls | Unmodified OS (VT-x traps) |
| Interrupts | Kernel handles all | VM-exit → hypervisor → inject to guest |
| Memory | Flat 4GB sections | EPT for nested translation |
| I/O | Direct port access (filtered) | Trapped and emulated |
| Timer | TSC + PIT | Virtual APIC timer + TSC offset |
| SMP | IPI via APIC | Virtual IPI via virtual APIC |

---

## 12. Documentation References

- **PSP Development Guide**: BSP creation workflow, .cmp/.bsp.dom model, linker scripts
- **x86 Platform Manual**: Memory layout, boot protocol, interrupt routing, PCI configuration
- **GCC Compiler Annex**: x86_64 cross-compilation flags, ABI conventions
- **Visual Studio Annex**: Windows host compilation (future: Windows development tooling)
- **TASKING VxToolset Annex**: Alternative compiler support
- **GHS Multi Annex**: Green Hills compiler support

---

## 13. Next Actions

1. **Write x86_64 architecture backend** (`kernel/src/arch/x86_64/`) replicating PikeOS structure
2. **Implement Multiboot2 boot** with GRUB2
3. **Set up GDT/IDT/TSS** for 64-bit long mode
4. **Implement 4-level paging** (PML4→PT)
5. **Add SYSCALL/SYSRET** support
6. **Implement context switch** (P4_regs_t equivalent)
7. **Add VT-x/VMX** initialization (for hardware virtualization)
8. **Implement VMCS** management and VM-exit handling
9. **Add EPT** (Extended Page Tables)
10. **Implement device emulation** (serial, keyboard, VGA, storage)
11. **Embed OVMF** or implement UEFI services
12. **Boot Windows** as guest