Fábio Coutada 47f5885da6 docs: add RTOS audit, bibliography, PikeOS audit

- RTOS audit: ChibiOS, RT-Thread, Contiki, RODOS comparison
- PikeOS x86 audit report
- Bibliography for safety-critical hypervisor research

2026-07-15 15:32:04 +01:00

37 KiB

Raw Blame History

RTOS Reference Deep Audit: seL4, ThreadX, NuttX, RTEMS

Purpose: Extract architecture abstraction patterns, no-MMU handling, kernel primitives, context switch, and isolation mechanisms to inform UniversalisOS microkernel design.

1. seL4

1.1 Architecture Support

Architectures: ARM (32/64), RISC-V (32/64), x86 (32/64)

seL4 arch	Directory	Sub-variants
ARM 32	`src/arch/arm/32/`	armv7-a, armv8-a/32
ARM 64	`src/arch/arm/64/`	armv8-a/64 (AArch64)
RISC-V	`src/arch/riscv/`	riscv32, riscv64
x86 32	`src/arch/x86/32/`	ia32
x86 64	`src/arch/x86/64/`	x86_64

Platforms: allwinnerA20, am335x, apq8064, ariane, bcm2711, bcm2712, bcm2837, cheshire, eswin, exynos4/5, fvp, hifive, hikey, imx6/7/8m/93, maaxboard, odroidc2/c4, omap3, pc99, qemu-arm-virt, qemu-riscv-virt, rk3399, spike, tk1, tx1, tx2, zynqmp, xilinx-versal, zcu102

Key files:

src/arch/arm/32/traps.S — ARM32 vector table + trap handlers
src/arch/arm/64/traps.S — AArch64 vector table (EL1/EL2)
src/arch/riscv/traps.S — RISC-V trap entry
src/arch/x86/32/traps.S / src/arch/x86/64/traps.S — x86 IDT-based traps
libsel4/sel4_arch_include/ — per-arch API headers: aarch32, aarch64, ia32, riscv32, riscv64, x86_64

Critical observation: seL4 does NOT support ARMv-M (Cortex-M). It requires an MMU. No MPU-only support.

1.2 Memory Model

MMU-only. seL4 is a capability-based microkernel that REQUIRES an MMU for its fundamental isolation model.

include/kernel/vspace.h — architecture-abstracted virtual space management
src/arch/arm/32/kernel/vspace.c — ARM32 page table manipulation (L1/L2)
src/arch/arm/64/kernel/vspace.c — AArch64 translation tables
Capabilities: cap_page_table_cap, cap_page_directory_cap, cap_frame_cap — all MMU-dependent

How isolation works: Capability-based access control. Each object (TCB, Endpoint, Page Table, etc.) is accessed only through capabilities held in CNodes. No process can access kernel memory or another process's memory without the appropriate capability. The MMU enforces address space separation between threads in different PDs.

No-MMU strategy: None. seL4 fundamentally requires MMU. There is no MPU path.

1.3 Kernel Primitives

Kernel Objects (src/object/):

Object	File	Purpose
TCB	`tcb.c`	Thread Control Block — the schedulable entity
Endpoint	`endpoint.c`	Synchronous IPC rendezvous point
Notification	`notification.c`	Async signal / event flag
CNode	`cnode.c`	Capability table node
Untyped	`untyped.c`	Raw memory for retype into kernel objects
Reply	`reply.c`	MCS scheduling reply object
SchedContext	`schedcontext.c`	MCS budget/timeslice
SchedControl	`schedcontrol.c`	CPU scheduling control
Domain	`domain.c`	Scheduling domain (ARINC-like)

Scheduling (include/kernel/thread.h):

Bitmap priority scheduler with L1/L2 bitmaps: getHighestPrio() uses clzl
Up to CONFIG_NUM_PRIORITIES priority levels
Domain support: multiple scheduling domains with time-slicing
MCS kernel: CBS (Constant Bandwidth Server) / sporadic server scheduling
Key functions: schedule(), chooseThread(), switchToThread(), switchToIdleThread()

IPC (src/object/endpoint.c):

Synchronous rendezvous: sendIPC() / recvIPC()
Message passing via IPC buffer in user space (registers: msgInfoRegister, capRegister)
Badge-based capability transfer over endpoints
Syscall dispatch: SysSend, SysNBSend, SysCall, SysRecv, SysReply, SysReplyRecv, SysWait, SysNBWait
Fastpath for Call/ReplyRecv in src/fastpath/fastpath.c

Synchronization:

Notifications: binary event-like primitive (sendSignal(), wait(), tryWait())
No mutexes/semaphores — IPC IS the synchronization mechanism

Memory management:

All memory created from Untyped objects via seL4_Untyped_Retype()
Capabilities control access to all objects including page tables and frames
Kernel does not allocate memory after boot — all created from initial untypeds

1.4 API Surface

Public API defined in libsel4/:

include/interfaces/sel4.xml — formal IDL defining all system calls
include/interfaces/object-api.xml — object method API

System calls (from src/api/syscall.c):

exception_t handleSyscall(syscall_t syscall);
// SysCall, SysSend, SysNBSend, SysRecv, SysReply, SysReplyRecv, SysWait, SysNBWait

API functions (from libsel4/sel4_arch_include/*/sel4/sel4_arch/):

seL4_Untyped_Retype() — create new kernel objects
seL4_TCB_* — thread control (Configure, SetPriority, SetIPCBuffer, WriteRegisters, ReadRegisters, Suspend, Resume)
seL4_Endpoint_Send/Recv/Call() — synchronous IPC
seL4_Signal/Wait() on Notification objects
seL4_CNode_* — capability space manipulation
seL4_VSpace_* — virtual address space management
seL4_IRQControl/Handler_* — interrupt management
seL4_SchedControl_* — MCS scheduling control

1.5 Hardware Abstraction

Three-layer architecture:

arch/ — architecture-specific code: ARM, RISC-V, x86
machine/ — machine-level (common within arch): registerset.h, fpu.c, hardware.h
plat/ — platform-specific: per-SoC timer, IRQ controller, serial

Key HAL interfaces (include/arch/machine.h):

void init_cpu(void);
void init_drivers(void);        // platform-level
void ackInterrupt(irq_t irq);
irq_t getActiveIRQ(void);
bool_t isIRQPending(void);
void setNextPC(tcb_t *tcb, word_t v);
word_t getRestartPC(tcb_t *tcb);
void switchToThread(tcb_t *tcb);  // architecture-level
void Arch_switchToThread(tcb_t *tcb);  // arch-specific (setVMRoot + clearExMonitor)

Register abstraction (include/machine/registerset.h → per-arch):

enum _register { R0, ..., R14, SP=13, LR=14, NextIP=15, CPSR=16, FaultIP=17, TPIDRURW=18, TPIDRURO=19, n_contextRegisters=20 };

Pattern: Arch_*() prefix for architecture-specific functions; arch_* directories per arch; mode/ subdirectories for 32/64-bit variants.

1.6 Partition/Isolation Mechanisms

Capability-based: All access mediated by capabilities in CNodes. No global names.
Address space isolation: Each thread can be in a different VSpace (page directory). setVMRoot() switches page tables.
Scheduling domains: CONFIG_NUM_DOMAINS > 1 enables ARINC-653-like temporal partitioning
MCS scheduling: Budget-based isolation prevents starvation/budget-exhaustion attacks
No no-MMU path: seL4 REQUIRES MMU. The capability system IS the isolation mechanism.

1.7 Context Switch

ARM32 context (src/arch/arm/32/traps.S):

arm_swi_syscall:
    srsia #PMODE_SUPERVISOR          @ Save CPSR + LR to SVC stack
    sub lr, lr, #4                   @ FaultIP = NextIP - 4
    str lr, [sp, #(PT_FaultIP - PT_NextIP)]
    stmdb sp, {r0-lr}^              @ Save all user regs (r0-r14)
    mrc p15, 0, sp, c13, c0, 4     @ Load kernel stack from TPIDRPRW

Saved state: 20 words per thread — R0-R14, NextIP, CPSR, FaultIP, TPIDRURW, TPIDRURO

FPU state saved lazily (optional CONFIG_HAVE_FPU)
Arch_switchToThread(): setVMRoot(tcb) + clearExMonitor()
No explicit register save/restore in switch — the trap handler saves everything on entry, the scheduler just swaps the kernel stack pointer

AArch64 context (src/arch/arm/64/traps.S):

@ Vector table with 128-byte aligned entries per ARM D1-7
@ Uses TPIDR_EL1/EL2 for kernel stack pointer
@ Saves: X0-X30, SPSR, ELR, TPIDR to kernel stack

Fastpath (src/fastpath/fastpath.c):

void NORETURN fastpath_call(word_t cptr, word_t msgInfo);
void NORETURN fastpath_reply_recv(word_t cptr, word_t msgInfo);

Optimized path that avoids full save/restore when only message registers change. Directly switches TCB and address space.

2. ThreadX (Eclipse ThreadX)

2.1 Architecture Support

Broadest port coverage of any RTOS in the audit.

ARM Cortex-A: cortex_a5, a7, a8, a9, a12, a15, a17, a34, a35, a53, a55, a57, a65, a72, a73, a75, a76, a77, a5x, a65ae, a76ae ARM Cortex-R: cortex_r4, r5, r7 ARM Cortex-M: cortex_m0, m23, m3, m4, m7, m33, m55, m85 ARM legacy: arm9, arm11 RISC-V: risc-v32, risc-v64, risc-v_common ARC: arc_em, arc_hs Renesas RX: rxv1, rxv2, rxv3 TI DSP: c667x Xtensa: xtensa Linux user-space: linux (gnu) Windows: win32, win64

Architecture-grouped ports (ports_arch/):

ARMv7-A, ARMv7-M, ARMv8-A, ARMv8-M

Key files per port:

ports/cortex_m0/gnu/src/tx_thread_context_save.S
ports/cortex_m0/gnu/src/tx_thread_context_restore.S
ports/cortex_m0/gnu/src/tx_thread_schedule.S
ports/cortex_m0/gnu/src/tx_thread_system_return.S
ports/cortex_m0/gnu/src/tx_thread_stack_build.S
ports/cortex_m0/gnu/inc/tx_port.h — port-specific type definitions + inline optimizations

2.2 Memory Model

No MMU/MPU required. ThreadX runs on flat memory model by default.

No-MMU (Cortex-M0/M3/M4): Single address space, no protection between threads
With MPU (Cortex-M33/M85): Optional MPU support via ThreadX Modules (separate product)
With MMU (Cortex-A): Flat model or optional virtual memory

Isolation without MMU: ThreadX provides NONE by default. All threads share the same address space. The tx_thread_system_state variable tracks ISR nesting but doesn't protect memory.

Memory management:

tx_byte_pool — variable-size memory allocator (malloc-like)
tx_block_pool — fixed-size block allocator (pool-based)
Both are user-space objects, not kernel-managed pages

2.3 Kernel Primitives

Threads (common/src/tx_thread_*.c):

tx_thread_create(), tx_thread_delete(), tx_thread_suspend(), tx_thread_resume()
Priority-based preemptive scheduling (0 = highest, up to TX_MAX_PRIORITIES = 32-1024)
Round-robin time-slicing within same priority
Preemption-threshold: disable preemption for priorities below threshold

Synchronization:

tx_mutex_create/get/put/delete — mutex with priority inheritance
tx_semaphore_create/get/put/delete — counting semaphore
tx_event_flags_create/get/set/delete — event flags (AND/OR)

Communication:

tx_queue_create/send/receive/delete — fixed-size message queues
No IPC/message passing between address spaces (single address space)

Timers:

tx_timer_create/activate/deactivate/delete — software timers
Tick-based: tx_timer_interrupt increments system tick

Memory:

tx_byte_pool_create/allocate/release/delete — dynamic memory pools
tx_block_pool_create/allocate/release/delete — fixed-block pools

2.4 API Surface

Header: common/inc/tx_api.h — single monolithic API header

Function naming: All public functions prefixed tx_:

UINT tx_thread_create(TX_THREAD *thread_ptr, CHAR *name_ptr, VOID (*entry)(ULONG), ULONG entry_input,
                      VOID *stack_start, ULONG stack_size, UINT priority, UINT preempt_threshold,
                      ULONG time_slice, UINT auto_start);
UINT tx_thread_delete(TX_THREAD *thread_ptr);
UINT tx_thread_suspend(TX_THREAD *thread_ptr);
UINT tx_thread_resume(TX_THREAD *thread_ptr);
UINT tx_thread_relinquish(void);
UINT tx_thread_sleep(ULONG timer_ticks);

UINT tx_mutex_create(TX_MUTEX *mutex_ptr, CHAR *name_ptr, UINT inherit);
UINT tx_mutex_get(TX_MUTEX *mutex_ptr, ULONG wait_option);
UINT tx_mutex_put(TX_MUTEX *mutex_ptr);

UINT tx_semaphore_create(TX_SEMAPHORE *sem_ptr, CHAR *name_ptr, ULONG initial_count);
UINT tx_semaphore_get(TX_SEMAPHORE *sem_ptr, ULONG wait_option);
UINT tx_semaphore_put(TX_SEMAPHORE *sem_ptr);

UINT tx_queue_create(TX_QUEUE *queue_ptr, CHAR *name_ptr, UINT message_size, VOID *queue_start, ULONG queue_size);
UINT tx_queue_send(TX_QUEUE *queue_ptr, VOID *source_ptr, ULONG wait_option);
UINT tx_queue_receive(TX_QUEUE *queue_ptr, VOID *destination_ptr, ULONG wait_option);

Error-checked variants: _txe_* prefix (with parameter validation) MISRA variants: _txr_* prefix

2.5 Hardware Abstraction

Port-based abstraction: Each target gets a complete port directory:

ports/<cpu>/<compiler>/
    inc/tx_port.h       — type definitions, macros, inline optimizations
    src/
        tx_thread_context_save.S
        tx_thread_context_restore.S
        tx_thread_schedule.S
        tx_thread_system_return.S
        tx_thread_stack_build.S
        tx_thread_interrupt_control.S
        tx_thread_interrupt_disable.S
        tx_thread_interrupt_restore.S
        tx_timer_interrupt.S

tx_port.h pattern (Cortex-M0 example):

#define TX_INT_DISABLE    1
#define TX_INT_ENABLE     0
#define TX_MINIMUM_STACK  200

// Inline interrupt control:
static inline unsigned int __disable_interrupts(void) { ... MRS PRIMASK; CPSID i ... }
static inline void __restore_interrupts(unsigned int primask_value) { ... MSR PRIMASK ... }

// Inline system return (PendSV-based):
static inline void _tx_thread_system_return_inline(void) {
    *((volatile ULONG *) 0xE000ED04) = ((ULONG) 0x10000000);  // Set PendSV
    // DSB + ISB
}

#define TX_DISABLE   interrupt_save = __disable_interrupts();
#define TX_RESTORE   __restore_interrupts(interrupt_save);

Key pattern: The C kernel code (common/src/) is 100% portable. All arch-specific behavior is in the port .S files and tx_port.h. The common code calls _tx_thread_context_save(), _tx_thread_schedule(), _tx_thread_system_return() which are entirely implemented in the port assembly.

2.6 Partition/Isolation Mechanisms

None in base ThreadX. Single flat address space, no protection.

ThreadX Modules (ports_module/): Optional module that provides memory-isolated modules using MPU/MMU when available. This is a separate add-on, not part of core ThreadX.

2.7 Context Switch

Cortex-M0 (ports/cortex_m0/gnu/src/tx_thread_schedule.S):

_tx_thread_schedule:
    MOVS    r0, #0
    LDR     r2, =_tx_thread_preempt_disable
    STR     r0, [r2, #0]              @ Clear preempt disable
    CPSIE   i                          @ Enable interrupts
    LDR     r0, =#0x10000000           @ PENDSVSET bit
    LDR     r1, =#0xE000ED04           @ NVIC ICSR
    STR     r0, [r1]                   @ Trigger PendSV
    ...PendSV handler does the actual switch...

Cortex-M PendSV context switch (in tx_thread_context_restore.S):

_tx_thread_context_restore:
    @ PendSV handler - hardware saves R0-R3, R12, LR, PC, xPSR automatically
    @ Save remaining: R4-R11, optionally S16-S31 (FPU)
    @ Load new thread's R4-R11 from its stack
    @ Hardware restores R0-R3, R12, LR, PC, xPSR on exception return

Cortex-A (ports/cortex_a9/gnu/src/tx_thread_context_save.S):

@ Saves R0-R12, LR, CPSR, SPSR to thread's stack
@ IRQ/FIQ nesting support: tx_thread_irq_nesting_start/end

Cortex-A scheduler (tx_thread_schedule.S):

@ Load _tx_thread_execute_ptr, compare with current
@ If different: save full context (R0-R12, SP, LR, CPSR) to old TCB stack
@              restore from new TCB stack
@ Handle FPU context (VFP D0-D31, FPEXC, FPSCR)

Stack building (tx_thread_stack_build.S):

@ Builds initial stack frame for new thread:
@ Pushes: CPSR, PC (entry), LR (thread shell), R12, R3-R0 (entry_input)
@ Sets SP to top of stack

3. NuttX

3.1 Architecture Support

Most architectures of any RTOS in the audit. NuttX is a POSIX-like RTOS.

Architectures (arch/):

Arch	Directory	Sub-variants
ARM	`arch/arm/`	arm, armv6-m, armv7-a, armv7-m, armv7-r, armv8-m, armv8-r
ARM64	`arch/arm64/`	AArch64
AVR	`arch/avr/`	ATmega
CEVA	`arch/ceva/`	DSP
HC	`arch/hc/`	HCS12
MIPS	`arch/mips/`	PIC32, MIPS32
MISC	`arch/misoc/`	MiSoC
OpenRISC	`arch/or1k/`	or1k
Renesas	`arch/renesas/`	SH-1, M16C, RX
RISC-V	`arch/risc-v/`	RV32, RV64
Simulator	`arch/sim/`	Linux/macOS user-space
SPARC	`arch/sparc/`	LEON
TriCore	`arch/tricore/`	Infineon
x86	`arch/x86/`	i486, QEMU
x86_64	`arch/x86_64/`	Intel
Xtensa	`arch/xtensa/`	ESP32
Z16	`arch/z16/`	Zilog
Z80	`arch/z80/`	Z80, eZ80, Z180

ARM sub-architectures in arch/arm/src/:

armv6-m (Cortex-M0/M0+)
armv7-a (Cortex-A5/A7/A8/A9/A53 with MMU)
armv7-m (Cortex-M3/M4/M7 — no MMU, optional MPU)
armv7-r (Cortex-R4/R5/R7 — MPU)
armv8-m (Cortex-M23/M33/M55/M85 — TrustZone-M + MPU)
armv8-r (Cortex-R52/R82)

Board support: Hundreds of BSPs under boards/ — STM32, NRF52, RP2040, ESP32, SAM, i.MX, Kinetis, etc.

3.2 Memory Model

Adaptive — supports no-MMU, MPU, and MMU.

No-MMU (Cortex-M, most MCUs):

Flat address space, all tasks in single memory map
CONFIG_ARCH_NO_INTERRUPT_STACK — uses thread stack for ISR
Protection: stack canary checking (CONFIG_STACK_CANARIES)
No memory isolation between tasks

MPU (Cortex-M with MPU, Cortex-R):

CONFIG_ARCH_USE_MPU — enables MPU support
CONFIG_BUILD_PROTECTED — kernel/user separation using MPU
- Kernel runs privileged, user runs unprivileged
- syscall/ directory: system call table (syscall.csv) with auto-generated proxies/stubs
- arch/arm/src/armv7-m/arm_dispatch_syscall.S — SVC-based syscall entry
CONFIG_MM_KERNEL_HEAP — separate kernel heap

MMU (Cortex-A, x86, etc.):

CONFIG_BUILD_KERNEL — full virtual memory isolation
arch/arm/src/armv7-a/ — full MMU page table management
sched/addrenv/ — address environment management
mm/kmap/ — kernel memory mapping

Memory management (mm/):

mm_heap/ — general heap allocator
mm/tlsf/ — TLSF (Two-Level Segregated Fit) allocator
mm/mempool/ — fixed-size memory pool
mm/mm_gran/ — granule allocator (for DMA, etc.)
mm/umm_heap/ — user memory heap
mm/kmm_heap/ — kernel memory heap
mm/shm/ — shared memory

3.3 Kernel Primitives

Tasks (sched/task/):

task_create(), task_spawn(), task_delete(), task_exit()
POSIX-like: nxtask_create() internal, task_create() public
Priority-based preemptive scheduling (FIFO, Round-Robin, Sporadic)

Pthreads (sched/pthread/):

Full POSIX threads: pthread_create(), pthread_join(), pthread_detach()
pthread_mutex_*, pthread_cond_*, pthread_rwlock_*
pthread_attr_* — full attribute support

Semaphores (sched/semaphore/):

nxsem_wait(), nxsem_post(), nxsem_trywait()
Binary and counting semaphores

Message queues (sched/mqueue/):

POSIX mqueue: mq_open(), mq_send(), mq_receive()

Signals (sched/signal/):

Full POSIX signals: kill(), sigaction(), sigwait(), pthread_sigmask()

Events (sched/event/):

NuttX-specific event flags

Scheduling (sched/sched/):

sched_addreadytorun() — core scheduling logic
nxsched_process_timer() — tick processing
SMP support: per-CPU ready queues
CONFIG_SCHED_INSTRUMENTATION — trace hooks

3.4 API Surface

POSIX-compliant API. The public API IS POSIX:

// Task management
int task_create(const char *name, int priority, int stack_size, main_t entry, char *const argv[]);
int task_delete(pid_t pid);
int task_setpriority(pid_t pid, int sched_priority);

// POSIX threads
int pthread_create(pthread_t *thread, const pthread_attr_t *attr, pthread_startroutine_t start_routine, void *arg);
int pthread_join(pthread_t thread, void **value_ptr);
int pthread_mutex_init(pthread_mutex_t *mutex, const pthread_mutexattr_t *attr);
int pthread_mutex_lock(pthread_mutex_t *mutex);

// Semaphores
int sem_init(sem_t *sem, int pshared, unsigned int value);
int sem_wait(sem_t *sem);
int sem_post(sem_t *sem);

// POSIX I/O
int open(const char *path, int oflags, ...);
ssize_t read(int fd, void *buf, size_t nbytes);
ssize_t write(int fd, const void *buf, size_t nbytes);

// Sockets
int socket(int domain, int type, int protocol);
int bind(int sockfd, const struct sockaddr *addr, socklen_t addrlen);

System call table (syscall/syscall.csv):

"task_create","nuttx/task.h","","int","FAR const char *","int","int","main_t","FAR char * const *"
"task_delete","unistd.h","","int","pid_t"
"pthread_create","pthread.h","","int","FAR pthread_t *","FAR const pthread_attr_t *","FAR pthread_startroutine_t","FAR void *"

Auto-generated syscall mechanism: The CSV file drives code generation for:

User-space proxies (in syscall/proxies/)
Kernel-space stubs (in syscall/stubs/)
SVC/SWI dispatch (in arch/*/arm_dispatch_syscall.S)

3.5 Hardware Abstraction

Three-layer architecture:

arch/<arch>/ — architecture-level
- src/common/ — shared across sub-architectures (e.g., arm_dataabort.c, arm_vectors.S)
- src/<subarch>/ — sub-architecture (e.g., armv7-m/, armv7-a/)
- src/<chip>/ — chip-specific (e.g., stm32/, nrf52/)
- include/ — headers per chip/subarch
boards/<arch>/<chip>/<board>/ — board-level configuration and drivers
drivers/ — portable device drivers (serial, SPI, I2C, GPIO, etc.)

Key HAL functions (include/nuttx/arch.h):

void up_initialize(void);          // Architecture init
int  up_saveusercontext(void *saveregs);  // Save CPU context
void up_initial_state(struct tcb_s *tcb); // Initialize new task context
void up_switch_context(struct tcb_s *tcb, struct tcb_s *rtcb); // Context switch
void up_irqinitialize(void);       // IRQ controller init
int  up_enable_irq(int irq);       // Enable interrupt
int  up_disable_irq(int irq);      // Disable interrupt

IRQ abstraction (include/nuttx/irq.h):

int irq_attach(int irq, xcpt_t isr, xcpt_t isrthread, FAR void *arg, FAR const char *name);
// Supports: ISR handlers, threaded IRQ handlers, work-queue-based IRQ handlers

Architecture-specific context (ARM, arch/arm/include/armv7-m/irq.h):

struct xcptcontext {
    uint32_t regs[XCPTCONTEXT_REGS]; // R0-R15, xPSR, BASEPRI, EXC_RETURN
    // FPU: S0-S31, FPSCR
    // Signal context overlay
};

3.6 Partition/Isolation Mechanisms

CONFIG_BUILD_PROTECTED — NuttX's primary no-MMU isolation mechanism:

Kernel runs in privileged mode (Handler mode or privileged Thread mode)
User tasks run unprivileged (Thread mode, PSP stack)
System calls via SVC instruction
MPU regions protect kernel memory from user access
Separate kernel/user heaps

CONFIG_BUILD_KERNEL — Full MMU isolation:

Per-process address spaces
sched/addrenv/ — address environment switching on context switch
mm/kmap/ — kernel virtual memory management

Key files:

arch/arm/src/armv7-m/arm_dispatch_syscall.S — SVC handler for protected mode
syscall/ — auto-generated syscall proxy/stub pairs
mm/umm_heap/ vs mm/kmm_heap/ — separate user/kernel heaps

3.7 Context Switch

ARMv7-M context save (arch/arm/src/armv7-m/arm_saveusercontext.S):

up_saveusercontext:
    str  r0, [r0, #(4*REG_R0)]    @ Save R0-R3, R12, R14, R15
    str  r1, [r0, #(4*REG_R1)]
    ...
    mrs  r1, XPSR
    str  r1, [r0, #(4*REG_XPSR)]
    @ FPU: vstmia r1!, {s0-s15}; vmrs fpscr
    mov  r2, sp
    mrs  r3, basepri
    stmia r0!, {r2-r11}           @ SP, BASEPRI, R4-R11
    mov  r1, #-1
    stmia r0!, {r1}               @ EXC_RETURN = 0xffffffff
    @ FPU: vstmia r0!, {s16-s31}

ARMv7-M exception entry (arch/arm/src/armv7-m/arm_exception.S):

exception_common:
    mrs  r0, ipsr                 @ IRQ number
    mrs  r12, control
    tst  r14, #EXC_RETURN_PROCESS_STACK  @ PSP or MSP?
    beq  1f
    mrs  r1, psp                  @ Context on PSP
    b    2f
1:  mrs  r1, msp                  @ Context on MSP
    sub  r2, r1, #SW_XCPT_SIZE
    msr  msp, r2
2:  @ Complete save: R2-R12, R14, BASEPRI, SP
    @ FPU: vstmdbeq r1!, {s16-s31}
    stmdb r1!, {r2-r12,r14}
    @ Call arm_doirq(irq_number, saved_context)

Context switch: On Cortex-M, NuttX uses the hardware PendSV mechanism:

up_switch_context() triggers PendSV
PendSV handler saves R4-R11 (hardware saves R0-R3, R12, LR, PC, xPSR)
Loads new thread's registers
Hardware restores on exception return

4. RTEMS

4.1 Architecture Support

14 CPU architectures (cpukit/score/cpu/):

Architecture	Directory	BSP Variants
AArch64	`score/cpu/aarch64/`	a53, a72, raspberrypi5, rk3399, xilinx-zynqmp, xen, frdm-imx93, xilinx-versal
ARM	`score/cpu/arm/`	beagle, stm32f4/h7/u5, atsam, lpc, imx, imxrt, raspberrypi, tms570, xen, fvp, efm32, lpc176x
i386	`score/cpu/i386/`	pc386, pc486
x86_64	`score/cpu/x86_64/`	amd64
M68k	`score/cpu/m68k/`	mcf5206, mcf52235, mcf5225x, mcf5329, mrm332
MicroBlaze	`score/cpu/microblaze/`	Xilinx
MIPS	`score/cpu/mips/`	Malta, JMR3904, RBtx4938
Moxie	`score/cpu/moxie/`	moxiesim
Nios II	`score/cpu/nios2/`	Altera
OpenRISC 1000	`score/cpu/or1k/`	or1ksim
PowerPC	`score/cpu/powerpc/`	MPC5xx, MPC8xx, MPC8260, MPC83xx, MPC85xx, QorIQ, mvme3100, beatnik
RISC-V	`score/cpu/riscv/`	rv32/rv64, generic, spike, frdm-k28f
SPARC	`score/cpu/sparc/`	ERC32, LEON2, LEON3, LEON4
no_cpu	`score/cpu/no_cpu/`	Template/porting guide

4.2 Memory Model

Adaptive — supports no-MMU, MPU, and MMU.

No-MMU (ARM Cortex-M, SPARC ERC32):

Flat address space
Stack-based protection (stack bounds checking via guard patterns)

MPU (ARMv7-M/R):

score/cpu/arm/include/rtems/score/armv7-pmsa.h — full PMSAv7 MPU support
Region-based protection with _ARMV7_PMSA_Write_region(), _ARMV7_PMSA_Add_regions()
Supports up to 16 MPU regions (or more with sub-region disable)
Access control: read-only/read-write, cached/uncached, shared/non-shared

MMU (AArch64, PowerPC, x86_64):

Full virtual memory support via BSP-specific MMU drivers
score/cpu/aarch64/ — EL1 page table management

Memory management:

Workspace allocator (kernel heap) — configured at link time
cpukit/libcsupport/src/ — POSIX malloc/free
cpukit/libblock/ — block device cache

4.3 Kernel Primitives

Classic RTEMS API (cpukit/include/rtems/rtems/):

Manager	Header	Functions
Tasks	`tasks.h`	`rtems_task_create/delete/start/restart/suspend/resume/wake_when/set_priority`
Semaphores	`sem.h`	`rtems_semaphore_create/delete/obtain/release/release_count`
Message Queues	`message.h`	`rtems_message_queue_create/delete/send/receive/broadcast/urgent`
Events	`event.h`	`rtems_event_send/receive`
Barriers	`barrier.h`	`rtems_barrier_create/delete/wait/release`
Partitions	`part.h`	`rtems_partition_create/delete/get/release`
Regions	`region.h`	`rtems_region_create/delete/get/return_segment`
Timers	`timer.h`	`rtems_timer_create/delete/server_fire_when/fire_after/reset/cancel`
Rate Monotonic	`ratemon.h`	`rtems_rate_monotonic/create/delete/period/cancel/get_status`
Dual-Port Memory	`dpmem.h`	`rtems_port_create/delete/external2internal/internal2external`
Signals	`signal.h`	`rtems_signal_send/catch`
Clock	`clock.h`	`rtems_clock_set/get/get_tod/get_seconds_since_epoch/tick`

POSIX API (via cpukit/posix/):

Full POSIX threads, mutexes, condition variables, semaphores, mqueues, timers

Super Core (score) (cpukit/score/src/):

corebarrier.c, coremsg.c, coremutexseize.c, coresem.c — internal implementations
thread*.c — thread management
scheduler*.c — pluggable scheduler framework
smp.c — SMP management

4.4 API Surface

Classic RTEMS API:

rtems_task_create(name, initial_priority, stack_size, modes, attributes, &id);
rtems_task_start(id, entry_point, argument);
rtems_task_suspend(id);
rtems_task_resume(id);

rtems_semaphore_create(name, count, attributes, priority, &id);
rtems_semaphore_obtain(id, options, timeout);
rtems_semaphore_release(id);

rtems_message_queue_create(name, count, max_size, attributes, &id);
rtems_message_queue_send(id, buffer, size);
rtems_message_queue_receive(id, buffer, &size, options, timeout);

rtems_event_send(task_id, event_in);
rtems_event_receive(event_in, options, ticks, &event_out);

POSIX API:

int pthread_create(pthread_t *thread, const pthread_attr_t *attr, void *(*start_routine)(void *), void *arg);
int sem_init(sem_t *sem, int pshared, unsigned int value);
int mq_send(mqd_t mqdes, const char *msg_ptr, size_t msg_len, unsigned msg_prio);

4.5 Hardware Abstraction

Score CPU layer (cpukit/score/cpu/<arch>/):

include/rtems/score/cpu.h         — CPU-level type definitions, macros, inline functions
include/rtems/score/cpuimpl.h     — implementation details
include/rtems/score/cpu_asm.h     — assembly prototypes

Key CPU interface (from cpu.h):

#define CPU_STACK_MINIMUM_SIZE       (1024 * 4)
#define CPU_SIZEOF_POINTER           4
#define CPU_STRUCTURE_ALIGNMENT      RTEMS_ALIGNED(32)

void _CPU_Context_switch(Context_Control *run, Context_Control *heir);
void _CPU_Context_restore(Context_Control *new_context);
void _CPU_Context_Initialize(Context_Control *context, void *stack_area_begin,
                              size_t stack_area_size, uint32_t new_level,
                              void (*entry_point)(void), bool is_fp, void *tls_area);

BSP layer (bsps/<arch>/<bsp>/):

include/bsp.h            — BSP configuration
start/                   — startup code
console/                 — UART driver
clock/                   — timer driver
irq/                     — interrupt controller

Pattern: CPU layer provides _CPU_*() functions. BSP provides _BSP_*() functions. The score (super core) is arch-independent and calls only _CPU_*() and _BSP_*().

4.6 Partition/Isolation Mechanisms

MPU-based protection:

score/cpu/arm/include/rtems/score/armv7-pmsa.h — comprehensive MPU management
_ARMV7_PMSA_Write_region() — write MPU region (base, size, attributes, sub-region disable)
_ARMV7_PMSA_Add_regions() — add memory region to MPU (handles power-of-2 alignment)
_ARMV7_PMSA_Find_region() — search for region containing address
_ARMV7_PMSA_Find_available_region() — find free MPU slot
Supports data and instruction regions separately (DRBAR/IRBAR)
Region attributes: cached/uncached, read-only/read-write, shareable, device

ARINC 653 partitioning:

RTEMS has an ARINC 653 API layer for safety-critical partitioning
Rate Monotonic scheduling provides temporal isolation

4.7 Context Switch

ARMv7-M context switch (score/cpu/arm/armv7m-context-switch.c):

void __attribute__((naked)) _CPU_Context_switch(
  Context_Control *executing,
  Context_Control *heir
) {
    __asm__ volatile (
        "movw r2, #:lower16:_Per_CPU_Information\n"
        "movt r2, #:upper16:_Per_CPU_Information\n"
        "ldr r3, [r2, %[isrpcpuoff]]\n"
        "stm r0, {r4-r11, lr}\n"       // Save callee-saved regs
#ifdef ARM_MULTILIB_VFP
        "add r4, r0, %[d8off]\n"
        "vstm r4, {d8-d15}\n"          // Save FPU D8-D15
#endif
        "str sp, [r0, %[spctxoff]]\n"  // Save SP
        "str r3, [r0, %[isrctxoff]]\n" // Save ISR nest level
        "ldr r3, [r1, %[isrctxoff]]\n" // Load new ISR nest level
        "ldr sp, [r1, %[spctxoff]]\n"  // Load new SP
#ifdef ARM_MULTILIB_VFP
        "add r4, r1, %[d8off]\n"
        "vldm r4, {d8-d15}\n"          // Restore FPU D8-D15
#endif
        "ldm r1, {r4-r11, lr}\n"       // Restore callee-saved regs
        "str r3, [r2, %[isrpcpuoff]]\n"// Store ISR nest level
        "bx lr\n"
    );
}

Context_Control structure (ARM, from cpu.h):

typedef struct {
    uint32_t register_r4;
    uint32_t register_r5;
    uint32_t register_r6;
    uint32_t register_r7;
    uint32_t register_r8;
    uint32_t register_r9;
    uint32_t register_r10;
    uint32_t register_r11;
    uint32_t register_lr;
    uint32_t register_sp;
    uint32_t isr_nest_level;
    uint32_t thread_id;          // TLS
#ifdef ARM_MULTILIB_VFP
    uint64_t register_d8;
    ...
    uint64_t register_d15;
#endif
} Context_Control;

Key pattern: RTEMS uses naked functions with inline assembly, directly manipulating the SP. Unlike ThreadX (which uses PendSV on Cortex-M), RTEMS performs the context switch in the calling function itself. The isr_nest_level is stored per-context to handle nested interrupts correctly.

5. Cross-RTOS Comparison Matrix

Feature	seL4	ThreadX	NuttX	RTEMS
Arch count	3 (ARM, RISC-V, x86)	10+ (ARM-M/A/R, RISC-V, ARC, RX, Xtensa, Win, Linux)	18 (ARM, AVR, MIPS, RISC-V, x86, SPARC, Z80...)	14 (ARM, AArch64, x86, PPC, MIPS, SPARC, M68k, RISC-V...)
Cortex-M support	NO	YES (M0 to M85)	YES (M0 to M85)	YES (M3/M4/M7/M33)
No-MMU support	NO	YES (flat)	YES (flat + MPU + protected)	YES (flat + MPU)
MPU support	NO	Modules (optional)	CONFIG_BUILD_PROTECTED	armv7-pmsa.h
MMU support	REQUIRED	Optional	CONFIG_BUILD_KERNEL	Per-arch
API style	Capability IPC	tx_*() proprietary	POSIX	Classic RTEMS + POSIX
Scheduling	Priority bitmap + MCS	Priority + preemption-threshold	Priority FIFO/RR/Sporadic	Pluggable scheduler
IPC	Synchronous endpoints	Message queues	POSIX mqueue/signals	Message queues/events
Context switch	Trap handler saves all	PendSV (Cortex-M) / IRQ	PendSV (Cortex-M) / IRQ	Direct (naked function)
Isolation model	Capabilities	None (flat)	MPU protected / MMU	MPU regions
Naming convention	`seL4_*`	`tx_` / `_tx_`	POSIX (`task_create`, etc.)	`rtems_*`

6. Design Patterns for UniversalisOS

6.1 Architecture Abstraction Pattern

Best model: NuttX's 3-layer + RTEMS's CPU interface

include/uos_arch.h          — arch-independent interface (like RTEMS score/cpu.h)
arch/<arch>/include/        — arch-specific type definitions
arch/<arch>/src/<subarch>/  — sub-arch implementation (like NuttX)
arch/<arch>/src/<chip>/     — chip-specific (like NuttX)

Key functions to abstract:

void uos_context_switch(uos_context_t *from, uos_context_t *to);  // RTEMS pattern
void uos_context_save(uos_context_t *ctx);   // ThreadX pattern
void uos_context_restore(uos_context_t *ctx); // ThreadX pattern
void uos_context_init(uos_context_t *ctx, void *sp, void *entry); // RTEMS pattern
void uos_irq_disable(void);    // ThreadX inline pattern
void uos_irq_enable(void);

6.2 No-MMU Strategy

Layer 1 (always): Flat memory, stack canaries, cooperative/idle-hooks (ThreadX baseline) Layer 2 (MPU available): Kernel/user separation via MPU (NuttX CONFIG_BUILD_PROTECTED pattern) Layer 3 (MMU available): Full process isolation (seL4 capability pattern)

MPU abstraction should follow RTEMS armv7-pmsa.h pattern:

void uos_mpu_write_region(uint32_t index, uintptr_t base, size_t size, uint32_t attrs);
void uos_mpu_enable(void);
void uos_mpu_disable(void);
uint32_t uos_mpu_find_region(uintptr_t addr, uint32_t start);

6.3 Context Switch Pattern

For Cortex-M (no MMU, PendSV): Use ThreadX/NuttX PendSV pattern — hardware saves half the context automatically.

For Cortex-A/R (MMU/MPU, IRQ): Use seL4/NuttX trap handler pattern — save all registers on kernel stack, switch stack pointer + page tables.

For RISC-V (all modes): Follow seL4's src/arch/riscv/traps.S pattern — save all CSRs + registers.

Minimal context (Cortex-M): ~68 bytes (17 words) — R4-R11, SP, LR, PC, xPSR, BASEPRI, EXC_RETURN + optional FPU (132 bytes for S0-S31+FPSCR)

Full context (Cortex-A): ~72 bytes (18 words) + FPU (256 bytes for D0-D31) + VFP regs

6.4 Kernel Primitive Naming Convention

Based on the audit, the uos_* naming should follow this pattern:

// Tasks/Threads
uos_task_create(), uos_task_delete(), uos_task_suspend(), uos_task_resume()
uos_thread_create() (for pthread-like)

// Scheduling
uos_sched_yield(), uos_sched_set_priority()

// Synchronization
uos_sem_create(), uos_sem_wait(), uos_sem_post()
uos_mutex_create(), uos_mutex_lock(), uos_mutex_unlock()
uos_event_create(), uos_event_wait(), uos_event_set()

// IPC
uos_mq_create(), uos_mq_send(), uos_mq_receive()
uos_endpoint_send(), uos_endpoint_recv()  // seL4-style synchronous

// Memory
uos_mem_alloc(), uos_mem_free()
uos_mpu_set_region(), uos_mpu_enable()

// Context
uos_context_switch(), uos_context_init()

6.5 Guest RTOS Personality Layer

To host FreeRTOS, ThreadX, Zephyr, Mbed as guests, create thin wrappers:

// FreeRTOS personality: wraps uos_task_create -> xTaskCreate
// ThreadX personality: wraps uos_task_create -> tx_thread_create
// Each personality provides the exact API the guest expects
// All call uos_* primitives underneath

Generated from deep audit of /home/fabiorafaelcoutada/portugalfuturista/rtos_ref/{seL4,threadx,nuttx,rtems}/

37 KiB Raw Blame History

RTOS Reference Deep Audit: seL4, ThreadX, NuttX, RTEMS

1. seL4

1.1 Architecture Support

1.2 Memory Model

1.3 Kernel Primitives

1.4 API Surface

1.5 Hardware Abstraction

1.6 Partition/Isolation Mechanisms

1.7 Context Switch

2. ThreadX (Eclipse ThreadX)

2.1 Architecture Support

2.2 Memory Model

2.3 Kernel Primitives

2.4 API Surface

2.5 Hardware Abstraction

2.6 Partition/Isolation Mechanisms

2.7 Context Switch

3. NuttX

3.1 Architecture Support

3.2 Memory Model

3.3 Kernel Primitives

3.4 API Surface

3.5 Hardware Abstraction

3.6 Partition/Isolation Mechanisms

3.7 Context Switch

4. RTEMS

4.1 Architecture Support

4.2 Memory Model

4.3 Kernel Primitives

4.4 API Surface

4.5 Hardware Abstraction

4.6 Partition/Isolation Mechanisms

4.7 Context Switch

5. Cross-RTOS Comparison Matrix

6. Design Patterns for UniversalisOS

6.1 Architecture Abstraction Pattern

6.2 No-MMU Strategy

6.3 Context Switch Pattern

6.4 Kernel Primitive Naming Convention

6.5 Guest RTOS Personality Layer

37 KiB

Raw Blame History