universalisos/implementation_plan.md

Implementation Plan: Mycelium Full Feature Parity (PikeOS & TFW)

Provide a comprehensive roadmap to bring the mycelium Rust toolchain to 100% feature parity with the proprietary SYSGO pikeos-configconv and the PikeOS Test Framework (TFW) generator (pikeos-configmore --test).

This plan moves beyond the current MVP status to address all known gaps documented in the CONFIGCONV_CONTRACT.md and ports the existing Node.js TFW prototype (universalisos/tfw) directly into the native Rust mycelium binary.

User Review Required

Warning

TFW Architecture Shift The current TFW generator is implemented in TypeScript/Node.js (universalisos/tfw). This plan proposes porting the generator logic natively into the Rust mycelium codebase (e.g., as a new mycelium-tfw crate) and exposing it via a --test flag. The VS Code extension mycelium-tfw-extension will then shell out to the mycelium binary instead of importing Node.js modules. Please confirm if this architectural consolidation is approved.

Open Questions

Important

XML Parsing Strategy PikeOS tools require parsing both the schema (.xsd) and the instance data (.xml) for binary and TFW generation. Mycelium currently uses quick-xml for the XSD. Should we use the same streaming approach for the instance XML, or load it into a DOM (e.g., using roxmltree) since we need to evaluate IDREFs and cross-link tables?

---

Proposed Changes

1. Mycelium Codegen: C-ABI Alignment & Relocation Overhaul (CRITICAL)

The current mycelium-codegen crate emits binary data by concatenating fields without regard for C-ABI alignment, and it emits strings/arrays inline instead of using pointers. This fundamentally breaks compatibility with the pikeos-configconv C structures (causing the SIGSEGV in runner_roundtrip). We must completely overhaul the binary and header generation.

[MODIFY] crates/mycelium-codegen/src/binary.rs

• C-ABI Padding & Alignment: Implement standard C-ABI alignment rules (1, 2, 4, 8 byte alignment based on field types). Track the current byte offset and insert padding bytes as needed when encoding struct fields.
• Out-of-line Data (Relocations):
  • Change char * and _vector_t field encoding: write a 32-bit or 64-bit pointer placeholder in the main struct body.
  • Append the actual string or array data to a separate data section at the end of the blob.
  • Record the offset in a relocation table.
• Header Parity: Adjust the header generation to correctly match the 16-byte PikeOS format instead of the current 32-byte layout. Append the relocation table (reloc_start) at the end of the blob and update the header.

[MODIFY] crates/mycelium-codegen/src/header.rs & crates/mycelium-model/src/lower.rs

• Vector & Array Fixes: Currently, header.rs incorrectly emits arrays of vector companions (e.g., person_vector_t prince[1]; for unbounded or person_vector_t aunt[10]; for bounded arrays). Fix the AST-to-IR lowering and header generation:
  • Bounded arrays (maxOccurs="10") should emit inline C arrays (person_t aunt[10];) OR use vectors, matching PikeOS behavior.
  • Unbounded arrays (maxOccurs="unbounded") should emit a single vector companion (person_vector_t prince;).

[MODIFY] adt/adt-kdev/uos_adt_kdev.c

• Relocation Patching: Update uos_drv_config_get_data to iterate over the reloc_start table and patch pointers by adding the base address when UOS_DRV_RELOCATE_ALLOW is passed, mirroring PikeOS kernel behavior.

---

2. Mycelium Core: Parser Completion & IR Lowering

[MODIFY] crates/mycelium-xsd/src/parser.rs

• Fix existing compiler errors: Resolve the 3 type mismatch errors in the current ComplexTypeBuilder and Attribute mapping.
• Implement Inheritance: Add support for xs:extension, xs:complexContent, and xs:simpleContent. This requires modifying the parser state machine to capture base types and derived fields, unblocking the final 10 missing VMIT types.

[NEW] crates/mycelium-xml/ (New Crate)

• Create a new crate dedicated to parsing the instance .xml data files, validating them against the lowered IR from the XSD, and resolving cross-references (ID/IDREF).

---

3. Mycelium TFW: Test Framework Generator (pikeos-configmore --test)

[NEW] crates/mycelium-tfw/ (New Crate)

• Port the logic from universalisos/tfw/src/generator.ts to Rust.
• Parse TestCaseConfig (Requirements, Objectives, TestEnv) from XML test specifications.
• Generate C test harnesses (.inc / .c) utilizing the TFW_EVAL, TFW_INSPECT, and TFW_PASS macros based on requirement methods (A, I, T).

[MODIFY] crates/mycelium-cli/src/main.rs

• Wire up the new features:
  • Add --test mode (acts as pikeos-configmore --test).
  • Add --catalog=<FILE> support for resolving OASIS XML catalogs (currently ignored).
  • Wire the XML instance parser for --bin and --test modes.

---

4. IDE Integration Updates

[MODIFY] universalisos/mycelium-tfw-extension/src/extension.ts

• Refactor the VS Code extension to invoke the mycelium --test <xml> --out <output> CLI command instead of instantiating the Node.js TfwGenerator.
• Remove the dependency on the legacy universalisos/tfw Node.js package.

[DELETE] universalisos/tfw/

• Once the Rust TFW generator is verified, deprecate and remove the Node.js prototype directory.

---

Verification Plan

Automated Tests

• Run cargo test across all Mycelium crates.
• Round-trip Conformance: Execute make in conformance/offline-test/ to verify that mycelium --enum --struct and mycelium --bin outputs pass the C harness tests. The SIGSEGV will be resolved once the binary layout correctly implements C-ABI alignment and relocations.
• TFW Verification: Feed conformance/offline-test/configtest.xml into mycelium --test and diff the output against the expected C macro harness.

Manual Verification

• Run the modified mycelium-tfw-extension in VS Code to ensure it successfully triggers the Rust CLI, generates the test harness, and parses the output correctly.
• Boot the Universalisos kernel using the newly generated VMIT binary blob to ensure the IDREF tables, relocation lists, and dumpbin headers are parsed successfully by the hypervisor.