92 lines
6.2 KiB
Markdown
92 lines
6.2 KiB
Markdown
# Implementation Plan: Mycelium Full Feature Parity (PikeOS & TFW)
|
|
|
|
Provide a comprehensive roadmap to bring the `mycelium` Rust toolchain to 100% feature parity with the proprietary SYSGO `pikeos-configconv` and the PikeOS Test Framework (TFW) generator (`pikeos-configmore --test`).
|
|
|
|
This plan moves beyond the current MVP status to address all known gaps documented in the `CONFIGCONV_CONTRACT.md` and ports the existing Node.js TFW prototype (`universalisos/tfw`) directly into the native Rust `mycelium` binary.
|
|
|
|
## User Review Required
|
|
|
|
> [!WARNING]
|
|
> **TFW Architecture Shift**
|
|
> The current TFW generator is implemented in TypeScript/Node.js (`universalisos/tfw`). This plan proposes porting the generator logic natively into the Rust `mycelium` codebase (e.g., as a new `mycelium-tfw` crate) and exposing it via a `--test` flag. The VS Code extension `mycelium-tfw-extension` will then shell out to the `mycelium` binary instead of importing Node.js modules.
|
|
> *Please confirm if this architectural consolidation is approved.*
|
|
|
|
## Open Questions
|
|
|
|
> [!IMPORTANT]
|
|
> **XML Parsing Strategy**
|
|
> PikeOS tools require parsing both the schema (`.xsd`) and the instance data (`.xml`) for binary and TFW generation. Mycelium currently uses `quick-xml` for the XSD. Should we use the same streaming approach for the instance XML, or load it into a DOM (e.g., using `roxmltree`) since we need to evaluate IDREFs and cross-link tables?
|
|
|
|
---
|
|
|
|
## Proposed Changes
|
|
|
|
### 1. Mycelium Codegen: C-ABI Alignment & Relocation Overhaul (CRITICAL)
|
|
|
|
The current `mycelium-codegen` crate emits binary data by concatenating fields without regard for C-ABI alignment, and it emits strings/arrays inline instead of using pointers. This fundamentally breaks compatibility with the `pikeos-configconv` C structures (causing the `SIGSEGV` in `runner_roundtrip`). We must completely overhaul the binary and header generation.
|
|
|
|
#### [MODIFY] `crates/mycelium-codegen/src/binary.rs`
|
|
- **C-ABI Padding & Alignment:** Implement standard C-ABI alignment rules (1, 2, 4, 8 byte alignment based on field types). Track the current byte offset and insert padding bytes as needed when encoding `struct` fields.
|
|
- **Out-of-line Data (Relocations):**
|
|
- Change `char *` and `_vector_t` field encoding: write a 32-bit or 64-bit pointer placeholder in the main struct body.
|
|
- Append the actual string or array data to a separate data section at the end of the blob.
|
|
- Record the offset in a relocation table.
|
|
- **Header Parity:** Adjust the header generation to correctly match the 16-byte PikeOS format instead of the current 32-byte layout. Append the relocation table (`reloc_start`) at the end of the blob and update the header.
|
|
|
|
#### [MODIFY] `crates/mycelium-codegen/src/header.rs` & `crates/mycelium-model/src/lower.rs`
|
|
- **Vector & Array Fixes:** Currently, `header.rs` incorrectly emits arrays of vector companions (e.g., `person_vector_t prince[1];` for `unbounded` or `person_vector_t aunt[10];` for bounded arrays). Fix the AST-to-IR lowering and header generation:
|
|
- Bounded arrays (`maxOccurs="10"`) should emit inline C arrays (`person_t aunt[10];`) OR use vectors, matching PikeOS behavior.
|
|
- Unbounded arrays (`maxOccurs="unbounded"`) should emit a single vector companion (`person_vector_t prince;`).
|
|
|
|
#### [MODIFY] `adt/adt-kdev/uos_adt_kdev.c`
|
|
- **Relocation Patching:** Update `uos_drv_config_get_data` to iterate over the `reloc_start` table and patch pointers by adding the base address when `UOS_DRV_RELOCATE_ALLOW` is passed, mirroring PikeOS kernel behavior.
|
|
|
|
---
|
|
|
|
### 2. Mycelium Core: Parser Completion & IR Lowering
|
|
|
|
#### [MODIFY] `crates/mycelium-xsd/src/parser.rs`
|
|
- **Fix existing compiler errors:** Resolve the 3 type mismatch errors in the current `ComplexTypeBuilder` and `Attribute` mapping.
|
|
- **Implement Inheritance:** Add support for `xs:extension`, `xs:complexContent`, and `xs:simpleContent`. This requires modifying the parser state machine to capture base types and derived fields, unblocking the final 10 missing VMIT types.
|
|
|
|
#### [NEW] `crates/mycelium-xml/` (New Crate)
|
|
- Create a new crate dedicated to parsing the instance `.xml` data files, validating them against the lowered IR from the XSD, and resolving cross-references (ID/IDREF).
|
|
|
|
---
|
|
|
|
### 3. Mycelium TFW: Test Framework Generator (`pikeos-configmore --test`)
|
|
|
|
#### [NEW] `crates/mycelium-tfw/` (New Crate)
|
|
- Port the logic from `universalisos/tfw/src/generator.ts` to Rust.
|
|
- Parse `TestCaseConfig` (Requirements, Objectives, TestEnv) from XML test specifications.
|
|
- Generate C test harnesses (`.inc` / `.c`) utilizing the `TFW_EVAL`, `TFW_INSPECT`, and `TFW_PASS` macros based on requirement methods (`A`, `I`, `T`).
|
|
|
|
#### [MODIFY] `crates/mycelium-cli/src/main.rs`
|
|
- Wire up the new features:
|
|
- Add `--test` mode (acts as `pikeos-configmore --test`).
|
|
- Add `--catalog=<FILE>` support for resolving OASIS XML catalogs (currently ignored).
|
|
- Wire the XML instance parser for `--bin` and `--test` modes.
|
|
|
|
---
|
|
|
|
### 4. IDE Integration Updates
|
|
|
|
#### [MODIFY] `universalisos/mycelium-tfw-extension/src/extension.ts`
|
|
- Refactor the VS Code extension to invoke the `mycelium --test <xml> --out <output>` CLI command instead of instantiating the Node.js `TfwGenerator`.
|
|
- Remove the dependency on the legacy `universalisos/tfw` Node.js package.
|
|
|
|
#### [DELETE] `universalisos/tfw/`
|
|
- Once the Rust TFW generator is verified, deprecate and remove the Node.js prototype directory.
|
|
|
|
---
|
|
|
|
## Verification Plan
|
|
|
|
### Automated Tests
|
|
- Run `cargo test` across all Mycelium crates.
|
|
- **Round-trip Conformance:** Execute `make` in `conformance/offline-test/` to verify that `mycelium --enum --struct` and `mycelium --bin` outputs pass the C harness tests. The `SIGSEGV` will be resolved once the binary layout correctly implements C-ABI alignment and relocations.
|
|
- **TFW Verification:** Feed `conformance/offline-test/configtest.xml` into `mycelium --test` and diff the output against the expected C macro harness.
|
|
|
|
### Manual Verification
|
|
- Run the modified `mycelium-tfw-extension` in VS Code to ensure it successfully triggers the Rust CLI, generates the test harness, and parses the output correctly.
|
|
- Boot the Universalisos kernel using the newly generated VMIT binary blob to ensure the IDREF tables, relocation lists, and dumpbin headers are parsed successfully by the hypervisor.
|