universalisos/docs/README_Migrate3_x.html
Fábio Coutada 7855eab092 feat(import): Complete PikeOS 5.0 ecosystem import from devvm-arch
MAJOR MILESTONE: Successfully imported complete PikeOS 5.0 ecosystem
from Virtual Machine 100 (devvm-arch) on gigabyte Proxmox server.

Import Summary:
- Source Code: 4.3GB, 63,441 files, 14,829 C/C++/H files
- Documentation: 78MB, 37+ PDF manuals covering all PikeOS aspects
- XSD Schemas: 1.8MB, 158 schema files for configuration/code generation
- Test Framework: Complete TFW infrastructure in src/tfw/

Key Components:
 Multi-architecture kernels (ARM v7/v8, PowerPC e500/e500mc/e5500, x86_64)
 Core source libraries (libstand, ssw, p4ext)
 Complete test framework with executables, includes, libraries
 37+ comprehensive PDF documentation (installation, development, platform manuals)
 158 XSD schemas for Eclipse-based code generation and configuration
 AUTOSAR/MISRA safety-critical compliance foundation
 Hardware virtualization support
 Eclipse IDE project configuration (.project, .cproject)

Documentation Categories:
- Installation & User Guides (pikeos-installation-guide.pdf, user manuals)
- Development References (kernel, driver, test framework manuals)
- API & Programming (C/C++ environments, native API extensions)
- Platform Manuals (ARM, PowerPC, x86 specific guides)
- CDK Documentation (gcc, binutils, ld, cpp internals)
- Release Notes (all components for 5.0.3)
- Specialized Topics (hardware virtualization, POSIX compliance)

Technical Achievements:
 Complete PikeOS 5.0 codebase across multiple processor architectures
 Safety-critical foundation with AUTOSAR/MISRA compliance
 Eclipse IDE integration with XSD-driven code generation pipeline
 Comprehensive test framework for validation and verification
 Complete documentation covering development, deployment, and optimization

Extraction Method:
- Source: VM 100 (devvm-arch) on gigabyte server (192.168.0.104)
- Method: Direct LVM disk mount via losetup, rsync transfer
- Size: Total 4.4GB extracted and organized

Next Phases:
- Phase 3: XSD Workflow Analysis (Day 8-10)
- Phase 4: Safety Standards Compliance Documentation (Day 11-12)
- Phase 5: Component Categorization (Day 13-14)
- Phase 6: Aurelio Brain Test (Day 15)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-06 21:01:41 +01:00

536 lines
17 KiB
HTML

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
<head>
<title>How To Migrate From PSSW PikeOS 3.x To PikeOS 4.0</title>
</head>
<body>
<h1>How To Migrate From PSSW PikeOS 3.x To PikeOS 4.0</h1>
<h2>General</h2>
<p>
PikeOS 4.0 changes several APIs to make them more consistent and also
to add new features. This file is a quick overview of what will probably
need to be changed from migrating from PikeOS 3.4/3.5 to PikeOS 4.0.
</p>
<h3>Renaming</h3>
<p>
To improve consistency in names, some enum identifiers have been
renamed or unified.
</p>
<pre>VM_O_RW --&gt; VM_O_RD_WR
VM_O_RWX --&gt; VM_O_RD_WR_EXEC
VM_MEM_ACCESS_READ --&gt; VM_MEM_ACCESS_RD
VM_MEM_ACCESS_WRITE --&gt; VM_MEM_ACCESS_WR
VM_MEM_ACCESS_RDWR --&gt; VM_MEM_ACCESS_RD_WR
VM_MEM_ACCESS_RWX --&gt; VM_MEM_ACCESS_RD_WR_EXEC
vmitErrorIdentifier_t --&gt; vm_hm_error_identifier_t
vmitErrorLevel_t --&gt; vm_hm_error_level_t
vm_se_pp_relation_t --&gt; vm_port_direction_t (unified)
VM_SE_PP_PROVIDE_SRC --&gt; VM_PORT_SOURCE (unified)
VM_SE_PP_PROVIDE_DST --&gt; VM_PORT_DESTINATION (unified)</pre><p>
The following identifiers were added:
</p>
<pre>VM_MEM_ACCESS_EXEC to have a name for each single bit</pre><p>
The following identifiers were removed:
</p>
<pre>VM_SE_PP_PROVIDE_BOTH
VM_O_STAT</pre><p>
Concerning VM_O_STAT: The right to stat() is granted as soon as an
entry in the VMIT partition file access list exists. The VM_O_RD and
VM_O_WR constants are now single bits, removing some bug potential.
</p>
<p>
There is a script that contains sed expressions for all trivial
changes: 'pikeos-upgrade-source'. It can be used to patch files as follows:
</p>
<pre>/opt/pikeos-4.0/bin/pikeos-upgrade-source FILE</pre><p>
You should always review what was actually changed before commiting!
</p>
<p>
To apply only minimal changes (e.g., no full renaming from VM_E to P4_E),
use:
</p>
<pre>MINIMAL=1 /opt/pikeos-4.0/bin/pikeos-upgrade-source FILE</pre><p>
If you have 'sedrec', it can be used to recursively apply the changes
and produce a patch on STDOUT:
</p>
<pre>sedrec -f /opt/pikeos-4.0/bin/pikeos-upgrade-source</pre><p>
The MINIMAL option should work here, too:
</p>
<pre>MINIMAL=1 sedrec -f /opt/pikeos-4.0/bin/pikeos-upgrade-source</pre><p>
It has a -C option to consider only files that are in CVS (or
Mercurial, sorry, no svn or git yet). It has a -x PATTERN option
to exclude other stuff. It also have a -h option for help.
</p>
<h3>vm_name_type_t</h3>
<p>
Typedefing arrays is usually a bad idea in C, because of the double
nature of arrays, which can also work as pointers in many contexts.
It is better to expose the actual type in order to clarify what users
deal with. Therefore, vm_name_type_t was removed. Variable
declarations like the following will not work anymore:
</p>
<pre>vm_name_type_t name;</pre><p>
These should be replaced by
</p>
<pre>char name[VM_NAME_LEN];</pre><p>
Function declarations using this type should be fixed as follows:
</p>
<pre>void foo(vm_name_type_t name)</pre><p>
Minimal fix either:
</p>
<pre>void foo(const char *name)</pre><p>
or (if name is an output or in/out parameter):
</p>
<pre>void foo(char *name)</pre><p>
Often, it's a good idea to pass the size of array so that the function
has a chance to see that there is enough space:
</p>
<pre>void foo(const char *name, P4_size_t name_sz)</pre><p>
Unfortunately, this is manual migration work that is not automated.
</p>
<h3>vm_se_pp_iterate()</h3>
<p>
The type vm_se_pp_relation_t was removed, because it became redundant.
Instead, vm_port_direction_t is used. In this case, the third
parameter of vm_se_pp_iterate changed. For the renaming, please refer
to the previous section.
</p>
<h2>vm_e_t vs P4_e_t</h2>
<p>
vm_e_t and P4_e_t have been merged. It is now possible to use only
P4_e_t and the corresponding error codes P4_E_* instead of VM_E_*.
</p>
<p>
In this process, a few error codes have been merged or split in order
to unify the two previously distinct error codes. Some changes are
trivial renamings (and typically, mergers are no general problem), but
some other changes like splits are more involved.
</p>
<h3>Complex Changes (Split of Error Codes)</h3>
<p>
P4_E_NOTAVAIL may be either P4_E_NOENT or P4_E_TIMEOUT. For any
functions ending in 'trylock' or 'trywait', it is P4_E_TIMEOUT. For
the others, it is P4_E_NOENT. In particular:
</p>
<pre>P4_E_NOTAVAIL -&gt; P4_E_NOENT:
p4_dev_call
p4_int_attach
p4_int_wait
p4_memmap_alloc_phys
p4_memmap_alloc_aligned
p4_task_activate
p4_timepart_define
p4_timepart_start
p4_timepart_change
P4_E_NOTAVAIL -&gt; P4_E_TIMEOUT:
p4_mutex_trylock
p4_rwlock_rdtrylock
p4_rwlock_wrtrylock
p4_sem_trywait</pre><p>
P4_E_PARTNER is either P4_E_STATE or P4_E_BUSY:
</p>
<pre>P4_E_PARTNER -&gt; P4_E_STATE:
p4_ipc (et. al.)
P4_E_PARTNER -&gt; P4_E_BUSY:
p4_thread_preempt</pre><p>
P4_E_FULL is either P4_E_SIZE (in case it is an error) or P4_E_TRUNC
(in case it is an warning).
</p>
<pre>P4_E_FULL -&gt; P4_E_SIZE:
p4_trace_map
P4_E_FULL -&gt; P4_E_TRUNC:
p4_mem_build_sglist</pre><p>
VM_E_BUSY has been split into VM_E_BUSY and VM_E_INCOMPLETE.
Generally, the specific error that a thread has already an ongoing
request in the PSSW is now VM_E_INCOMPLETE. This error is reserved
for raising by the PSSW. A general BUSY signal, e.g. from drivers, is
VM_E_BUSY.
</p>
<p>
The code VM_E_NO_CONNECTION is no error code anymore: the behaviour
was changed so that VM_E_OK is returned and the channel behaves as if
there was an inactive buddy on the other side of the channel.
</p>
<p>
The code VM_E_INDICATED is deprecated. Other error codes should be
used (this was a driver-only error code).
</p>
<p>
The code VM_E_LOAD_FILE was removed. It was exclusively used inside
the PSSW.
</p>
<h3>Simple Changes (Merging of Error Codes)</h3>
<p>
P4_E_NOTOWNER was merged wit P4_E_PERM.
</p>
<p>
The code VM_E_INVAL_VERSION was merged with VM_E_INVAL_CONFIG. This
error code was exclusively generated by vm_init().
</p>
<p>
The error case VM_E_NOT_INITIALIZED was removed. Instead, if
vm_init() is not invoked, the libvm has undefined behaviour.
</p>
<p>
In the case of the PSSW, the error codes VM_E_FULL, VM_E_EMPTY,
VM_E_AGAIN and VM_E_TIMED_OUT have been merged. The only remaining
code is VM_E_TIMED_OUT (which is identical to P4_E_TIMEOUT now). This
means that system extensions should be fixed to return only
VM_E_TIMED_OUT (or P4_E_TIMEOUT) instead of the error codes mentioned
before.
</p>
<h3>Automatic Updating</h3>
<p>
The pikeos-upgrade-source script can do the necessary renamings
automatically. The split error codes, however, must be handled
manually.
</p>
<p>
By default, the update script will replace of all vm_e_t and VM_E_* so
that exclusively P4_e_t and P4_e are used. To only do minimal changes
that are necessary due to merging, set the environment variable
MINIMAL to 1 prior to using pikeos-upgrade-source.
</p>
<h2>System Extensions</h2>
<h3>Malloc</h3>
<p>
PikeOS 4.0 uses different memory pools for different partitions in the
PSSW. To let system extensions have access to this feature, the
vm_malloc() function family was reworked to allow a partition number
to be passed.
</p>
<p>
Furthermore, all functions allocating physical memory now follow the
POSIX calloc() model to take an element count and an element size
argument, i.e., they allocate arrays of elements. This was introduced
to protect against integer overflows, which are a frequent bug and a
thread to security. Multiplication of element count and element size
is now encapsulated into the allocation function to take away the
burdon from the user of checking for an integer overflow in the
multiplication. The functions indicate an out-of-memory condition in
case of an integer overflow.
</p>
<p>
If possible, the second parameter should be constant. This way, the
code will be more efficient, because most compilers optimise better.
</p>
<p>
BEFORE:
</p>
<pre>vm_malloc(size, alignment)</pre><p>
AFTER, if alignment &gt; 0:
</p>
<pre>vm_malloc_aligned(elem_cnt, elem_size, alignment, partition)</pre><p>
AFTER, if alignment == 0:
</p>
<pre>vm_malloc(elem_cnt, elem_size, partition)</pre><p>
There are also function to clear the memory now:
</p>
<pre>vm_calloc(elem_cnt, elem_size, partition)
vm_calloc(elem_cnt, elem_size, alignment, partition)</pre><p>
For global memory (the behaviour the function had in previous PikeOS
versions), the value VM_GLOBAL can be specified for the partition. A
trivial mapping to get a driver running again would be:
</p>
<pre>BEFORE AFTER
vm_malloc(X,0) --&gt; vm_malloc(X, 1, VM_GLOBAL)
vm_malloc(N * sizeof(T), 0) --&gt; vm_malloc(N, sizeof(T), VM_GLOBAL)
vm_malloc(X,a) --&gt; vm_malloc_aligned(X, 1, a, VM_GLOBAL)</pre><p>
Furthermore, there are functions for saturated multiplication and
addition to safely calculate values for memory sizes.
</p>
<pre>unsigned long adt_sat_mul(unsigned long x, unsigned long y)
unsigned long adt_sat_add(unsigned long x, unsigned long y)</pre><p>
These return ~(unsigned long)0 on overflow (this constant is also
called ADT_SATURATED for convenience).
</p>
<p>
These functions are static inline functions from the util-adt base
package in the adt/saturated.h header file.
</p>
<h3>Allocate Virtual Memory</h3>
<p>
The function vm_vmem_alloc was renamed to vm_alloc_virt. Due to
architectural requirements on some machines, the function now has an
extended API, taking a reference physical address. The API was also
simplified a bit.
</p>
<p>
BEFORE:
</p>
<pre>rc = vm_vmem_virt(size, align, &amp;addr);
if (rc != VM_E_OK)
...error...</pre><p>
AFTER:
</p>
<pre>addr = vm_alloc_virt(size, phys_addr, align);
if (addr == 0)
...error...</pre><p>
Here, phys_addr should be the address to be mapped into the virtual
memory region.
</p>
<p>
This function does not have two params for the size, because it is
relatively unlikely to allocate an array here (in contract to normal
malloc). If needed, use adt_sat_mul() to do the multiply safely.
</p>
<h2>Health Monitoring</h2>
<p>
The health monitoring was moved from the PSSW to the kernel. It was
completely restructured and made more consistent. It now supported
the newest APEX specification that is also used by the ARINC653
personality of PikeOS.
</p>
<p>
Due to the restructuring, system extensions and external file
providers that dealt directly with health monitoring will have to be
rewritten. The changes are numerous, to please refer to the
introductory chapters in the Fundamentals and in the Kernel Reference
Manual.
</p>
<h3>Renamings</h3>
<p>
The system extensions are now linked directly with the PSSW, and thus
there is no libivm anymore. Some functions were renamed in order to
achieve better consistency and also to resolve conflicts. The
following is a list of renamings:
</p>
<pre>vm_se_add_romimage -&gt; vm_add_romimage
vm_se_crit_enter -&gt; vm_crit_enter
vm_se_crit_leave -&gt; vm_crit_leave
vm_se_fp_notify -&gt; vm_fp_notify
vm_se_fp_queue_init -&gt; vm_fp_queue_init
vm_se_fp_unit_init -&gt; vm_fp_unit_init
vm_se_get_acl -&gt; vm_get_acl
vm_se_hm_raise_error -&gt; vm_hm_inject_error
vm_se_kfd_reserve -&gt; vm_kfd_reserve
vm_se_monitor_hook -&gt; vm_monitor_hook
vm_se_pp_config_data -&gt; vm_pp_config_data
vm_se_pp_iterate -&gt; vm_pp_iterate
vm_se_pp_qp_notify -&gt; vm_pp_qport_notify
vm_se_qp_control -&gt; vm_se_qport_control
vm_se_qp_open -&gt; vm_se_qport_open
vm_se_qp_probe -&gt; vm_se_qport_probe
vm_se_qp_read -&gt; vm_se_qport_read
vm_se_qp_write -&gt; vm_se_qport_write
vm_se_register_part_change_cb -&gt; vm_register_part_change_cb
vm_se_shm_start -&gt; vm_shm_iter_init
vm_se_shm_step -&gt; vm_shm_iter_take
vm_se_sp_control -&gt; vm_se_sport_control
vm_se_sp_open -&gt; vm_se_sport_open
vm_se_sp_read -&gt; vm_se_sport_read
vm_se_sp_write -&gt; vm_se_sport_write
vm_se_thread_create -&gt; vm_thread_create
vm_se_thread_start -&gt; vm_thread_start
vm_se_thread_stop -&gt; vm_thread_stop
vm_se_pp_qp_create_t -&gt; vm_se_qport_create_t
vm_se_pp_qp_stat_t -&gt; vm_se_qport_stat_t
vm_se_pp_qp_read_t -&gt; vm_se_qport_read_t
vm_se_pp_qp_read_routed_t -&gt; vm_se_qport_read_routed_t
vm_se_pp_qp_write_t -&gt; vm_se_qport_write_t
vm_se_pp_qp_write_routed_t -&gt; vm_se_qport_write_routed_t
vm_se_pp_qp_control_t -&gt; vm_se_qport_control_t
vm_se_pp_sp_create_t -&gt; vm_se_sport_create_t
vm_se_pp_sp_read_t -&gt; vm_se_sport_read_t
vm_se_pp_sp_write_t -&gt; vm_se_sport_write_t
vm_se_pp_sp_control_t -&gt; vm_se_sport_control_t</pre><p>
The script '/opt/pikeos-4.0/bin/pikeos-upgrade-source' will do these
renamings for a given .c or .h file together with the other renamings
from this file.
</p>
<h3>Blocking</h3>
<p>
The old VM_E_BLOCK API with all its derivatives was removed. SEs can
only use the vm_req-Framework now. This change was previously
anticipated by deprecating VM_E_BLOCK in PikeOS 3.5 already. PikeOS
4.0 replaces the old API completely.
</p>
<p>
Typical updates to an SE include:
</p>
<p>
Declarations:
</p>
<p>
OLD:
</p>
<pre>/*
* Static allocation and ID assignment of blocking queues in driver.
*/
#define BQ_READ 0
#define BQ_WRITE 1
#define BQ_COUNT 2
#define BU_DEV1 0
#define BU_DEV2 1
#define BU_COUNT 2</pre><p>
NEW:
</p>
<pre>/*
* Reserve data slots for ID assignment of blocking queues by framework.
* We need one per provider per device per function that might block.
*/
typedef struct {
...old data per device...
vm_req_queue_t q_read;
vm_req_queue_t q_write;
} dev_data_t;</pre><p>
Init: allocation of blocking queues:
</p>
<p>
OLD:
</p>
<pre>rc = vm_se_fp_unit_init(s, dev_count);
if (rc != VM_E_OK) {
return rc;
}
...allocation of the blocking queues...</pre><p>
NEW:
</p>
<pre>for (int i = 0; i &lt; dev_count; i++) {
prov_data-&gt;dev[i].q_read = vm_req_queue_new();
prov_data-&gt;dev[i].q_write = vm_req_queue_new();
}</pre><p>
Function Prototypes of callbacks:
</p>
<p>
OLD:
</p>
<pre>vm_e_t my_read(
void *private,
P4_uid_t client,
vm_file_desc_int_t *fd,
void *buff,
P4_size_t buff_size,
vm_origin_t origin,
vm_off_t offset,
P4_size_t *read_size_p,
vm_off_t *new_pos_p,
P4_uint32_t *logical_unit,
P4_uint32_t *blocking_queue)</pre><p>
NEW:
</p>
<pre>vm_e_t my_read(
void *private,
P4_uid_t client,
vm_file_desc_int_t *fd,
void *buff,
P4_size_t buff_size,
vm_origin_t origin,
vm_off_t offset,
P4_size_t *read_size_p,
vm_off_t *new_pos_p)</pre><p>
Program Logic:
</p>
<p>
OLD:
</p>
<pre>if (!can_transmit) {
*blocking_queue = BQ_READ;
*logical_unit = dev_id; /* 0 or 1 */
return VM_E_BLOCK;
}</pre><p>
NEW:
</p>
<pre>P4_prio_t state;
vm_crit_enter(&amp;state);
if (!can_transmit) {
vm_req_defer(prov_data-&gt;dev[dev_id].q_read);
rc = P4_E_TIMEOUT;
}
vm_crit_leave(&amp;state);
if (rc != P4_E_OK) {
return rc;
}</pre><p>
Also, a call to vm_se_fp_notify() should now be an appropriate call to
vm_req_retry().
</p>
<p>
For queuing port providers, this is similar. Except there were no
blockning queues before, but these are needed now. Also, the result
was VM_E_FULL/EMPTY before and this can be kept. vm_se_pp_notify() is
to be replaced by a corresponding vm_req_retry().
</p>
<h2>File Format Changes</h2>
<p>
The VMIT format has changed. In PikeOS 4.0, the project configurator
does not read the VMIT anymore, but only generates it. Therefore,
with normal SYSGO tooling, the project.xml file is now the top-level
file.
</p>
<p>
For this reason, no tool is provided for converting the VMIT format,
but instead the project.xml file needs to be converted and possibly
reconstructed, because some information like component dependency
information is not stored in PikeOS 3.x's project.xml.conf file
format.
</p>
<h3>Converting project.xml.conf to project.xml</h3>
<p>
This is not possible automatically for integration or ukernel
projects. But for anything else, it is:
</p>
<pre>pikeos-upgrade-project-pre4</pre><p>
By default, it reads project.xml.conf and writes project.xml.
It should work without problems. If there are any problems,
please contact product support at:
<a href="https://www.sysgo.com/support/">https://www.sysgo.com/support/</a>
</p>
<h3>Converting blah.perso.xml to blah.profile.xml</h3>
<p>
Same tool:
</p>
<pre>pikeos-upgrade-project-pre4 blah.perso.xml blah.profile.xml</pre><p>
It is currently not able to restructure the Depend="..." into
&lt;Condition condition="..."&gt; properly. It <b>does</b> convert the
stuff, but the nesting may be a little broken, because &lt;Option&gt;
cannot appear inside &lt;Condition&gt;, so some manual work may be needed.
You will probably see that you can simplify the condition structure
when checking and fixing the generated file.
</p>
<p>
Otherwise, it converts quite well: it will convert types, even
MULTIPLE, etc. Just the nested needs to be checked, normally.
</p>
<h3>Option files: blah.option.xml</h3>
<p>
Same tool, same call:
</p>
<pre>pikeos-upgrade-project-pre4 blah.option.xml blah.option.xml.new</pre><p>
Again, you need to check and edit
like before. With options, you will run in even more &lt;Condition&gt;al
stuff.
</p>
<p>
Plus, there is another corner case you need to fix: the option file
often has a top-level Depends="..." which causes all content to be
moved to a &lt;Condition&gt;. This usually also includes the &lt;Parameter&gt;
the &lt;Condition&gt; depends on. You need to move that Parameter out of
the &lt;Condition&gt;.
</p>
</body>