PikeOS 4.0 changes several APIs to make them more consistent and also to add new features. This file is a quick overview of what will probably need to be changed from migrating from PikeOS 3.4/3.5 to PikeOS 4.0.
To improve consistency in names, some enum identifiers have been renamed or unified.
VM_O_RW --> VM_O_RD_WR VM_O_RWX --> VM_O_RD_WR_EXEC VM_MEM_ACCESS_READ --> VM_MEM_ACCESS_RD VM_MEM_ACCESS_WRITE --> VM_MEM_ACCESS_WR VM_MEM_ACCESS_RDWR --> VM_MEM_ACCESS_RD_WR VM_MEM_ACCESS_RWX --> VM_MEM_ACCESS_RD_WR_EXEC vmitErrorIdentifier_t --> vm_hm_error_identifier_t vmitErrorLevel_t --> vm_hm_error_level_t vm_se_pp_relation_t --> vm_port_direction_t (unified) VM_SE_PP_PROVIDE_SRC --> VM_PORT_SOURCE (unified) VM_SE_PP_PROVIDE_DST --> VM_PORT_DESTINATION (unified)
The following identifiers were added:
VM_MEM_ACCESS_EXEC to have a name for each single bit
The following identifiers were removed:
VM_SE_PP_PROVIDE_BOTH VM_O_STAT
Concerning VM_O_STAT: The right to stat() is granted as soon as an entry in the VMIT partition file access list exists. The VM_O_RD and VM_O_WR constants are now single bits, removing some bug potential.
There is a script that contains sed expressions for all trivial changes: 'pikeos-upgrade-source'. It can be used to patch files as follows:
/opt/pikeos-4.0/bin/pikeos-upgrade-source FILE
You should always review what was actually changed before commiting!
To apply only minimal changes (e.g., no full renaming from VM_E to P4_E), use:
MINIMAL=1 /opt/pikeos-4.0/bin/pikeos-upgrade-source FILE
If you have 'sedrec', it can be used to recursively apply the changes and produce a patch on STDOUT:
sedrec -f /opt/pikeos-4.0/bin/pikeos-upgrade-source
The MINIMAL option should work here, too:
MINIMAL=1 sedrec -f /opt/pikeos-4.0/bin/pikeos-upgrade-source
It has a -C option to consider only files that are in CVS (or Mercurial, sorry, no svn or git yet). It has a -x PATTERN option to exclude other stuff. It also have a -h option for help.
Typedefing arrays is usually a bad idea in C, because of the double nature of arrays, which can also work as pointers in many contexts. It is better to expose the actual type in order to clarify what users deal with. Therefore, vm_name_type_t was removed. Variable declarations like the following will not work anymore:
vm_name_type_t name;
These should be replaced by
char name[VM_NAME_LEN];
Function declarations using this type should be fixed as follows:
void foo(vm_name_type_t name)
Minimal fix either:
void foo(const char *name)
or (if name is an output or in/out parameter):
void foo(char *name)
Often, it's a good idea to pass the size of array so that the function has a chance to see that there is enough space:
void foo(const char *name, P4_size_t name_sz)
Unfortunately, this is manual migration work that is not automated.
The type vm_se_pp_relation_t was removed, because it became redundant. Instead, vm_port_direction_t is used. In this case, the third parameter of vm_se_pp_iterate changed. For the renaming, please refer to the previous section.
vm_e_t and P4_e_t have been merged. It is now possible to use only P4_e_t and the corresponding error codes P4_E_* instead of VM_E_*.
In this process, a few error codes have been merged or split in order to unify the two previously distinct error codes. Some changes are trivial renamings (and typically, mergers are no general problem), but some other changes like splits are more involved.
P4_E_NOTAVAIL may be either P4_E_NOENT or P4_E_TIMEOUT. For any functions ending in 'trylock' or 'trywait', it is P4_E_TIMEOUT. For the others, it is P4_E_NOENT. In particular:
P4_E_NOTAVAIL -> P4_E_NOENT:
p4_dev_call
p4_int_attach
p4_int_wait
p4_memmap_alloc_phys
p4_memmap_alloc_aligned
p4_task_activate
p4_timepart_define
p4_timepart_start
p4_timepart_change
P4_E_NOTAVAIL -> P4_E_TIMEOUT:
p4_mutex_trylock
p4_rwlock_rdtrylock
p4_rwlock_wrtrylock
p4_sem_trywaitP4_E_PARTNER is either P4_E_STATE or P4_E_BUSY:
P4_E_PARTNER -> P4_E_STATE:
p4_ipc (et. al.)
P4_E_PARTNER -> P4_E_BUSY:
p4_thread_preemptP4_E_FULL is either P4_E_SIZE (in case it is an error) or P4_E_TRUNC (in case it is an warning).
P4_E_FULL -> P4_E_SIZE:
p4_trace_map
P4_E_FULL -> P4_E_TRUNC:
p4_mem_build_sglistVM_E_BUSY has been split into VM_E_BUSY and VM_E_INCOMPLETE. Generally, the specific error that a thread has already an ongoing request in the PSSW is now VM_E_INCOMPLETE. This error is reserved for raising by the PSSW. A general BUSY signal, e.g. from drivers, is VM_E_BUSY.
The code VM_E_NO_CONNECTION is no error code anymore: the behaviour was changed so that VM_E_OK is returned and the channel behaves as if there was an inactive buddy on the other side of the channel.
The code VM_E_INDICATED is deprecated. Other error codes should be used (this was a driver-only error code).
The code VM_E_LOAD_FILE was removed. It was exclusively used inside the PSSW.
P4_E_NOTOWNER was merged wit P4_E_PERM.
The code VM_E_INVAL_VERSION was merged with VM_E_INVAL_CONFIG. This error code was exclusively generated by vm_init().
The error case VM_E_NOT_INITIALIZED was removed. Instead, if vm_init() is not invoked, the libvm has undefined behaviour.
In the case of the PSSW, the error codes VM_E_FULL, VM_E_EMPTY, VM_E_AGAIN and VM_E_TIMED_OUT have been merged. The only remaining code is VM_E_TIMED_OUT (which is identical to P4_E_TIMEOUT now). This means that system extensions should be fixed to return only VM_E_TIMED_OUT (or P4_E_TIMEOUT) instead of the error codes mentioned before.
The pikeos-upgrade-source script can do the necessary renamings automatically. The split error codes, however, must be handled manually.
By default, the update script will replace of all vm_e_t and VM_E_* so that exclusively P4_e_t and P4_e are used. To only do minimal changes that are necessary due to merging, set the environment variable MINIMAL to 1 prior to using pikeos-upgrade-source.
PikeOS 4.0 uses different memory pools for different partitions in the PSSW. To let system extensions have access to this feature, the vm_malloc() function family was reworked to allow a partition number to be passed.
Furthermore, all functions allocating physical memory now follow the POSIX calloc() model to take an element count and an element size argument, i.e., they allocate arrays of elements. This was introduced to protect against integer overflows, which are a frequent bug and a thread to security. Multiplication of element count and element size is now encapsulated into the allocation function to take away the burdon from the user of checking for an integer overflow in the multiplication. The functions indicate an out-of-memory condition in case of an integer overflow.
If possible, the second parameter should be constant. This way, the code will be more efficient, because most compilers optimise better.
BEFORE:
vm_malloc(size, alignment)
AFTER, if alignment > 0:
vm_malloc_aligned(elem_cnt, elem_size, alignment, partition)
AFTER, if alignment == 0:
vm_malloc(elem_cnt, elem_size, partition)
There are also function to clear the memory now:
vm_calloc(elem_cnt, elem_size, partition) vm_calloc(elem_cnt, elem_size, alignment, partition)
For global memory (the behaviour the function had in previous PikeOS versions), the value VM_GLOBAL can be specified for the partition. A trivial mapping to get a driver running again would be:
BEFORE AFTER vm_malloc(X,0) --> vm_malloc(X, 1, VM_GLOBAL) vm_malloc(N * sizeof(T), 0) --> vm_malloc(N, sizeof(T), VM_GLOBAL) vm_malloc(X,a) --> vm_malloc_aligned(X, 1, a, VM_GLOBAL)
Furthermore, there are functions for saturated multiplication and addition to safely calculate values for memory sizes.
unsigned long adt_sat_mul(unsigned long x, unsigned long y) unsigned long adt_sat_add(unsigned long x, unsigned long y)
These return ~(unsigned long)0 on overflow (this constant is also called ADT_SATURATED for convenience).
These functions are static inline functions from the util-adt base package in the adt/saturated.h header file.
The function vm_vmem_alloc was renamed to vm_alloc_virt. Due to architectural requirements on some machines, the function now has an extended API, taking a reference physical address. The API was also simplified a bit.
BEFORE:
rc = vm_vmem_virt(size, align, &addr); if (rc != VM_E_OK) ...error...
AFTER:
addr = vm_alloc_virt(size, phys_addr, align); if (addr == 0) ...error...
Here, phys_addr should be the address to be mapped into the virtual memory region.
This function does not have two params for the size, because it is relatively unlikely to allocate an array here (in contract to normal malloc). If needed, use adt_sat_mul() to do the multiply safely.
The health monitoring was moved from the PSSW to the kernel. It was completely restructured and made more consistent. It now supported the newest APEX specification that is also used by the ARINC653 personality of PikeOS.
Due to the restructuring, system extensions and external file providers that dealt directly with health monitoring will have to be rewritten. The changes are numerous, to please refer to the introductory chapters in the Fundamentals and in the Kernel Reference Manual.
The system extensions are now linked directly with the PSSW, and thus there is no libivm anymore. Some functions were renamed in order to achieve better consistency and also to resolve conflicts. The following is a list of renamings:
vm_se_add_romimage -> vm_add_romimage vm_se_crit_enter -> vm_crit_enter vm_se_crit_leave -> vm_crit_leave vm_se_fp_notify -> vm_fp_notify vm_se_fp_queue_init -> vm_fp_queue_init vm_se_fp_unit_init -> vm_fp_unit_init vm_se_get_acl -> vm_get_acl vm_se_hm_raise_error -> vm_hm_inject_error vm_se_kfd_reserve -> vm_kfd_reserve vm_se_monitor_hook -> vm_monitor_hook vm_se_pp_config_data -> vm_pp_config_data vm_se_pp_iterate -> vm_pp_iterate vm_se_pp_qp_notify -> vm_pp_qport_notify vm_se_qp_control -> vm_se_qport_control vm_se_qp_open -> vm_se_qport_open vm_se_qp_probe -> vm_se_qport_probe vm_se_qp_read -> vm_se_qport_read vm_se_qp_write -> vm_se_qport_write vm_se_register_part_change_cb -> vm_register_part_change_cb vm_se_shm_start -> vm_shm_iter_init vm_se_shm_step -> vm_shm_iter_take vm_se_sp_control -> vm_se_sport_control vm_se_sp_open -> vm_se_sport_open vm_se_sp_read -> vm_se_sport_read vm_se_sp_write -> vm_se_sport_write vm_se_thread_create -> vm_thread_create vm_se_thread_start -> vm_thread_start vm_se_thread_stop -> vm_thread_stop vm_se_pp_qp_create_t -> vm_se_qport_create_t vm_se_pp_qp_stat_t -> vm_se_qport_stat_t vm_se_pp_qp_read_t -> vm_se_qport_read_t vm_se_pp_qp_read_routed_t -> vm_se_qport_read_routed_t vm_se_pp_qp_write_t -> vm_se_qport_write_t vm_se_pp_qp_write_routed_t -> vm_se_qport_write_routed_t vm_se_pp_qp_control_t -> vm_se_qport_control_t vm_se_pp_sp_create_t -> vm_se_sport_create_t vm_se_pp_sp_read_t -> vm_se_sport_read_t vm_se_pp_sp_write_t -> vm_se_sport_write_t vm_se_pp_sp_control_t -> vm_se_sport_control_t
The script '/opt/pikeos-4.0/bin/pikeos-upgrade-source' will do these renamings for a given .c or .h file together with the other renamings from this file.
The old VM_E_BLOCK API with all its derivatives was removed. SEs can only use the vm_req-Framework now. This change was previously anticipated by deprecating VM_E_BLOCK in PikeOS 3.5 already. PikeOS 4.0 replaces the old API completely.
Typical updates to an SE include:
Declarations:
OLD:
/* * Static allocation and ID assignment of blocking queues in driver. */ #define BQ_READ 0 #define BQ_WRITE 1 #define BQ_COUNT 2 #define BU_DEV1 0 #define BU_DEV2 1 #define BU_COUNT 2
NEW:
/*
* Reserve data slots for ID assignment of blocking queues by framework.
* We need one per provider per device per function that might block.
*/
typedef struct {
...old data per device...
vm_req_queue_t q_read;
vm_req_queue_t q_write;
} dev_data_t;Init: allocation of blocking queues:
OLD:
rc = vm_se_fp_unit_init(s, dev_count);
if (rc != VM_E_OK) {
return rc;
}
...allocation of the blocking queues...NEW:
for (int i = 0; i < dev_count; i++) {
prov_data->dev[i].q_read = vm_req_queue_new();
prov_data->dev[i].q_write = vm_req_queue_new();
}Function Prototypes of callbacks:
OLD:
vm_e_t my_read(
void *private,
P4_uid_t client,
vm_file_desc_int_t *fd,
void *buff,
P4_size_t buff_size,
vm_origin_t origin,
vm_off_t offset,
P4_size_t *read_size_p,
vm_off_t *new_pos_p,
P4_uint32_t *logical_unit,
P4_uint32_t *blocking_queue)NEW:
vm_e_t my_read(
void *private,
P4_uid_t client,
vm_file_desc_int_t *fd,
void *buff,
P4_size_t buff_size,
vm_origin_t origin,
vm_off_t offset,
P4_size_t *read_size_p,
vm_off_t *new_pos_p)Program Logic:
OLD:
if (!can_transmit) {
*blocking_queue = BQ_READ;
*logical_unit = dev_id; /* 0 or 1 */
return VM_E_BLOCK;
}NEW:
P4_prio_t state;
vm_crit_enter(&state);
if (!can_transmit) {
vm_req_defer(prov_data->dev[dev_id].q_read);
rc = P4_E_TIMEOUT;
}
vm_crit_leave(&state);
if (rc != P4_E_OK) {
return rc;
}Also, a call to vm_se_fp_notify() should now be an appropriate call to vm_req_retry().
For queuing port providers, this is similar. Except there were no blockning queues before, but these are needed now. Also, the result was VM_E_FULL/EMPTY before and this can be kept. vm_se_pp_notify() is to be replaced by a corresponding vm_req_retry().
The VMIT format has changed. In PikeOS 4.0, the project configurator does not read the VMIT anymore, but only generates it. Therefore, with normal SYSGO tooling, the project.xml file is now the top-level file.
For this reason, no tool is provided for converting the VMIT format, but instead the project.xml file needs to be converted and possibly reconstructed, because some information like component dependency information is not stored in PikeOS 3.x's project.xml.conf file format.
This is not possible automatically for integration or ukernel projects. But for anything else, it is:
pikeos-upgrade-project-pre4
By default, it reads project.xml.conf and writes project.xml. It should work without problems. If there are any problems, please contact product support at: https://www.sysgo.com/support/
Same tool:
pikeos-upgrade-project-pre4 blah.perso.xml blah.profile.xml
It is currently not able to restructure the Depend="..." into <Condition condition="..."> properly. It does convert the stuff, but the nesting may be a little broken, because <Option> cannot appear inside <Condition>, so some manual work may be needed. You will probably see that you can simplify the condition structure when checking and fixing the generated file.
Otherwise, it converts quite well: it will convert types, even MULTIPLE, etc. Just the nested needs to be checked, normally.
Same tool, same call:
pikeos-upgrade-project-pre4 blah.option.xml blah.option.xml.new
Again, you need to check and edit like before. With options, you will run in even more <Condition>al stuff.
Plus, there is another corner case you need to fix: the option file often has a top-level Depends="..." which causes all content to be moved to a <Condition>. This usually also includes the <Parameter> the <Condition> depends on. You need to move that Parameter out of the <Condition>.