* Add serenity to librt whitelist
This patch adds serenity to the list of operating systems that dont require librt to be explicity linked
* Disable PCRE2 JIT compilation
* Remove resolve_heap_tcache implementation
Serenity doesn't implement thread caching in its memory allocator, so tcache resolution is not applicable.
* Define rz_sys_pipe as pipe2 for serenity
* Add serenity identifier to rz_types.h
Define serenity as __UNIX__ like, and set RZ_SYS_OS to 'serenity' if __serenity__ identifier is defined
* Add zlib dependency for libzip
* core: Add unified API for describing addresses (resolves rizinorg#4993)
* core: Fix pxW and prompt tests by adding backward-compatible delta limit
This commit fixes the test failures caused by behavioral changes in the
unified address description API:
1. pxW tests failing (cmd_pxw, helloworld-gcc-elf):
- The old pxW code had a hardcoded 8192 limit for flag delta
- Added max_flag_delta option to RzAddrDescribeOptions
- rz_core_addr_get_flag_offset() now uses max_flag_delta=-1 (8192 limit)
2. prompt settings test failing (test/db/tools/rz):
- The old prompt code used spaces around +/- (e.g., 'entry0 + 1')
- Added use_spaces_around_delta option to RzAddrDescribeOptions
- Added rz_core_addr_get_flag_offset_prompt() for prompt display
* core/caddr: Add proper error handling and API improvements
- Add RZ_NONNULL annotations to helper function parameters
- Add null pointer checks after rz_str_dup() to detect allocation failures
- Change helper functions to return bool for error propagation
- Handle memory allocation failures in rz_core_addr_describe() by
checking helper return values and cleaning up on failure
- Use RZ_STR_ISEMPTY macro for consistent empty string checks
- Extract rz_core_addr_description_to_pj() as separate API for
converting RzAddrDescription to JSON format
* core: Add unified API for describing addresses (resolves rizinorg#4993)
core: Fix pxW and prompt tests by adding backward-compatible delta limit
This commit fixes the test failures caused by behavioral changes in the
unified address description API:
1. pxW tests failing (cmd_pxw, helloworld-gcc-elf):
- The old pxW code had a hardcoded 8192 limit for flag delta
- Added max_flag_delta option to RzAddrDescribeOptions
- rz_core_addr_get_flag_offset() now uses max_flag_delta=-1 (8192 limit)
2. prompt settings test failing (test/db/tools/rz):
- The old prompt code used spaces around +/- (e.g., 'entry0 + 1')
- Added use_spaces_around_delta option to RzAddrDescribeOptions
- Added rz_core_addr_get_flag_offset_prompt() for prompt display
* Make checks against defined Unicode points optional
* Allow to decode UTF-16 without writing the result.
* Remove PCRE2_NO_UTF_CHECK as default, since it can lead to undefined behavior.
* First refactor regex to support utf16 and utf32
* Add UTF-16-BE encoding function
* Add UTF-8 counting helper functions.
- One for counting the number of Unicode code points.
- The other to get the number of bytes required to represent the given UTF-8 string in UTF-16.
* Remove unused code
* Add UTF-8 to UTF-16 conversion function.
* Add type annotations
* Implement utf8 to utf32 string conversion
* Add UTF-16/32 versions of all other necessary regex functions for str search.
* Another regex refactor for utf16/32
* Add UTF16/32 regex matching tests.
* Implement still segfaulting (possibly JIT double usage) regex search.
* Duplicate match_first functions to reduce necessary branch predictions.
* Reduce number of required branches for encoding UTF16/32 to one.
* Duplicate match_all_internal functions to reduce necessary branch predictions.
* Fix too early free
* Only allocate match vector when needed.
* Fix: use code point size of buffer.
* Add missing return
* Normalize pointers to UTF16/32 strings to use proper code point with
* Also replace spaces with in utf16/32
* Add an additional host endian tests for UTF16/32 string encodings.
* Ensure thread savety.
JIT compiled patterns need to be owned by a single thread.
For the search we need to clone it.
JIT matching structures are optionally cloned as well.
* Enforce NO_UTF_CHECK in regex search.
This improves performance and currently is
default because we always match on binary data.
* Fix matching of UTF strings which are not suported by direct buffer matching.
PCRE2 only supports matching against memory which is aligned
to a code point width of the encoding.
These changes prevent taking the fast (direct matching with PCRE2) path
and use the slow string search path if the alignment doesn't match
the UTF string encoding.
To not complicate the change and additional alignment member
is added to each searched string in the search collection.
* Create search hit description on the stack
* Unset complete JIT matching if user provided custom jflags.
* Document what passing NULL to copy function pointers does.
* Replace the retarded idea of tracking offsets with a hashmap with a linear buffer.
This improves performance something like 10x.
* Remove const for the non-JIT builds.
* Remove additional flags for skip checking.
It is not needed because each decoded character is checked for printablity below anyways.
* Enfore no setup of IO mem with 0xff
* Fix: Set JIT complete flag for multi regex patterns
* Fix heap.
* Add note about worsed performance path.
* Fix unit test with string terminated by undefined code point.
* Run clang-format
* Fix order of arguments
* Add warning about string search with encoding=guess to tests.
* Fix string lengths, they no longer count the final invalid code point.
* Fix endian macro on Windows
* Fix NULL dereference
* Fix number tests
* Use endianness check not dependent on stdbit
* Fix type annotations.
* Reintroduce rz_str_len_utf8char
* Fix command description.
* Fix and unify RZ_SYS_ENDIAN macros.
- Don't allow unhandled architectures anymore.
- Check endianness for Sparc and PPC using non GCC/Clang compilers.
- Fix several endianness checks using the value instead of the macros.
* Fix tests
* Apply review comments.
* Added build files [Capstone to Zydis]
* x86 Analysis [Capstone to Zydis]
* Changed x86 RzIL [Capstone to Zydis]
* Changed asm and arch files [Capstone to Zydis]
* Compilation, build error and test fixes [Capstone to Zydis]
* Test changes [Capstone to Zydis]
* Remaining changes [Capstone to Zydis]
* Added BE support [Capstone to Zydis]
---------
Co-authored-by: tushar3q34 <tushar3q34@gmail.com>
* Add xtensa arch capstone plugin
* Remove xtensa-gnu
* fix: `asm.cpu` is not set correctly, when there is no “,” in asm plugin -> cpus
* fix core_disassembly when n_bytes==0
* fix xtensa_analyze_op_esil
* fix xtensa esil pc mod
* fix xtensa esil l32r
* fix xtensa esil call0
* fix xtensa esil extui
* fix xtensa esil store
* fix xtensa stack inc and ret
* keep `asm.cpu` when valid in cb_asmarch
* fix cb_asmarch
* fix max op size
* xtensa: add cpu select
* xtensa: mark PUSH POP
* xtensa: set src dst
* xtensa: set dir
* xtensa: fix stackframe
* Setting GNU99/C99 standard for subprojects
* Remove use of `add_global_arguments`
* Added extern "C" linkage specification to some public headers of rizin
* Changed the meson_git_wrapper.py-script to explicitly specify the output-path
* Made the RIZIN_BUILD_PATH conditional in the integration-test meson-build
Co-authored-by: amibranch <amibranch@users.noreply.github.com>
* Bumps Capstone version to newest Capstone next (beyond first v6-Alpha1).
* Fixes leaks
* Fixes build and change to AArch64 and SystemZ compatibility headers.
* Marks M68k test as broken (see commit message).
* Fix AArch64 and SystemZ tests
* Handle op.size == 0 for x86 IL ops
* Replace OpenBSD regex library with PCRE2.
PCRE2 has way better performance than the OpenBSD
library (something around 20 times faster).
The following flags are enabled for every pattern:
- PCRE2_UTF
- PCRE2_MATCH_INVALID_UTF
- PCRE2_NO_UTF_CHECK
All the others are optional.
Changes made:
- Adds PCRE2 as subproject.
- Changes the API away from POSIX to PCRE2.
- Edits many regex patterns because:
- ' ' is skipped in patterns, if the EXTENDED flag is set for matching. '\s' must be set now.
- '.' doesn't match newlines by default.
- Changes the API so matches and their groups are bundled into PVectors.
- Moves the regex component to rz_util.
* Fix cross build - add copy of PCRE2 dependecy
Meson currently doesn't support subprojects to be native and non-native at the same time.
See: https://github.com/mesonbuild/meson/issues/10947
Unfortunately, sdb depends on rz_util which in turn depends on PCRE2.
Excluding PCRE2 from the native build makes linking of rz_util not possible anymore.
Adding it, will make Meson complain that the dependencies cannot be mixed.
Hence, we compile a copy of PCRE2 for the native build if required.
This commit refactors the AArch64 and partially the ARM
plugin to make it Capstone v6 compatible.
Due to the big API changes in Capstone v6 several changes
had to be made.
Because we need to be compatible to Capstone v4 and v5
many include guards are added as well.
Overview of changes done:
**ARM**
- Instruction alias were introduced. This leads to different
decoding and analysis paths taken for certain instructions.
Some alias have their IL code generated like the real
instruction now (no special handling needed anymore).
This change is responsible for many changes you'll encounter.
- The operand details of each instruction are now always the
one of the real instruction. Also for alias.
For example, if "MOV <Wd>, #<imm>" is an alias for `ORR <Wd>, WZR, #<imm>`,
the details by CS hold all three operands of "ORR".
Before, they held only the two of "MOV".
- Several bugs in variable and argument generation
were fixed. Especially the default variable width
for ARM Thumb was changed to 32bit instead of the 16bit.
**AArch64/ARM64**
The changes listed above for ARM, also apply to AArch64.
Additionally:
- Capstone v6 changed the name ARM64 now everywhere to AArch64.
To be compatible with Capstone v4/v5 AArch64 names must be wrapped
into macros which resolve the name, depending on the CS version used.
- Capstone v6 is now more consistent with
register real and alias names.
From now on we use the register alias by default.
**List squashed commit messages:**
[AArch64 CS v6 BEGIN] Change subproject config to use cs-auto-sync-aarch64 branch
Replace ARM64 with version sensitive macros.
Exclude alias if CS version >= 6
Update access to writeback member
Exclude instr alias from inclusion
Update memory operand printing to json.
Enable real instr. detail only for AArch64
Set correct arch name in meson.build for CS
Fix U/SBFM instructions and their alias.
Mark parameters with RZ_OUt/BORROW
Optimize register extension to skip some, if the width already matches.
Adapt width and lsb of U/SBFM alias instructions (ImmR and ImmS are from U/SBFM).
Fix tests correct semantic buy bad syntax
Pass alias MOV instructions to mov()
Handle CSET and CSETM alias
Fix lsl, lsr and asr by handling them as alias.
Fix mov alias.
Handle TST alias
Fix CNEG, CINV alias
Fix bfi and bfxil alias.
Fix sign extensions.
Fix compare instructions.
Fix NEG, NGC, NGCS, NEGS, MVN
Fix CINC
Fix multiply instructions.
Fix ROR
Run clang-format
Handle CMP for ESIL
Handle new position of memory disponents of post index operands.
Fix post-index operations.
Add missing writeback checks for Post and preindex
Handle UBFM and SBFM alias
Handl BFM alias
Handle CMP, CSET and CINC alias
Update meson file of for cs-aarch64 branch
Fix asm tests. Use reg alias now.
Fix condition confusion and incorrect operand usage.
Fix plf test.
Run clang-format
Use register alias in tests
Add support for fp and lr reg alias assembly.
Use reg alias in test
Rename cond tranlate functions r2 -> rz
Fix condition check which assume 0 == invalid.
Fix issues intruduced by rebase
Set CS commit to current next branch.
Rename ARM64 -> AArch64
Add missing source file to meson.build
Remove DisassemblerExtension.c file for CS v5
Update to newest capstone next branch
Bump up CS version
REVERT ME: Get Capstone v4/v5 via git clone until new tars are released.
Wrap setting of CS_DETAIL_REAL into CS version check
Add maybe-unitialized to Capstone C args.
Fix CS pre v6 build by adding guards.
Use reg alias now printed by default by CS.
Bump CS version to most recent next.
Fix build errors due to stircter alias handling in ARM.
Fix RzIL tests introduced by alias introduction to ARM.
Fix ESIL bugs introduced with ARM alias introduction.
- stackptr hasn't been set for POP and PUSH
Add support again for Thumb1 pop/push
Handle PUSHW and POPW alias
Update test case
Add more POP and PUSH alias and enrich detail for other versions of them.
Fix incorrect mem access width guesses for ARM thumb.
Set POP return info if it writes to PC
Fix tests about default var size and POP mem write direction.
Bump CS version to newest next.
Fix incorrect tests.
- TriCore: Functions were in ro section.
- Default arg width in ARM thumb is 32bit.
Revert check for a set stackptr.
stackptr is used in different ways:
1. Safes the offset from the stack frame base.
2. Is interpreted as somthing else for x86 and I cannot find out what, in a reasonably time.
Hence we cannot use it here consistently.
Remove check for non existing ARM_GRP_RET in CSv5
Fix incorrect stack offsets of variables.
'push <reg-list>' instructions for which the second register was the FP,
reset the stackptr variable to 0. This led to wrong bp offsets in the variable names.
In this case it was +0xc.
Bump CS version.
Add copy of meta-programming macros for capstone-sys build.
Update capstone-next.wrap
Use bracket-less met-programming macro to fix Windows build warnings.
Update wrap files for Capstone with branch names
Add new meta-programming macro
Add workaround for MSVC pre-processor bug.
* Remove macros checking for CS_API_VERSION 3
* Remove CS 3 from meson_options and subprojects.
* Remove Capstone v3 from CI
* Remove CS_API_VERSION >= 4 checks since always true from now on.
* Add CSv5 workflow to CI.
* Handle ARM pre v6 build errors.
* Fix PPC build errors for CS < v6
* Fix correct macro name
* Remove wrong include.
* Fix fallthrough warning
* Fix getter macros for WB and post-fix flags.
* Fix faulty macro name
* Check for all float extensions for vabs.
* Add return to assert cases.
* Update cs to 46154e8605aaefdcca5fecf4ea88b92db5a40ad3
* Remove cs3 wrap file.
---------
Co-authored-by: wargio <wargio@libero.it>
ARM
- Updated ARM identifiers and API changes.
- Fixed shift amounts to be always decimal.
- Fixed incorrect usages of memory disponents.
- Fixed restoring of condition codes.
- Updated IDs for new ones.
- Fixed 8-byte stores.
- Added flag checking for MOV with shifts.
Capstone Integration
- Removed capstone-auto-sync subproject and replaced with next.
- Checked for NEON features.
- Used https://github.com/capstone-engine/capstone/pull/2122 for better system operand support.
- Distinguished between 32 and 64-bit cc checks.
- Checked for CS API version >5.
- Checked for CS_NEXT_VERSION instead of CS_API_MAJOR.
- Fixed VSTn and VLDn instructions to use corrected memory operands.
- Excluded multiple instruction aliases that are no longer valid IDs.
- Used CS_NEXT_VERSION as include guard.
- Added CS v6 include guards.
- Added CS v6 support to branch conditions for Rzil.
- Added more CSv6 guards.
PPC
- Fixed: Compare instructions do not use the branch predicate.
- Fixed conditional and CTR checks.
- Fixed MTSPR and MFSPR instructions.
- Excluded more branch aliases no longer present in v6 from switch cases.
- Fixed incorrectly assigned variables.
- Fixed condition checks for branches.
- Used ITE for condition checks, to prevent unnecessary reads of CTR or CR.
- Fixed rzil tests with new semantics using register 0.
- Updated include guards for Capstone versions to use CS_NEXT_VERSION.
- Initialized spr_name to prevent uninitialized use.
- Used mem.offset register for CSv6.
- Always used real operand details.
- Used mem operand for DCBZ.
- Fixed tests that are semantically identical.
- Handled LI LIS alias.
- Fixed rzil tests (with simplified semantics).
- Fixed branch alias with new cond test method.
- Handled clrl. alias.
- Handled SL/SR alias.
- Fixed conditional branches in ESIL.
- Fixed possible multiplication result overflow.
- Added RZ_NONNULL.
- Added 0 register.
- Handled LIS alias.
- Fixed: Print crX reg name in CS v6.
- Handled all general branch instructions into a single case statement.
- Fixed no semantic issues in tests.
- Added newly discovered calls.
- Moved direction check to inline function.
- Added link of root cause for broken test.
- Added QPX support.
- Set Capstone next branch to latest commit.
- Fixed uninitialized warning.
- Set CS next branch to newest commit.
Functions like MEM_read32 could still perform unaligned accesses on
OpenBSD/sparc64 with gcc, possibly related to gcc bug 51628. The
solution is to switch to a safe memcpy-based solution and let the
compiler do any optimizations.
