arm64_32 is AArch64, but using 32bit pointers, which is primarily used
on watchOS. Handling DYLD_CHAINED_PTR_32 fixups is necessary to
correctly load such binaries.
* Fbasic dev build test
* Add fbasic structure and enums
* Add fbasic op new
* Add fbasic op dup
* Add fbasic op free
* Add partial theory of float
* Add fbasic handlers to pure table default
* Add fbasic stringfy
* Add temporary float functions
* Add rz-float value and evaluation
* Add float json dump functions, todo add mode info in il_opdmp_* functions and compelete il_resolve_pure
* Complete fbasic il in il_export
* Fix detected error in compilation
* Add doxygen for il/definition/float
* Pack float_round function
* Add new version of rounding
* Add cast and convert, todo add 'float_get_sign' and 'float_get_no_bias_exp'
* Add comments and fix new version rounding
* Add util float functions
* Implement theory fbasic : cast, convert and round
* Remove bv_one warning
* Move basic functions into rz_util from definition/float
* Add two basic test cmp and extra format test
* Add test to fsucc, fpred and fneg
* Add round significant test case 1 and fix bug to pass it
* Test round significant and bug fixes
* Add 6 types test cases to round_and_pack function
* Finish round significant and round_pack test with bug fixes and signature changes
* Leave fround as unimplement now
* Pass all test to cast and convert
* Add SPDX header
* Add SPDX header
* Add round to integral float with test and bug fixes
* Add more tests to cast_sint and bug fixes
* Fix mem leak in cast_sfloat
* Replace the deprecated rounding method, fix bugs and pass tests
* Resolve some reviewed issues
* Fix most memleak and bug found
* Remove deprecated round_bv
* Remove unused functions to eliminate warnings
* Fix typo in comments
* Uncommented implmented operations
* Add test cases and bug fixes for fbasic theory in rzil
* Fix ret type of new_bitv_from_* from Bool to BitVector
* Add new float op from f32 and f64
* core: 'wow' command was replaced with 'wb'
* io: on_map_skyline returns false when nothing was done
Before this patch on_map_skyline returned true even when the callback
was not called on any part of the skyline. In other words,
reading/writing to an unmapped memory returned true.
This commit changes the behaviour to return false in those cases.
* analysis/esil: mem_read/mem_write were alway returning true
* use o malloc instead of oC
* librz: deal with seek errors
The arrow info is needed to print custom reflines, for example in
Cutter. In such a case, asm.lines will generally be false to not have
the lines also printed inside the text.
Parsing with sscanf turned out to cause an extreme slowdown. As only hex
values are being read, this can be done with RzUtil-provided
functionality. Relevant test cases exist in test/db/io/srec.
Regression from 7af1353166, but most
likely already broken before and only made visible by it: String flags
set from bin had their size set to how many bytes the string takes, but
were overwritten to a size of 1 after running aar.
* cmd: Fix summary of few "L" sub-commands
* util: start decluttering plugins handling
- Add Doxygen docs
- Remove unused API functions and make others internal only
- Use RzLibType enum for plugin type instead of int
* util: move dlopen/dlsym/dlclose to rz_sys
* util: make rz_lib_open/close return bool
* util: simplify rz_lib_add_handler and rz_lib_open*
* libs: implement rz_*_plugin_del to remove a plugin
* core: remove commands from RzCmd when removing plugins
* librz: call plugins 'fini' method when removing a plugin
* core/cmd: do not fail if the hashmap removal fails
When removing a group command, the inner command might have the same
name, thus it would result in trying to remove the same name twice.
Ignore it, as it is fine.
* librz: make RzCrypto plugins handling like the others
* librz/crypto: add rz_crypto_reset and use it
* librz: allocate the plugin structures before adding them
Passing arch and bits as args instead of commands ensures they take
effect earlier. This fixes these tests on platforms where the default
asm.bits is 32.
Applying a function signature with afs to a function was using a
combination of the existing args and wrong stack addresses to apply the
args to function variables. This generally resulted in incorrect
locations. Instead, we now use the info from the cc.
RzDebug.bits often has nonsensical contents set from all kinds of
places. Specifically, when debugging fat binaries on arm64, it had a
value not matching the debuggee when the reg profile was loaded,
resulting in the arm32 profile being used.
The right way to determine the profile is to do it after attach, when
the cpu type of the debuggee is known.
Patching the chained ptrs on the fly during every read as before turned
out to be a major bottleneck on larger binaries. So now we patch
everything once into a sparse overlay buffer, like it is already done in
ELF and classic Mach-O relocs.
Due to the DWARF parsing having been implemented before proper RzTypes,
it was still generating C-like strings that we then parsed into RzType.
This is fragile and was identified to be a major performance bottleneck
on OpenBSD/sparc64. Instead, we now generate RzTypes directly where
possible.
There are still cases left where strings have to be generated
from those types to be stored in an SDB. This should be changed in the
future too.
rz_type_pointer_of_type() is now also always creating a pointer type of
the given type. Before, it would have failed when the pointee was an
identifier that did not exist in the database. But this may be a valid
case e.g. while still building the database from DWARF and such
sanitization goes beyond what one would expect from
rz_type_pointer_of_type().
Even if the case is nonsensical in practice, we can't rule out the
possibility that a struct or union may contain itself as a member
(optionally through multiple recursion steps). This must be caught when
calculating the size of the type in order to not recurse infinitely.
Structs containing pointers to itself are not meant by this, as that is
perfectly valid and the size is determined without recursion.
However typedefs and arrays must be taken into account.
Variables on the stack are not identified by bp/sp+<offset> anymore, but
by their address from the bottom of the stack frame (RzStackAddr),
independent of how they are accessed.
So now there are only two kinds of variables: stack and register.
This required some major refactoring and other changes:
* RzAnalysisVar.isarg was removed. Whether a variable is an argument is
now specified implicitly by its storage location.
* Varsub of struct fields had to be rewritten so fields can be queried
by arbitrary stack addresses using the recently introduced sp
tracking, as the old approach to fill a list with all fields would not
work anymore.
* analysis.vars.stackname was removed, new behavior is more similar to
this being true before.
* Variables will not be created at stack+0 now, because the return
address is there. Before, vars were only created sometimes in such
cases.
* Variables created from bp offsets in x86 are not deleted anymore if
the function's bp_frame is false (see removed
rz_analysis_function_delete_vars_by_kind(fcn,
RZ_ANALYSIS_VAR_KIND_BPV); calls). This may lead to some
false-positive detected variables. Whether this really is a practical
issue is yet to be seen. At least there are no meaningful tests that
are broken by this.
* Applying variables from dwarf needed some fixes for determining the
correct stack locations of variables in order to write meaningful
tests. The handling is still not entirely correct for all
possibilities of dwarf info, but at least the changed/added test cases
are right and serve as a reference for future changes.
* Projects version 11 is introduced.
* afvb commands have been removed, afvs now handles all stack vars.
* Use signed long long for `src_readlen`, so we can check for -1
* Fixes Coverity defect 396947 and 356103
* Replace Unicode quotes with ASCII
* Ue signed structure in case -1 is returned (CID 396933)
* Need signed to check negative (CID 395560)
* Fix memleaks in `test_autocmplt_global`
* Related CIDs: 356309, 356307
* Set initial value of `ret` to false
* Initial function signatur and struct for `RzPty` API
* Change type annotation to borrow
* Implement `rz_subprocess_login_tty`
* Add fork mode enum and move pty struct; restructure ifdef guards
* Add rough implementation of forkpty
* Add `rz_sys` versions for `forkpty` family of functions
* Add forkpty code; Make changes according to review
* Implement forkpty properly and add unit test
* Add `test_interactive_pty`
* Add documentation
* Remove const in function argument
* Apparently, MacOS as non-const arguments for `openpty` and `forkpty`
* Add more tests for subprocess forkin using forkpty
* `test_interactive_pipe_pty`: Create pipes between the slave's PTY
and master and use them
* `test_interactive_custom_pty`: Use a custom PTY for the slave
* Add PTY functions exist checks for unit tests
* Move terminal param code inside the forkpty branch
* Add comment about why we need to check `n_bytes = 0` in `subprocess_wait`
* Remove forkpty with a specified `RzPTY`
* Also, no piping done in forkpty
* Add checks to disallow using `forkpty` with piping
* Make changes according to review
* No need to check proc file descriptors for -1 in `forkpty` mode, since
they *will* be -1
* No need to check `n_bytes == 0` in for `std{out,err}_pty`
* Extract pipe initialization code in `init_pipes` function
* Allow using custom PTY for `rz_subprocess_forkpty`
* Remove const qualifiers for termios and winsize arguments
* This makes the build pass on MacOS, which does accepts non-const
arguments
* Change `pid_t` to int so that it compiles on Windows
* Add empty implementation for PTY API for Windows
* Update function signature for `rz_subprocess_start_opt` for Windows
* Re-add `#define` for `pid_t` in `librz/socket/run.c`
* Move functions to remove duplicate definitions
* Allow specifying piping options even in `forkpty` mode
* Only expose the master fd for the streams which have pipe create
as their piping option
* Add comment about not using `master_fd` directly
* Fix types in definition for Windows
* Free the allocated `RzPTY` object
* Add const qualifiers for `RzPTY` argument
* Assign `fork_mode` and `pty` in all instances of subprocess opt start
* for good measure
* Compare master fd with -1, not 0
* Remove stray file
* Add second comment about the terminal attribute
* Overhaul the whole PTY implementation
* Now the caller can either use a pipe or PTY to send and receive
data from any of the streams
* Extra option `make_raw` to use the slave PTY in raw mode
* Add documentation for all the new functions
* Fix UAF bug
* Add error handling for `tcgetattr`
* Move the term param setting code
* Add documentation for the subprocess API and sys API
* Format according to clang-format
The stack pointer was previously already tracked during analysis for
variables and their accesses, but not stored in any sensible form for
further use. RzAnalysisBlock.stackptr and parent_stackptr were used in
some places, but they had no evident meaning.
Now we store the sp at the entry of a basic block and the difference
from that for every instruction inside the block to allow for efficient
querying of the sp value at arbitrary analyzed addresses.
RzAnalysisFunction.stackptr is now deprecated as its previous use was
primarily as a temporary accumulator, which is now handled locally, but
full removal of it would go beyond the scope here.
asm.stackptr visualizes both the absolute sp value and the delta of each
instruction in disassembly.
Changes in librz/analysis/p fix some test cases with the new tracking.
Introduces project version v10 with sp_entry/sp_delta instead of
stackptr/parent_stackptr.
* Fix searching for strings in `rz-bin` when the file format is invalid
* Always try with "any" bin plugin before giving up searching a file
* Add a test in `db/tools/rz_bin` for this
* Remove `EXPECT_ERR` from `rz-bin broken MZ` test
Tests and extends deriving paths into types at a given offset in the
following ways:
- Parameter for limiting the depth of the path
- Array support, e.g. `.member[5].anothermember`
- Typedef support
- Paths can be taken from arbitrary RzTypes, not only RzBaseType
- Removed the typename in the beginning of paths, which can be added by
the caller if needed
Even if the case is nonsensical in practice, we can't rule out the
possibility that the reference chain induced by a typedef may not
contain a loop, which could lead to infinite recursion when trying to
infer the actual underlying type.
Previously only the case where a typedef points to itself directly was
handled, but loops in multiple steps are possible too. So we introduce
rz_type_db_base_type_unwrap_typedef(), which handles the full
(multi-step) loop-aware unwrapping in one place and use it wherever
needed.
Even though many of the migrations were trivial, for all previous
versions X it is important to have both tests for X->X+1 migrations
and loading X in current rizin.
See leading comment in test_project_migrate.c for details.
Leaks detected by these tests were also fixed.
Fatmacho loading selects the file based on both arch and bits, so if the
preselected asm.bits was not 64 (which is the case on OpenBSD/sparc64
for some reason), these tests would fail.
This function only did a needlessly inefficient clone of
RzAnalysisFunction.vars, which can also be iterated directly.
Test differences are from different ordering of variables iterated, but
both are fine as there was no deliberate order before either.
A clone function that also copies the free function is very prone for
misuse and resulting double-free/uaf. Current cases were only fine
because they either did not have a free function or pathced it to NULL
on the result manually, meaning they do not need this behavior anyway.
The previous reading code was slow (performed too many
vm_region_recurse_64 calls) and broken, for example at boundaries of
memory maps. Details can be found in the code comments.
* Fix endianness issue by passing `cs_ppc_op.reg` instead of `.imm` to cs_reg_name.
The issue with using the `.imm` union member is that it is a `st64` while `.reg` is an enum (on SystemZ 32bit wide).
On little endian systems `.reg == .imm`. Not so on big endian systems. In which
case it is `.reg == .imm >> 32`.
* Set correct test for broken instruction `bne cr5, <target>`.
See issue: https://github.com/capstone-engine/capstone/issues/1914
- Introduce the following new methods:
* `rz_analysis_similarity_basic_block()`
* `rz_analysis_similarity_function()`
* `rz_analysis_similarity_basic_block_2()`
* `rz_analysis_similarity_function_2()`
* `rz_analysis_match_basic_blocks()`
* `rz_analysis_match_functions()`
* `rz_analysis_match_basic_blocks_2()`
* `rz_analysis_match_functions_2()`
- Remove `RzAnalysisDiff` and its usages
- Remove `librz/analysis/diff.c` entirely to and adds `librz/analysis/similarity.c`
- Remove a lot of unused structures and simplified the code by a lot
- Fix typo in levenshtein
- Remove `librz/core/gdiff.c` and `gdiff` functions since they were unused or only used in rz-diff
- Remove `difftype` (or "color") from `RzANode` which lead to dead code
- Decrease serialized data in projects.
- Add `RzAnalysisVarKind` and `RzAnalysisVarType` and replaces all the hardcoded values
- Remove command `agd`
- Remove all `rz_core_gdiff` usages from `rz_core.h`
- Refactor `rz_analysis_var_list` and `rz_analysis_var_count` due unused argument (`RzAnalysis` was not used)
- Add new method `rz_analysis_var_count_total` to simplify some usages
- Remove `rz_core_graph_diff`
- Add `typedef RzAnalysisFcnType` on `enum``used for function types
- Remove unused `RZ_ANALYSIS_FCN_VARKIND_LOCAL`
- Remove from `RzAnalysisFunction` the following variables:
* `fingerprint`
* `fingerprint_size`
* `diff`
* `diff_ops`
* `diff_thbb`
* `diff_thfcn`
- Remove from `RzAnalysisBlock` the following variables:
* `fingerprint`
* `diff`
- Remove `RZ_ANALYSIS_THRESHOLDBB` and `RZ_ANALYSIS_THRESHOLDFCN`
- Remove the following callbacks from `RzAnalysisPlugin`
* `RzAnalysisDiffBBCallback`
* `RzAnalysisDiffFcnCallback`
* `RzAnalysisDiffEvalCallback`
- Fix fancy table columns/rows when a color string is in a cell
- Bumped project version to 9
Previously, on macOS we were detaching properly when the user quit rizin
after an interactive `rz -d <file>` session, but not on
`rz -d -Qc <cmd> <file>`, which would leave behind a zombie process.
Now, the process is detached and keeps running, which also matches the
behavior on Linux. Hence the `dk 9` additions in the tests, to not
spoil the output with whatever the processes print.
Fixes the remaining test in test/db/cmd/cmd_c on big endian hosts,
tested on OpenBSD/sparc64.
The mask calculation was removed because it was unnecessary (only
comparing selected bytes anyway) and wrong by a factor of 8 (bits vs.
bytes confusion).
* Print a warning if config for VM was NULL.
* Add warning if register is not added to VM due to overlap.
* Update PPC register profile.
* Add vector and float registers. As well as some control and system registers.
* Enable to write values 4bit registers.
* PPC: Uplift most common instructions.
* Print warning if reserved SPR instruction is encountered.
* Update PC/LR addresses and add ca32 writes.
* Write cache needs to get flushed so load and store tests don't share the same memory.
* Add missing T/F branch mnemonics.
* Add Move to/from CR/CR0-7
* Fix TA address calculation for branch instructions.
* Add branch tests for branch mnemonics.
* Add XNOP
* NOP cache touch instructions.
* Add undocumented ATTN instruction to the not_implemented group.
* Add isel instruction.
* Implement CRCLR, CRSET, CROR.
* Add CNTLZ instructions.
* Add mcrf instructions.
* Add inacive test for cmpb.
* Add Load bytes reverse instructions.
* Add test and add address alignment to dcbz.
* Add eqv test
* Correct DIV and MUL operations
* Add div tests.
* Add tests and correct MT/MFXER
* Remove register ca32, ov32
* Remove explici setting of cr register because QEMU does not do it. Otherwise we get a mismatch in the trace
* Simplify carry set for add and sub. Sub instructions are exclusivly defined with addition. Hence no sub case needed.
* Unify BD and fix branch instructions. Fix: Check the single bit not the cr reg
* Document Conditional branches and replace NOPs with EMPTY
* Add mulli instruction to double word instructions.
* Fix ca set for shift instructions:
* ca value had to be determined before the shift happened. The wrong ca value was calculated if the src and target reg were the same.
* Replace NOP wit EMPTY.
* Use MSB isntead of SLT.
* Brought fixup of ADD and SUB instructions: The add and sub instructions had several issues which let to incorrect execution.
* The carry was incorrectly if three add operations happened (only the last add were checked, not both).
* The carry was incorrectly set if the src and target register matched.
* Same applies for the CR bit.
* It was too complex. Several local variables were introduced for this.
* Set result in local var, since it would change if src and target reg are the same.
* Remove MTMSR and MFMSR since it is too complex and untestable currently.
* Use unsigned int for shift. Otherwise the 0x1c shift produces a runtime error since 0xf is int as default.
* Fix mtxer: Only write flag bits.
* Let NOT_IMPLEMENTED macro return NULL.
* Mark st[wd]cx and l[wd]cx as not implemented.
* Increase dcache_line_size to 128 bytes.
* Fix cntlz for ppc32. m was set incorrectly, since it is 0 not 32 for 32bit cpus.
* Fix isel: Use op.crx reg instead of imm.
* Unify helper function names: Prependnig `ppc_` mark as IPI
* Add more "Move to SPR" cases.
* MULLI opeartes only on double word on 64bit CPUs.
* Most registers are now assigned the control register type and no longer show up in the ar command.
* Fix xor if dest and src registers match by saving result in local var.
* Fix BE/LE issue for Load BRX instructions.
* Fix shifts: Use only lower 6bits of n.
* User Pure local variables for ROT macros.
* Fix rldimi instructions.
* n was not inverted.
* more than 6 bits of n could be used
* Add 32bit emulateme tests.
* Add 64bit emulateme tests.
* Determine lg(v) in inline function.
* Calculate CR bit in C not in the VM.
* Check if `~mask = 0` and skip mask calculation if yes.
* Check for `sh == 0` and skip rotations where possible.
* Remove `la` instruction. `la` is a mnemonic for `addi`.
* Remove SPR instructions which are not supported by QEMU or not traced yet. For most set/read SPR instructions QEMU segfaults.
In case of SPR 1 (xer), 8 (lr) and 9 (ctr) the assembler resolves them to their mnemonics (mtxer, mtlr etc.). This means the code here is never reached.
To test the get_xer code MFXER was added again. The rz-tracetests will fail for this instructions (due to missing ca32, ov32). But this case is covert in an issue.
* Chane HexInsn -> HexInsnContainer in asm plugin.
* Make HexInsn -> HexInsnContainer in hexagon_disas.c
* Add function to set concatinate textual disassembly parts.
* Switch expressions, fix segfault.
* Print number of unhandle ops, if they can not be added to RzAnalysisOp.
* Add lookup function to determine sub-instructions for duplex iclass.
* Fix asm tests.
* Add memcpy for instruction containers
* Fix off by one.
* Fix segfault if the packet held only the instruction which is currently resolved. `hic->bin.insn` would be NULL in this case.
* Remove duplicate index name.
* Set instruction address.
* Clean up hic->parse_bits use
* Fix: .new registers were not resolved dur to too early return.
* Fix rebase mistakes.
* Add missing code
* Fix build error: Use sizeof(`text_infix`) since it is way smaller than `text`
* Initialize `HexInsn` inside `setup_new_hic`. Fixes NULL access.
* Mark RzAsm/RzAnalysis members as deprecated and fix typo.
ARM_THREAD_STATE64 fetches arm_thread_state64_t but our struct was
arm_unified_thread_state_t, which adds a header. The pc in the register
profile was hacked to account for the shift, but all the x-regs were
not, so x7 had the value of x8 for example. Now we just use the specific
state for 32 or 64.
This fixes basic single-stepping on macOS/arm64.
dbg->bits is affected by all kinds of things going on in rizin, in
particular it currently randomly switches to 16 when calling pd during
an arm64 debug session, causing the stepping code to break.
We now fetch the cpu type of the process once during attach, similar to
how lldb does it. They also use 0 for the "invalid" value, so we do that
as well.
The non-libuv tcp server implementation had several issues:
* Relied on EINTR or self-connect through global vars for breaking,
which did not work during recv() for example.
* Never retried on EINTR
* Called recv() only once, thus subject to fragmentation issues
To support breaking otherwise blocking socket calls, RzStopPipe has been
imported from chiaki (I am the sole author so I can relicense).
rz_socket_block_time() was also detected to be doing the exact opposite
than it should on Windows through the tests, and fixed.
* Replace on s390x failing tms320 c55x test with c55x+ test.
* Fix resource leaks of parse paramters.
* Revive unreachable code.
* Raplace test with one which has not a different asm string in Capstone v3.
* Rename `new` to `newt` since `new` is a keyword and should not be used as var name.
The warning about non-existent register types is useful for register
profile parsing errors, but this function is also used by e.g.
`ar`/`dr`, so the overloaded `ar` would print a warning when given a
specific register name for example.
This fixes the following minor issues:
* Errors like "Cannot create flag (...) because there is already (...)
flag" during debug startup
* Unnecessary seeks during .dm* causing io reads and e.g. delays of up
to 5s on x86 macOS during debug startup
* Flags are now created in the "maps" flagspace, rather than globally,
making them easier to clean up later
* Flags now have the same size as the maps and not +1
This introduces the ability to split asm strings into tokens and assign a type to them.
This means:
* Each asm strings is assigned a list of tokens. Each token points to a sub-string.
* A token has a type attached which describes it (e.g. register token, operator token, mnemonic token etc.)
* A generic parsing method is introduced which splits an asm string into tokens. The pattern it parses is `<mnem> <op> <op> ...`.
* Instead of the generic method, plugins can create the token strings on their own.
* An API method was introduced to help plugins implement their own parsing methods via regex patterns.
* Custom token strings are stored in `RzAsmOp.asm_toks`
Additionally the coloring of asm strings is now exclusively done via the token strings.
* For this a function was added to colorize tokenized asm strings according to their token types.
