Parsing with sscanf turned out to cause an extreme slowdown. As only hex
values are being read, this can be done with RzUtil-provided
functionality. Relevant test cases exist in test/db/io/srec.
Regression from 7af1353166, but most
likely already broken before and only made visible by it: String flags
set from bin had their size set to how many bytes the string takes, but
were overwritten to a size of 1 after running aar.
* cmd: Fix summary of few "L" sub-commands
* util: start decluttering plugins handling
- Add Doxygen docs
- Remove unused API functions and make others internal only
- Use RzLibType enum for plugin type instead of int
* util: move dlopen/dlsym/dlclose to rz_sys
* util: make rz_lib_open/close return bool
* util: simplify rz_lib_add_handler and rz_lib_open*
* libs: implement rz_*_plugin_del to remove a plugin
* core: remove commands from RzCmd when removing plugins
* librz: call plugins 'fini' method when removing a plugin
* core/cmd: do not fail if the hashmap removal fails
When removing a group command, the inner command might have the same
name, thus it would result in trying to remove the same name twice.
Ignore it, as it is fine.
* librz: make RzCrypto plugins handling like the others
* librz/crypto: add rz_crypto_reset and use it
* librz: allocate the plugin structures before adding them
Passing arch and bits as args instead of commands ensures they take
effect earlier. This fixes these tests on platforms where the default
asm.bits is 32.
Applying a function signature with afs to a function was using a
combination of the existing args and wrong stack addresses to apply the
args to function variables. This generally resulted in incorrect
locations. Instead, we now use the info from the cc.
RzDebug.bits often has nonsensical contents set from all kinds of
places. Specifically, when debugging fat binaries on arm64, it had a
value not matching the debuggee when the reg profile was loaded,
resulting in the arm32 profile being used.
The right way to determine the profile is to do it after attach, when
the cpu type of the debuggee is known.
Patching the chained ptrs on the fly during every read as before turned
out to be a major bottleneck on larger binaries. So now we patch
everything once into a sparse overlay buffer, like it is already done in
ELF and classic Mach-O relocs.
Due to the DWARF parsing having been implemented before proper RzTypes,
it was still generating C-like strings that we then parsed into RzType.
This is fragile and was identified to be a major performance bottleneck
on OpenBSD/sparc64. Instead, we now generate RzTypes directly where
possible.
There are still cases left where strings have to be generated
from those types to be stored in an SDB. This should be changed in the
future too.
rz_type_pointer_of_type() is now also always creating a pointer type of
the given type. Before, it would have failed when the pointee was an
identifier that did not exist in the database. But this may be a valid
case e.g. while still building the database from DWARF and such
sanitization goes beyond what one would expect from
rz_type_pointer_of_type().
Even if the case is nonsensical in practice, we can't rule out the
possibility that a struct or union may contain itself as a member
(optionally through multiple recursion steps). This must be caught when
calculating the size of the type in order to not recurse infinitely.
Structs containing pointers to itself are not meant by this, as that is
perfectly valid and the size is determined without recursion.
However typedefs and arrays must be taken into account.
Variables on the stack are not identified by bp/sp+<offset> anymore, but
by their address from the bottom of the stack frame (RzStackAddr),
independent of how they are accessed.
So now there are only two kinds of variables: stack and register.
This required some major refactoring and other changes:
* RzAnalysisVar.isarg was removed. Whether a variable is an argument is
now specified implicitly by its storage location.
* Varsub of struct fields had to be rewritten so fields can be queried
by arbitrary stack addresses using the recently introduced sp
tracking, as the old approach to fill a list with all fields would not
work anymore.
* analysis.vars.stackname was removed, new behavior is more similar to
this being true before.
* Variables will not be created at stack+0 now, because the return
address is there. Before, vars were only created sometimes in such
cases.
* Variables created from bp offsets in x86 are not deleted anymore if
the function's bp_frame is false (see removed
rz_analysis_function_delete_vars_by_kind(fcn,
RZ_ANALYSIS_VAR_KIND_BPV); calls). This may lead to some
false-positive detected variables. Whether this really is a practical
issue is yet to be seen. At least there are no meaningful tests that
are broken by this.
* Applying variables from dwarf needed some fixes for determining the
correct stack locations of variables in order to write meaningful
tests. The handling is still not entirely correct for all
possibilities of dwarf info, but at least the changed/added test cases
are right and serve as a reference for future changes.
* Projects version 11 is introduced.
* afvb commands have been removed, afvs now handles all stack vars.
* Use signed long long for `src_readlen`, so we can check for -1
* Fixes Coverity defect 396947 and 356103
* Replace Unicode quotes with ASCII
* Ue signed structure in case -1 is returned (CID 396933)
* Need signed to check negative (CID 395560)
* Fix memleaks in `test_autocmplt_global`
* Related CIDs: 356309, 356307
* Set initial value of `ret` to false
* Initial function signatur and struct for `RzPty` API
* Change type annotation to borrow
* Implement `rz_subprocess_login_tty`
* Add fork mode enum and move pty struct; restructure ifdef guards
* Add rough implementation of forkpty
* Add `rz_sys` versions for `forkpty` family of functions
* Add forkpty code; Make changes according to review
* Implement forkpty properly and add unit test
* Add `test_interactive_pty`
* Add documentation
* Remove const in function argument
* Apparently, MacOS as non-const arguments for `openpty` and `forkpty`
* Add more tests for subprocess forkin using forkpty
* `test_interactive_pipe_pty`: Create pipes between the slave's PTY
and master and use them
* `test_interactive_custom_pty`: Use a custom PTY for the slave
* Add PTY functions exist checks for unit tests
* Move terminal param code inside the forkpty branch
* Add comment about why we need to check `n_bytes = 0` in `subprocess_wait`
* Remove forkpty with a specified `RzPTY`
* Also, no piping done in forkpty
* Add checks to disallow using `forkpty` with piping
* Make changes according to review
* No need to check proc file descriptors for -1 in `forkpty` mode, since
they *will* be -1
* No need to check `n_bytes == 0` in for `std{out,err}_pty`
* Extract pipe initialization code in `init_pipes` function
* Allow using custom PTY for `rz_subprocess_forkpty`
* Remove const qualifiers for termios and winsize arguments
* This makes the build pass on MacOS, which does accepts non-const
arguments
* Change `pid_t` to int so that it compiles on Windows
* Add empty implementation for PTY API for Windows
* Update function signature for `rz_subprocess_start_opt` for Windows
* Re-add `#define` for `pid_t` in `librz/socket/run.c`
* Move functions to remove duplicate definitions
* Allow specifying piping options even in `forkpty` mode
* Only expose the master fd for the streams which have pipe create
as their piping option
* Add comment about not using `master_fd` directly
* Fix types in definition for Windows
* Free the allocated `RzPTY` object
* Add const qualifiers for `RzPTY` argument
* Assign `fork_mode` and `pty` in all instances of subprocess opt start
* for good measure
* Compare master fd with -1, not 0
* Remove stray file
* Add second comment about the terminal attribute
* Overhaul the whole PTY implementation
* Now the caller can either use a pipe or PTY to send and receive
data from any of the streams
* Extra option `make_raw` to use the slave PTY in raw mode
* Add documentation for all the new functions
* Fix UAF bug
* Add error handling for `tcgetattr`
* Move the term param setting code
* Add documentation for the subprocess API and sys API
* Format according to clang-format
The stack pointer was previously already tracked during analysis for
variables and their accesses, but not stored in any sensible form for
further use. RzAnalysisBlock.stackptr and parent_stackptr were used in
some places, but they had no evident meaning.
Now we store the sp at the entry of a basic block and the difference
from that for every instruction inside the block to allow for efficient
querying of the sp value at arbitrary analyzed addresses.
RzAnalysisFunction.stackptr is now deprecated as its previous use was
primarily as a temporary accumulator, which is now handled locally, but
full removal of it would go beyond the scope here.
asm.stackptr visualizes both the absolute sp value and the delta of each
instruction in disassembly.
Changes in librz/analysis/p fix some test cases with the new tracking.
Introduces project version v10 with sp_entry/sp_delta instead of
stackptr/parent_stackptr.
* Fix searching for strings in `rz-bin` when the file format is invalid
* Always try with "any" bin plugin before giving up searching a file
* Add a test in `db/tools/rz_bin` for this
* Remove `EXPECT_ERR` from `rz-bin broken MZ` test
Tests and extends deriving paths into types at a given offset in the
following ways:
- Parameter for limiting the depth of the path
- Array support, e.g. `.member[5].anothermember`
- Typedef support
- Paths can be taken from arbitrary RzTypes, not only RzBaseType
- Removed the typename in the beginning of paths, which can be added by
the caller if needed
Even if the case is nonsensical in practice, we can't rule out the
possibility that the reference chain induced by a typedef may not
contain a loop, which could lead to infinite recursion when trying to
infer the actual underlying type.
Previously only the case where a typedef points to itself directly was
handled, but loops in multiple steps are possible too. So we introduce
rz_type_db_base_type_unwrap_typedef(), which handles the full
(multi-step) loop-aware unwrapping in one place and use it wherever
needed.
Even though many of the migrations were trivial, for all previous
versions X it is important to have both tests for X->X+1 migrations
and loading X in current rizin.
See leading comment in test_project_migrate.c for details.
Leaks detected by these tests were also fixed.
Fatmacho loading selects the file based on both arch and bits, so if the
preselected asm.bits was not 64 (which is the case on OpenBSD/sparc64
for some reason), these tests would fail.
This function only did a needlessly inefficient clone of
RzAnalysisFunction.vars, which can also be iterated directly.
Test differences are from different ordering of variables iterated, but
both are fine as there was no deliberate order before either.
A clone function that also copies the free function is very prone for
misuse and resulting double-free/uaf. Current cases were only fine
because they either did not have a free function or pathced it to NULL
on the result manually, meaning they do not need this behavior anyway.
The previous reading code was slow (performed too many
vm_region_recurse_64 calls) and broken, for example at boundaries of
memory maps. Details can be found in the code comments.
* Fix endianness issue by passing `cs_ppc_op.reg` instead of `.imm` to cs_reg_name.
The issue with using the `.imm` union member is that it is a `st64` while `.reg` is an enum (on SystemZ 32bit wide).
On little endian systems `.reg == .imm`. Not so on big endian systems. In which
case it is `.reg == .imm >> 32`.
* Set correct test for broken instruction `bne cr5, <target>`.
See issue: https://github.com/capstone-engine/capstone/issues/1914
- Introduce the following new methods:
* `rz_analysis_similarity_basic_block()`
* `rz_analysis_similarity_function()`
* `rz_analysis_similarity_basic_block_2()`
* `rz_analysis_similarity_function_2()`
* `rz_analysis_match_basic_blocks()`
* `rz_analysis_match_functions()`
* `rz_analysis_match_basic_blocks_2()`
* `rz_analysis_match_functions_2()`
- Remove `RzAnalysisDiff` and its usages
- Remove `librz/analysis/diff.c` entirely to and adds `librz/analysis/similarity.c`
- Remove a lot of unused structures and simplified the code by a lot
- Fix typo in levenshtein
- Remove `librz/core/gdiff.c` and `gdiff` functions since they were unused or only used in rz-diff
- Remove `difftype` (or "color") from `RzANode` which lead to dead code
- Decrease serialized data in projects.
- Add `RzAnalysisVarKind` and `RzAnalysisVarType` and replaces all the hardcoded values
- Remove command `agd`
- Remove all `rz_core_gdiff` usages from `rz_core.h`
- Refactor `rz_analysis_var_list` and `rz_analysis_var_count` due unused argument (`RzAnalysis` was not used)
- Add new method `rz_analysis_var_count_total` to simplify some usages
- Remove `rz_core_graph_diff`
- Add `typedef RzAnalysisFcnType` on `enum``used for function types
- Remove unused `RZ_ANALYSIS_FCN_VARKIND_LOCAL`
- Remove from `RzAnalysisFunction` the following variables:
* `fingerprint`
* `fingerprint_size`
* `diff`
* `diff_ops`
* `diff_thbb`
* `diff_thfcn`
- Remove from `RzAnalysisBlock` the following variables:
* `fingerprint`
* `diff`
- Remove `RZ_ANALYSIS_THRESHOLDBB` and `RZ_ANALYSIS_THRESHOLDFCN`
- Remove the following callbacks from `RzAnalysisPlugin`
* `RzAnalysisDiffBBCallback`
* `RzAnalysisDiffFcnCallback`
* `RzAnalysisDiffEvalCallback`
- Fix fancy table columns/rows when a color string is in a cell
- Bumped project version to 9
Previously, on macOS we were detaching properly when the user quit rizin
after an interactive `rz -d <file>` session, but not on
`rz -d -Qc <cmd> <file>`, which would leave behind a zombie process.
Now, the process is detached and keeps running, which also matches the
behavior on Linux. Hence the `dk 9` additions in the tests, to not
spoil the output with whatever the processes print.
