Plugins do not create register-based variables themselves anymore, but
they are derived from the register profile. However not all registers
may be bound and not all variables may be actual registers. The concrete
relations between an RzILVM and registers is described by an
RzILRegBinding object. It is simply a list of register names and
variables are created of the same names. When stepping with aezs, the
registers are synchronized automatically. From now on, aezv is primarily
useful for debugging, but interacting with emulation from the user side
should be done with ar.
This adds an RzBuffer implementation that binds against RzIO's
rz_io_read_at/rz_io_write_at in order to access the mapped memory. This
is in contrast to the previous io RzBuffer, which has been renamed to
..._io_fd and which reads directly from a single file descriptor without
mapping.
To keep bf working, bin_bf now maps an area of zeroes because
otherwise the initial memory contents in RzIL would read as 0xff.
The root type of a lifted op is now simply a single RzILOpEffect, which
can be for example a chain of seq ops. This is in line with what BAP
uses.
For convenient creation of sequences, `rz_il_op_new_seqn(ut32 n, ...)`
is a drop-in replacement for `rz_il_make_oplist(ut32 n, ...)`.
This brings our concept of effects in line with BAP and fixes issues
like values being evaluated prematurely in seq. The main point is that
ops themselves are the effects, rather than returning them to be
evaluated later.
It is often useful to get a slice of some maximum len, also accepting
smaller results if the source is not large enough.
This also makes the function always operate on byte indices. Before,
it was inconsistently skipping ansi codes sometimes.
This also fixes potential oob writes in panels.c
Add pac and sig creation with options & compression via commands and via new rz-sign
Add missing arch (arc, pic, riscv, spc700 xtensa, and more)
Side note:
on other tools public functions and references are generated in a sequential
way besides the first one which is the first to be assigned.
since we do not search for sequential references, then we can avoid to
add extra public or referenced functions
Extra:
* Refactoring rz_analysis_mask definition
* Add rz_str_startswith_const and rz_list_iter_get_next_data
ESIL Pins were an untested feature to run a command when emulation hit a
specific address. This can be solved in other ways, e.g. scripting.
Project version 7 has been introduced for this change.
* Remove `RIZIN_EXTRAS` because useless
Extras plugin should be treated as any other regular plugin, they should
not have their own directory. I believe we did not have anyway any
plugin using that directory.
* Do not use `<version>` in `/usr/share`
Users can only install one version of Rizin at a time anyway, thus
splitting the `/usr/share/rizin` directories does not really bring
multi-version install possible. At the same time, we stick to what's
common for other tools, which do not use version in `/usr/share`
* Move plugin directory to `/usr/lib64/rizin/plugins`
* Remove `RZ_HOME_BIN` env var because not useful anymore
* Introduce `rz_util/rz_path` to provide rizin paths
* Use `rz_path_system_sdb_types` whenever it makes sense
* Use some others `rz_path_*` functions instead of custom ones
* Use `rz_path` APIs for themes, fortunes and signatures
* Remove `rz_sys_prefix`
* Reduce the number of defines needed to handle home and system paths
* Reduce the number of `rz_path` functions
* Make the lib debug depending on log level and `RZ_LOG_DEBUG`
* Refactor a bit how directory of plugins are loaded
* Keep supporting old plugins directory for at least 0.4.0
Adds RZIL_OP_BITV and removes RZIL_OP_INT due host limitation
Fixed type RZIL_OP_JMP to work and apply effect.
Implemented rz_il_bv_new_from_st32 rz_il_bv_new_from_st64
Refactors the brainfuck RzIL code.
Implements aezv which returns the RzIL VM status and the output can be changed via rzil.status.compact=true/false
Refactors the oldshell help messages to clean some useless code
Refactors and optimize RzILBitVector methods
Fixes rz_il_bv_as_string, rz_il_bv_hash and implements rz_il_bv_as_hex_string and rz_il_bv_set_from_ut32/ut64
Assigns all the handlers to op_handler_table to avoid null derefs and to notify which operations are unimplemented
Implements rz_il_handler_logical_and, rz_il_handler_logical_or, rz_il_handler_logical_xor and rz_il_handler_cast
* Cleanup RzAnalysisRzil when freeing RzAnalysis
* Fix str leaking in rz_core_write_string_at
* Fix buf leaking in rz_core_yank_paste
* Fix memleak in json unit tests
* Fix features leaking in RzBinInfo
* Fix features leaking in RzAsm
* Fix memleaks in RzTable
* Fix test_str_search memleaks
* Fix memleak in test_io.c
* Fix memleaks in RzBuf and test_buf.c
* Fix memleaks in test_cmd.c
* Fix memleaks in test_config.c
* Fix memleaks in RzCons
* Fix hashes leaking in test_bin.c
* Fix memleaks in test_id_storage.c
* Fix memleaks in test_il*.c and adjust APIs
* `<cmd>?` allows to list the sub-commands of <cmd>. If <cmd> has no
sub-commands, the full help of <cmd> is printed instead.
* `<cmd>??` allows to always list the full help of <cmd>. Useful when
<cmd> is both the name of a group and a command itself.
* output modes are only listed as part of the full help and not
considered as sub-commands.
* fix a bug where `<cmd><mode>?` was just printing the list of
sub-commands of <cmd> instead of the full help of <cmd><mode>.
* Rename `rz_str_escape_latin1()` to `rz_str_escape_8bit()`
* test_serialize_analysis.c: Change string from "latin1" to "8bit"
* `bin.str.enc`: Rename `latin1` to the more generic `8bit`
* Do not escape strings during the search
* Port `C` (metainformation) commands to the rzshell
* Improve string autodetection
* Remove `Cr` and RZ_META_TYPE_RUN
* Escape `\x1b` as `\e` character in strings
* Move string encoding options to `RzStrEncOptions` struct
* Add `esc_double_quotes` option to `RzStrEncOptions`
* Do not filter out unprintable characters for uppercase check
* Use `esc_double_quotes` when printing the string between '"' in disasm
* Use signed chars on all platforms/architectures
Co-authored-by: Riccardo Schirone <sirmy15@gmail.com>
The new intermediate language is based on the BAP (Binary Analysis Platform) Core Theory.
It operates on SMT-like bitvectors and their arrays to represent values and memory.
Apart from that, it has also representation for the data and code side effects.
* Rewrite PDB parser to become endianess-independent
* Use type pretty-printing API
* Use `RzCmdStateOutput` instead of `RzOutputMode` to solve invalid JSON issue
* Move PDB-related print functions to `cpdb.c`
* Use flag API instead of calling commands from the code
* Separate printing from parsing and applying the type information
* Delete useless `main.c`
* Replace `rz_type_db_base_type_as_string` with `rz_type_db_base_type_as_pretty_string`
* Use the new type pretty printing API to pretty print base types
* Add option to return newline terminated string by pretty printing API
* Add `RZ_TYPE_PRINT_END_NEWLINE` option to RzTypePrintOpts
* Improve printing in cases of empty structs/unions/enums
* Initially, the separator was printed between the curly breaces
irrespective of whether the struct/union/enum is empty. Now the
separator is not printed if the given type is empty
* Add pretty printing support for typedefs
* Also, add another type printing option to show typedefs. To be used in
`rz_cores_types_typedef_as_c`
* Refactor `.*_as_c` functions to use the new type pretty printing API
* Add tests for typedef and `unknown_t` for pretty printing API
* Add `rz_type_db_base_type_as_string` back. Update tests.
* Use ternary operator to make expression more concise
* Add doxygen documentation for `rz_type_db_base_type_as_string`
* Use `rz_vector_len` to determine if empty, instead of convoluted logic
* Remove redundant `RZ_NONNULL` for RzTypeDB
* Fix integration tests for type printing
* Update Doxygen documentation
* Fix failint enum printing itegration tests
* Fix according to clang-format
* Add function to check if a register is a system register
* Only checks for ARM64 system registers as of now
* Check for system registers in ARM64 when printing registers using "dr"
* Does not completely remove the unnecessary register, but removes a
huge number of useless registers printed by "dr"
* Rename `RZ_REG_TYPE_ALL` to `RZ_REG_TYPE_ANY`
* ADd register types for "sys" and "pauth": `RZ_REG_TYPE_SYS` and
`RZ_REG_TYPE_PAUTH` respectively
* Remove `rz_reg_is_system_reg`
* Add `sys` and `pauth` register profiles
* Update debug tests to account for new arenas and register profiles
* REplace "pauth" with "sec" reg profile
* Update integration tests to account for new register profiles: `sys`, `sec`
* Migrate project from v5 to v6
* Add serialization for debug (RzDebug) instance (only for breakpoints as of now)
* Replace commit hash once the PR is merged into dev
* Add new namespaces added in v6
* Refactored string search
string search logic moved from librz/bin/bfile.c to
librz/util/str_search.c
* UTF-16BE and UTF-32BE support
* Decoupled rz_str_search with rz_bin
* Changed parameter order in rz_scan_strings
* Fixed needle reset in rz_string_search
Now when the string search fails, needle is restored to original_needle+1,
i.e., we continue the searching process starting from the next byte.
* Changed heuristic for UTF BE detection
one more byte in for the detection (6 instead of 5)
* Fixed unit test in str_search
* Refactored rz_scan_strings and better UTF16-LE and UTF16-BE detection
Now if there is an ambiguity between UTF16-LE and UTF16-BE, we take the
longest one.
* Changed UTF BE heuristic
If there is any ambiguity about big-endian or little-endian, always
choose little endian
* Small format fixes in str_search
* Added preferred endianess switch in RzUtilStrScanOptions
* Fixed str_search regression tests
* Start to create some RzBinObject specific API
Adding some unit tests
Fix for new relocs storage system
Fix test_bin unit test for oob-write
* Add `iA` command to newshell
Add `ie`/`iee` commands
Add support for `is`/`iE` commands
Add support for `ii` command
Add support for `il` command
Add support for `iM` command
Add support for `ir` command
Add support for `iS`/`iSS` commands
Add support for `iz` command
Add support for `i` command
Add support for `ic` command
Add support for `iC` command
Add support for `ih` and `iHH` commands
Add support for `iI` command
Add support for `iL` command
Fix descriptions of commands already converted
Add support for `idp` commands
Add support for `iD` command
Add support for `im` commands
Add support for `iR` command
Add support for `it` command
Add support for `iV` commands
Add support for `iw` command
Add support for `ix` commands
Add support for `iZ` command
Add support for `ik` command
Add support for `is.` command
Add support for `iS.` command
Reduce arguments in print_arch
Few Doxygen and ownership annotations
Add support for `iS=` command
Add support for `izz` and `izzz` commands
Add support for `ib` command
Add support for `ic*` command
Put section flags as the last field of the table
* Proper language detection.
* Rewritten blang.c
* implemented `icc/icm/icf <class name>`
* ignore blocks when printing langs
Remove num from `iA` output
* Fixed flags on java and added class column
Add support for `ia` commands
Fix `iCj` command
* RzBin: add rz_bin_object_sections_mapping_list API
* Fix rz-bin for new API
Remove unused va var in rz-bin.c
Fix `iA`/`-A` output
* implemented classdump_cpp
* Support namespaces
Use `icc`, `icl` does not exist anymore
Update output of broken tests for `icc`
Add `rz_return_*()` asserts in new `*_print()` functions
* Provide rz_core_bin_print API
* Select which parts to print also based on the mode
* Return bool from all `rz_core_bin_*_print()` API
* Use RzBinFile in new `rz_core_bin_*_print()` API
* Restore `dbg_dmi` test
* Remove oldshell handlers for `i` commands
* Remove `ieq` call in rizin, not needed
* Allow NULL `bf` in `rz_cmd_info_handler()`
* Re-add support for RZ_BIN_PREFIX for symbols/sections/segments/imports/relocs
* Move `ib` command to `obR`
* Show 'N/A' in `i` command when info are not available
* Assert on bf->o, just to be sure
* Make error messages clearer in `idp` commands
Co-authored-by: wargio <wargio@libero.it>
Co-authored-by: Giovanni <561184+wargio@users.noreply.github.com>
* Add autocomplete function and cmd arg type for global variables
* Add unit tests for global var auto completion
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
* Add rz_analysis_var_global_*_flag functions
* Add init, destroy and update functions
* Incorporate flag handling functionality in the existing functions
* Add `avgm` command to remove global variable by name
* Add error logs when trying to find a non-existing global variable
* Add unit tests for flag functionality of global variables
Some commands might have one of the output modes as default one. If
that's the case, RzCmd should be aware of that so that it can
initializes everything needed for that default mode.
For example, if command `c` supports TABLE, JSON and QUIET and you want
to have `c`(with no suffix) print the TABLE mode, the parser would have
to initialize the RzTable data structures anyway, so it must be aware of
it.
* Fill `op->reg` for ARM64 `br`/`blr` analysis
* This fixes software single-stepping on `br`/`blr` instructions on ARM64
* Fill `op->reg` for ARM `bx`/`blx` analysis
* This fixes software single-stepping on `bx`/`blx` instructions on ARM
* Set `bx` as `RZ_ANALYSIS_OP_TYPE_RJMP`
* Fill `RzAnalysisValue->type`
* Use only `RzAnalysisValue` in `rz_core_link_stroff()`
* Fill `op->ireg/disp/scale` for ARM `ldr` analysis
* This fixes software single-stepping for `ldr pc` on ARM
* Fix riscv inverted src/dst values for store instructions
* Fix riscv analysis with buffer of size 2
* Add unit tests for `RzAnalysisOp` and `RzAnalysisValue`
Co-authored-by: Giovanni <561184+wargio@users.noreply.github.com>
* Add copyright and license to all librz/include/*.h files in dep5
* SPDX headers for most include files except bin/arch/debug
* Add SPDX headers in asm/arch/{8051,arm}
* Add SPDX header for shlr/gdb include files
* Add SPDX header for shlr/bochs include files
* Add SPDX header for librz/reg
* Add SPDX header for librz/debug/p/native/xnu
* Add SPDX header for librz/debug/p/native/windows
* Add SPDX header for librz/debug/p/native/reg include files
* Add SPDX header for librz/debug/p/native/maps include files
* Add SPDX header for librz/debug/p/native/linux include files
* Add global var concept
* Doxygen & ownership & delete `delete_all` & argc check
* Use RBTree to manage the address of global variable
* Kill comment and access & use ht_pp_new
* Add unit test and regressions test
* Remove GlobalVarNode
* Implement global variable [de]serialization
* Moved `rz_cmd_state_output_array_start`
* Added complex type tests
* delete/get at or in address & convert type links to global variable & minor fix
* Remove `size` field
* Migration of typelinks to global var
* Use exists type parser
* Recover typelink unit test
* Add more leaf_types for TPI parsing
* Get type list by its exact type index
* Print newly added LF_CLASS_19 and LF_STRUCTURE_19
* Newly added LF_CLASS_19 and LF_STRUCTURE_19 should also be formatted
* Newly added LF_CLASS_19 and LF_STRUCTURE_19 should also be parsed
* Fix wrong data type `Sval`, it should be `Numeric`
* Add LF_BCLASS LF_VBCLASS LF_IVBCLASS
* Add missing `LF_VFUNCTAB` leaf type
* Integrate PDB information into new types database
Co-authored-by: Basstorm <fuli114.ml@gmail.com>
- Removed oldshell version of `t` (types) commands
- Removed `tk` command because types storage switched to the hashtable
- Added `tfc` command to show the calling convention of the function or set it
- Added `tsl` (RZ_OUTPUT_MODE_LONG) to show structure members' sizes and offsets
- Added `tul` (RZ_OUTPUT_MODE_LONG) to show union members' sizes
- Switched the types storage to the hashtable instead of SDB
- Added load and export from/to SDB for standard type libraries and serialization for Projects
- Changed all `char *type` occurences to the proper `RzType`
- Added `RzCallable` type for the function/class method types
- Changed all function types API to use the `RzCallable` API
- Switched the function types (`RzCallable`) storage to the hashtable instead of SDB
- Added a connection between `RzAnalysisFunction` and `RzCallable` - `rz_analysis_function_derive_type()` API function to derive new `RzCallable` type from the `RzAnalysisFunction` even if it doesn't exist in the RzTypeDB.
- Moved type links to the RzAnalysis where they belong
- Removed TCC and MPC-based parsers
- Added Tree-Sitter-based parser for C grammar (see `librz/type/parser/*`)
- Reworked type databases in `librz/analysis/d/` to be consistent with C standard, libc, platform-specific definitions
- Switched the type links from `char *` to `RzType`
- Switched the type links from SDB to the hashtable
- Added concept of the "type paths", e.g. `a.b[20].c` where `b` is a member of `a` (`a` is either structure or a union) and `c` is a member of `b`, and `b` is an array where we take 20-th element.
- Added concept of the typeclasses (a bit similar to Haskell) for numerical types: `None` (the most generic one), `Num` (includes `Integral` and `Floating`), `Integral` (includes `Signed Integral` and `Unsigned Integral`), `Floating`.
- Added concept of type equality (`rz_types_equal(a, b)` API function)
- Added Doxygen documentation to every API function in the `librz/type/*`.
- Updated the project format from **V2** to **V3**, added migration.
* Use resource packing and fix the broken integration test for CPU profiles
* Use `sizeof` to calculate size and run `clang-format`
* Use `rz_file_temp` to create a temporary SDB file rather than manually naming it
Unit tests need the path of the rizin executable to properly run in some
cases, thus define rizin_exe variable both in regular build and when
rizin is built with -Dblob=true.
* Implement `aLj` in cmd_analysis.c ##print ##json
* Replace `,` with a `:` in `Loj` json ##test
* Replace `L` and `Lj` with `Ll` and `Llj` ##test
* Add tests for rz plugin json cmds ##print ##json
* List BinLdrPlugins for `Lij` ##print ##json
* Add spaces to represent no authors for `Li` ##test
* Convert `L` and `Ll` to newshell
* Convert `La` and `Laj` to newshell
* Move `Ll` handler to `clang.c`, rename `La` handler ##print ##json
* Convert `Lc` to newshell ##print ##json
* Convert `Ld` cmd to newshell ##print ##json
* Add `Laq` cmd test ##test
* Convert `Lh` to newshell and fix `La` check ##print ##json
* Fix `Lh` and convert `Li` cmd to newshell ##print ##json
* Convert `Lo` to newshell and modify `Lh` ##print ##json ##test
* Convert `Lp` to newshell ##print ##json ##test
* Use newshell cmd for `aL` ##print ##json
* Fix `Li`, lowercase json properties, add tests ##print ##json ##test
* Fix `Ld` and add `Lds` to set dbg backend ##print ##json ##test
* Revert `oL` and `aL` and replace `Lds` with `Ld` ##print ##json
* Use newshell for `obL`
* Replace `lang->cb_printf` with `rz_cons_printf`
* Replace `bin->cb_printf` with `rz_cons_printf` ##print
* Use `rz_lang_list()` instead of cmd0 ##print
* Set `Lo` and `Loj` tests as broken ##test
old RzBinReloc.vaddr contained either the reloc address or
the target address, depending on how the bin was loaded.
Now there are two members and dedicated query function for each.
* Added authors and licenses and plugin access to crypto related files
* removed `rz_hash` and related files
* introduced `rz_msg_digest_*` with plugins
* updated all old methods with newer ones.
* added HMAC support to `rz-hash` (moved key option under `-K`)
* removed wrong algorithms on `woD`/`woE`
* fixed parity digest size
* fixed the behaviour on all hash functions
* added unit test for hash
* optimized code for HMAC key calculation
* removed hash legacy tests
* just use one kill_word/backward_kill_word function to reduce dupped
code
* fix kill_word to correctly move buffer data
* add unit test for kill_word behaviour
* RzType introduction
* Move format processing to RzTypes from RzUtil
* Rename RzType to RzTypeDB
* Join RzType and RzAnalysis noreturn functions
* Add noreturn column in `aflt` output
* Remove D (disassembly) pf tests
* Fix `tn` test
* Add function noreturn serialization
* Modify meaningless type test
* Fix wrongly written `Cf` test
* Fix `za t` (zignature type) command
* Fix signature type test
* Set default meson buildtype to debugoptimized
* Add separate Codecov build and remove linux-meson-gcc-build build
* Set test buildtypes to highest optimization level possible
* Run Codecov build only on dev branch
glibc-compiled rizin first finds the stderr output, then the stdout
output, while on musl-compiled rizin the stdout output is first (it's
also printed first by the program, so it makes more sense) then comes
the stderr output.
This is probably due to different buffering policies within glibc and
musl.
Removed unused code and comments.
Removed dead code
Moved print method outside class.c
Moved json method outside class.c
Removed yc_w_refs and i_mref which never worked.
Fixed set_flags method to work as intended.
Fixed various parsing errors.
Fixed many memory leaks.
Fixed "insecure" rz_cons_printf warning.
removed `#define V`, `#define U` and `#define _`
Implemented newshell calls.
Optimized code.
Refactoring rz_bin_java_get_field_offsets
Avoid loading static core plugins when not needed.
Removed useless `if 0`
* Move rz_debug_reg_list from RzDebug to RzCore and directly use rz_cons_printf
* rz_reg_get_list returns always the same list, so make it const RzList *
MODE_DIST (-s) is the same as MODE_DIST_LENVENSHTEIN (-ss) but much slower.
Drop it. Make MODE_DIST_MYERS (-sss, faster than MODE_DIST_LENVENSHTEIN) take its place (-s).
The original comment was incorrect (-s is not Eugene W. Myer's algorithm). It is correct now.
Co-authored-by: eagleoflqj and Maskray
* Rename `sj`/`s*` and move them under `sH`. Remove `s=`
* Rename `s-*` to `sH-`
* Use `sd` for "seek delta", which moves relative to the current offset
* Enforce <cmd> <arg> syntax. So no more `s+16` but `s +16`. Command
`sHr` is for redo history, `sHu` for undo history. If one wants to move
relative to the current address, they need to do `sd +10`/`sd -10`.
* Remove `ss` commands in favour of `cmd.seek.silent`. This was anyway
used mainly for scripts, so no need to replicate all `s` commands
under `ss`.
Change a bit rz_core_seek API to avoid dupped entries in seek history.
* Remove undo write API, it was not used
* Move seek from IO to Core
* Remove IO undo files
* Fix cursor positionig in visual when doing undo/redo
* Add doxygen doc for rz_core_seek API
* Add cfg.seek.{histsize,silent} evals
* Add unit test for rz_[p]vector_foreach_prev
* Print proper error message when command does not exist
* Improve error messages on wrong arguments or wrong help
* Fix cmd_w test
* Check char * and avoid calling old input commands
* Fix english
* Check for NULLs and fix memleak
* Disable colors on Windows because terminal looks bad otherwise
* Use environment on Windows to set PATH and make sure unit tests use
the compiled libraries and not the installed ones
* Define ssize_t on Windows as well by using SSIZE_T
Update test_analysis_block.c
Update test_intervaltree.c
Update test_unum.c
By default, pipes file descriptors are not closed on exec. This means
they are inherited by the child process. However, this may be a problem
in particular in multi-threading programs (e.g. rz-test), because EOF is
not sent to a pipe until the last pipe file descriptor is closed.
This commit makes sure that by default pipes are created with O_CLOEXEC,
so they are closed whenever an exec syscall is executed. Usually pipe
file descriptors are duplicated to be stdin/stdout/stderr of the
children process and those dupped file descriptors won't be closed on
exec.
In case pipe2 is not available (e.g. MacOS), we do a pipe + fcntl, which
is in theory racy (see man open(2)). For this reason pipe, exec, system,
etc. are wrapped in rz_sys and they use a global mutex to make sure a
pipe is either created and set as O_CLOEXEC or not created at all before
an exec operation is done.
On very old systems, O_CLOEXEC may not be available at all. In those
cases, which should be rare, we just keep a list of file descriptors
created with rz_sys_pipe that we need to close before executing another
process.
Interaction with external libs may be tricky. We expect external libs to
create files/pipes that need to be closed on exec with the right flags
and we expect rizin code to manually close pipes file descriptors as
appropriate if the exec syscall is executed by an external library
(non-rizin code).
Make sure execl is found by acr/make
