* Add utility functions to search for UTF-8 chars in strings.
* Break loop if number of read bytes goes beyond str.
* Add doxygen to `rz_mem_cmp`
* Add NULL checks to functions.
* Drop sdb_json in `librz/cons/grep.c`
* Do not print the JSON key in grep
* Use proper JSON paths in tests
* Use signed number in case of RzJson in tests
* Do not add quotes for sole RZ_JSON_STRING
Allowing converting the primitive json values null, number, string and
bool to be converted to strings is useful, simplifies the code and fixes
crashes when converting otherwise valid RzJson objects.
* Add RzBinStrDb and api to interact with it via rz_bin_object_string_*
* use update instead of insert
* check for null deref in rz_analysis_set_stringrefs
* Never delete strings just add them and fixed comments
* CI: install sigdb in static version of rizin
* RzCore: use bool2status whenever possible
* RzCore: add flirt.sigdb.load vars to select which signatures to load
* test: adjust testing to not consider system/home flirt db
* sign: use a RzSigDb structure instead of a raw list to store signatures
* sign: switch RzSigDb to use a hashtable and avoid duplicates
**Register profile parsing**
- Types, sizes, and offsets get now parsed separately in their own functions.
- More logging added.
- The string is first parsed into a list of register definitions and alias (`RzRegProfileDef`, `RzRegProfileAlias`) and afterward applied to `RzReg.regset`.
- Documentation was added on how the register profile needs to be formatted.
- Functions are more consistent with return types (most return `bool` now instead of `Error Msg` + `NULL`).
**New register profile structs**
The register profile of `RzReg` consists of two lists with alias and definitions.
I chose lists over NULL-terminated arrays because they were a little easier to handle during parsing (no memory management).
Please let me know if you would strongly prefer the NULL-terminated version.
* Kill the thread only if is alive.
* Remove rz_th_lock_guard.
* Use rz_sys_usleep since pthread_yield/sched_yield are not portable.
* Kill threads only on error.
* Avoid killing already-dead threads.
* RzThreadFunction now returns void* and added test_thread_queue.
* Removed pthread_exit
* Improved false-positive detection in str_search
This commit adds the following features:
- Extend the false-positive check on ASCII frequencies to all UTF strings
- Add a global option to activate/deactivate di check
- Improve the false-positive heuristic by adding a special case for extended-ASCII strings
Before this patch the function was using RZ_PREFIX even when IS_PORTABLE
was set, making the function not working well because RZ_PREFIX might
not exist in the system where the portable build is used (the path was
passed to realpath(), which requires an existing path).
This patch fixes the problem by searching for the BINDIR without using
realpath() on the RZ_PREFIX. Moreover, it avoid retrieving the same
prefix every time by saving it in a global variable.
* Add rz_constructor.h to allow defining constructors/destructors on
multiple compilers.
Range is now explicitly inclusive/inclusive to allow generating stats
for the entire 64bit range.
This especially also removes the obscure "Cannot alloc for this range",
which could be observed in Cutter, and adds a more meaningful error to
the p- command whenever no range is available.
* RzBin/ELF: avoid the creation of useless flags at 0
Use UT64_MAX as an error value for addresses like symbols, imports,
sections, etc., and do not create flags for those because they are not
meaningful. This is useful for imports that do not have an address in
the virtual/file space and for non-alloc sections (e.g. .comment).
Entire files are now avoided to be built by checking in meson rather
than using #if DEBUGGER. This now also includes bsd_debug.c, which
is currently slightly broken on OpenBSD/sparc64 and caused compile
errors even with -Ddebugger=false.
* Removed eprintf from librz/bin/format/pyc/marshal.c
* Removed eprintf from librz/bin/format/pyc/pyc_magic.c
* Removed eprintf from librz/bin/format/te/te.c
* Removed eprintf from librz/bin/format/wasm/wasm.c
* Removed eprintf from librz/bin/format/xnu/rz_cf_dict.c
* Removed eprintf from librz/bin/p/bin_bf.c
* Removed eprintf from librz/bin/p/bin_cgc.c
* Removed eprintf from librz/bin/p/bin_elf.inc
* Removed eprintf from librz/bin/p/bin_luac.c
* Removed eprintf from librz/bin/p/bin_mach0.c
* Removed eprintf from librz/bin/p/bin_mdmp.c
* Removed eprintf from librz/bin/p/bin_menuet.c
* Removed eprintf from librz/bin/p/bin_nes.c
* Removed eprintf from librz/bin/p/bin_pe.c
* Removed eprintf from librz/bin/p/bin_pebble.c
* Removed eprintf from librz/bin/p/bin_psxexe.c
* Removed eprintf from librz/bin/p/bin_sfc.c
* Removed eprintfs from librz/bin/bfile.c
* Removed rz_bin_package and rz-bin -X because untested and not implemented
* Removed eprintfs from librz/bin/bin.c
* Be verbose on clang-format and not on command execution
* Removed eprintfs from librz/bin/bobj.c
* Removed eprintfs from librz/bin/dwarf.c
* Removed eprintfs from librz/bin/format/coff/coff.c
* Removed eprintfs from librz/bin/format/dmp/dmp64.c
* Removed eprintfs from librz/bin/format/luac/luac_bin.c
* Removed eprintf from librz/bin/format/luac/luac_common.c and refactoring
* Removed eprintf from librz/bin/format/luac/v53/parse_53.c
* Removed eprintf from librz/bin/format/mach0/coresymbolication.c
* Removed eprintf from librz/bin/format/mach0/dyldcache.c
* Removed eprintf from librz/bin/format/luac/v54/parse_54.c
* Removed eprintf from librz/bin/format/mach0/dyldcache_rebase.c
* Removed eprintf from librz/bin/format/mach0/fatmach0.c
* Removed eprintf from librz/bin/format/mdmp/mdmp.c
* Removed eprintf from librz/bin/format/mz/mz.c
* Removed eprintf from nso/nxo bin format
* Removed eprintf from librz/bin/format/objc/mach0_classes.c
* Removed eprintf from librz/bin/format/omf/omf.c
* Removed eprintf & bprintf from librz/bin/format/pe/pe.c
* Removed eprintf from librz/bin/format/pe/pemixed.c
* Fixed luac test
* pd + pda[jJqi=]
* Added pdb/pdC/pdf/pdk/pdl/pdp/pdr and removed pdi in favour of pdq
* removed pdx since is an alias for 2 other commands.
* Added pdR and pds[fb]
* Added example and fixed bug when cw
* Removed pdx test file since it is the same as cmd_pix
This this an Out of Bound read caused by missing sanitization of the
parsed dyldcache parsed structure.
Introduced via c19eaa8843
Original r2 commit: 09e20cd53d00a1497bf50349fe6eb812b4f54ac5
All uses of rz_str_filter already passed either strlen(..) or -1 or 0,
which was the same as computing strlen(...). The only case where a
different value was passed was in astr.c, where the string was anyway
allocated with rz_str_ndup(). Thus the string passed to rz_str_filter()
is always zero-terminated and we don't need to compute the length in
rz_str_filter() to traverse it all, but we can just stop at the first
NULL byte.
Although the reg profile was not really used yet it will be relevant for the RZIL uplifting.
Introduces:
* Double/quadruple registers now properly overlap with their counterparts.
* Registers have correct types assigned (vec, ctr, sys etc. Before, all regs were of type gbr)
* Consistent register names (asm name style: c3:2, r31 etc, alias style: lc1:sa1, lr etc.)
* Adds _tmp registers in the profile. Later used by uplifted instructions (think of .new registers).
* Add function to add config node to config by `RzConfigNode*`
* Move `SET<NODE>` macros to `rz_config.h`
* Add un/set functions for plugin specific configs.
* Add RzConfig and node setter to hexagon plugin.
* Use plugins RzConfig to determine sdk syntax setting.
* Use plugins RzConfig to determine if numbers should be signed.
* Use plugins RzConfig to determine if immediate hash should be printed.
* Remove unused parameter `rz_asm`.
* Remove unused variable.
* Remove unused callback.
* Remove unused flags from `RzAsm`
* Fix asm tests: Add NULL check; Set correct sign in asm test.
* Moved `rz_analysis_var_list_show()` under rz_core as static function
* Removed `cb_printf()` from RzAnalysis and refactoring zignatures
* Removed newline from matches
* Use `stdout` for ESIL traps
* Converted commands to use RzCmdStateOutput
This has the same meaning as rz_analysis_get_address_bits(), i.e. the
size of an address, which is often, but not always the same as the
"asm.bits" bitness, especiall in thumb where it has to be 32, rather
than 16. This fixes pointer size calculation for such cases.
Fix#1835
* Remove `cb_printf()` from RzBin
* Removed `bin.rawstr` from `rz-bin`
* Dropping `izzz` since does the same as `izz`
* Fix PE one section executables
* Fix behaviour of BFLT, LE, and SMD bin plugins
* Added rz_lang_byte_array and removed rz_print_code + refactoring cmds
* Added test for big endian
* Removed vlang and sync oldshell with newshell cmd changes
* Fixed in a more portable way conversion on signed byte for Java/Kotlin
* Resolved comments and added examples to help users
Due to some commands/tests still using the legacy "..." commands, we
can't fully remove oldshell code yet, but with this patch we just hide
it from the end users. Rzshell is now the default and only shell and
rzshell autocompletion is the default and only autocompletion engine.
* Create `rz_windows.h`
* Move `RzThread` structure definitions to internal header
* Create and use `rz_th_get_user()` API to avoid exposing `RzThread` layout in external header
* Make `rz_th_self()` an IPI
* Use `_wmkdir()` in `rz_sys_mkdir()` on Windows
* Remove unused `winkd_break_read` function
* Remove `RzIORap` definition from `rz_io.h`
* Remove include of `Windows.h` from `rz_types.h`
* Use `rz_windows.h` when necessary
* Remove `RzCoreRtrHost` definition from `rz_core.h`
* Remove `rz_socket.h` include from `rz_core.h`
* `!!`: Fix extra `\r` on Windows
* Use more efficient algorithm
* "Fix" cmd_pipe test
* "Fix" shared_memory.py
* Move feat_redirect test to archos/not-windows-any
This allows tracing an io map back to the RzCoreFile and thus also its
bin file and other info. This is used in rz-ghidra to determine which
memory ranges can be considered readonly to enable constant propagation.
There may be multiple maps containing the current seek. This indicates
only the highest-priority one, which is also the one whose data is
visible at the seek address.
Leaks detected by running rz-test with asan-built rz-asm. The sbc
instructions didn't do the carry subtraction right and threw away half
of the computation, which is why it was detected through leaks.
* Moved rz_print_2bpp_* under rz_core
* Moved rz_print_string to rz_str_stringify_raw_buffer
* Added test for print urlencode and added pu0
* Removed RzCoreString and implemented utf32be/utf16be
So far all the help sections were statically generated from the YAML
files, however sometimes you may want to list things in the help
messages that depend on what is loaded in Rizin (e.g. plugins, etc.).
This patch adds support for a details_cb that provides a dynamically
created RzCmdDescDetail array.
The code was duplicated in both asm and anylsis, so now it's in
arm_it.c.
Previously, conditions were applied to N offsets after an IT instruction
using a fixed stride of 2 bytes. This obviously breaks for 4-byte thumb2
instructions, but whenever we find such an instruction, we can check if
there is a condition inside of it and readjust if necessary.
* Add option to print instruction packets syntax in objdump style.
* Add the config `plugins.hexagon.sdk` to toggle syntax.
* Remove unused trimming.
* Trim only newlines from the assembly string.
* Add new trim functions to allow trimming of certain chars.
Problems fixed:
* Division by repeated addition is horribly slow, now it's classic
shift+sub
* a / a = 1, not a
* rz_bv_set_all() was setting too many bits if len % 8 != 0
* Removed print on div by 0 because we define it to all 1s (like BAP)
mod just uses div and we use regular ut64 division when possible.
Previously, bits were stored in the big endian in reversed order, such
that bit 0 of byte 0 would contain the msb. This requires reversing them
for any calculation and becomes even more confusing when the size is not
a multiple of 8, so let's just store them in classic little endian with
bit 0 of byte 0 containing the lsb.
This also fixes rz_bv_as_hex_string() for bvs that have a size not
divisiable by 0.
* Fix missing SEH scopes in x64 PE
* Fix multiple exception scopes in same address not adding flags
* Also add flags for the exception filter if it exists
* Fix assertion in `rz_analysis_block_update_hash()`
* Add call reference for exception filter
* Update old block's hash when creating a new block in trycatch analysis
* Fix try-catch control flow analysis
* Fix test
This fixes a bug that showed up after 2bdec10f98
because the bits value was set after cpu, but pcalign was already 4 then
and not updated by bits for example in:
rz-asm -a arm -b 16 -o 0x1059e -A dff81000
* Rename functions properly and ensure to remove the old flags.
* Rename flag instead of deleting it
* When renamed, the flags are not set in the right flag space.
This reverts commit bf39802452b7545551f52c4c32f8c01764ab2a75.
asm.cmt.esil was handled in a somewhat peculiar way, printing below the
disassembly if asm.cmt.right=0, even though every other comment is
printed above. Now it is more in line with all other commenting and the
new asm.cmt.il works in the same way.
* Changed behavior of `f` to only add flag if there is none
* Changed behavior of `f+` to add flag in any case
* Moved listing subcommands from `f` to `fl`
* Moved listing subcommands from `f.` to `f.l`
* Added subcommand `f.l*` to list all local flags in all functions
* Listing flags at the current offset became `fl.`
* Moved `f=` to `fl=`
* Old `fl` (flag length) now became `fL`
* Removed original `fe` commands
* Old `f?` (check if flag exists) now became `fe`
* Removed `fV` commands
* Removed `fn` commands in favor of `fl` and `asm.flags.real=true`
* Removed `fS` commands (`fSo` and `fSn`)
* `fo` became `fortune` and moved from "flags" to "shell" category
* Removed oldshell handlers of the `z` commands
* Fix a crash during variables type propagation
* Fix test int64_t variable overlapping removal
Co-authored-by: Anton Kochkov <anton.kochkov@gmail.com>
* Removed nullability where not needed
* Made events take RzILVal instead of force-converting to RzILBitVector
for variables
* Added more fine-grained tests for events from specific ops
RzAnalysisILVM wraps around the low-level RzILVM and enables emulation
of real code from disassembly, rather than raw IL.
Analysis plugins now don't actively initialize the vm anymore, but
return a fully declarative description RzAnalysisILConfig of how to set
up the vm and optionally its initial state.
This also enables multiple IL vms to exist at the same time as plugins
can not mess with the global vm anymore. See the added integration test
for an example.
This adds RzIL lifting for all legal 6502 ops, except the special bcd
modes of adc and sbc for now. The lifting of every one of these ops is
covered by an asm test, making sure all IL is valid. Some manual
execution tests are available too, including one that runs a simple
real-world xor-"decryption" loop.
The lifting has not been tested against traces yet, so minor mistakes in
the semantics are currently expected.
Also new in this commit:
- Uses system registers as defined in LLVM definitions.
- Several new Hexagon v69 (HVX) instructions.
- File header now has the LLVM commit and timestamps of the source file generation.
Because there is currently no reason to have non-ctrl effects after a ctrl
effect (because then we have already jumped somewhere else), we reject this
kind of code altogether.
This leaves future semantics of such code open without breaking any
existing IL, in case it will be needed at some point.
Previously, after a (jmp x) with opsize y, the jump target would
actually be x + y because the addition happened after the evaluation.
Now we actually jump to the target specified by the op.
* removal of `omo` command
* use of `oml`, `oml.` and `oml=` to list maps instead of `om`
* removal of RIZIN output from `oml`
* removal of `omn.` and just assume `omn` uses core->offset
Before:
store(key:var(v:ptr), value:add(x:load(key:var(v:ptr), mem:0),
y:int(value:1, length:8)), mem:0)
After:
(store 0 (var ptr) (+ (load 0 (var ptr)) (bv 8 0x1)))
S-Expressions like this are more concise and will be trivial to parse,
in case we want a parser for this later. It is also very similar to what
BAP uses, so it will be easier to compare.
This adds the IL validator, which performs static type-checking of both
pure and effect ops among other checks. In particular, assuming the
validator is correctly implemented, if it considers an op to be valid
under some context, evaluating the op in the vm will never yield a
runtime error, that is, an error where the vm itself errors, not an
expected error state of the code being executed. In our case, this
includes for example:
* Any kind of type error: Conditions not being bool, bitvector sizes not
matching, ...
* Variables not being available when they are accessed Using local
* variables with multiple different types in a single effect etc.
Any code that we lift must obey these rules. Thus, any analysis can rely
on it. The plan for this is to use the validator primarily in testing,
development of lifters and for IL code coming from the outside. If our
lifting code is covered well enough by tests using the validator, we can
omit the validation at runtime.
The only ops that do not have well-defined validation yet are blk and
goto since their semantics, in particular regarding label handling are
still a bit vague. This also removes the concat and unk ops since they
are unimplemented and not needed.
There are now three kinds of vars: global, local and local pure. Global
and local pure are exactly like in BAP, local ones are defined by their
initial set op and have the scope of a single lifted instruction.
The set op now handles both global and local vars, let is now pure and
binds only inside its body. Vars have static types, global and local are
always mutable, local pure naturally immutable.
The var op specifies the kind of variable to take from, and the
individual variable sets are separate. This corresponds to BAP's
behavior where the kind of variable is part of the identifier.
Variable content storage has also been rewritten and RzILBag removed.
