- Introduce the following new methods:
* `rz_analysis_similarity_basic_block()`
* `rz_analysis_similarity_function()`
* `rz_analysis_similarity_basic_block_2()`
* `rz_analysis_similarity_function_2()`
* `rz_analysis_match_basic_blocks()`
* `rz_analysis_match_functions()`
* `rz_analysis_match_basic_blocks_2()`
* `rz_analysis_match_functions_2()`
- Remove `RzAnalysisDiff` and its usages
- Remove `librz/analysis/diff.c` entirely to and adds `librz/analysis/similarity.c`
- Remove a lot of unused structures and simplified the code by a lot
- Fix typo in levenshtein
- Remove `librz/core/gdiff.c` and `gdiff` functions since they were unused or only used in rz-diff
- Remove `difftype` (or "color") from `RzANode` which lead to dead code
- Decrease serialized data in projects.
- Add `RzAnalysisVarKind` and `RzAnalysisVarType` and replaces all the hardcoded values
- Remove command `agd`
- Remove all `rz_core_gdiff` usages from `rz_core.h`
- Refactor `rz_analysis_var_list` and `rz_analysis_var_count` due unused argument (`RzAnalysis` was not used)
- Add new method `rz_analysis_var_count_total` to simplify some usages
- Remove `rz_core_graph_diff`
- Add `typedef RzAnalysisFcnType` on `enum``used for function types
- Remove unused `RZ_ANALYSIS_FCN_VARKIND_LOCAL`
- Remove from `RzAnalysisFunction` the following variables:
* `fingerprint`
* `fingerprint_size`
* `diff`
* `diff_ops`
* `diff_thbb`
* `diff_thfcn`
- Remove from `RzAnalysisBlock` the following variables:
* `fingerprint`
* `diff`
- Remove `RZ_ANALYSIS_THRESHOLDBB` and `RZ_ANALYSIS_THRESHOLDFCN`
- Remove the following callbacks from `RzAnalysisPlugin`
* `RzAnalysisDiffBBCallback`
* `RzAnalysisDiffFcnCallback`
* `RzAnalysisDiffEvalCallback`
- Fix fancy table columns/rows when a color string is in a cell
- Bumped project version to 9
Previously, on macOS we were detaching properly when the user quit rizin
after an interactive `rz -d <file>` session, but not on
`rz -d -Qc <cmd> <file>`, which would leave behind a zombie process.
Now, the process is detached and keeps running, which also matches the
behavior on Linux. Hence the `dk 9` additions in the tests, to not
spoil the output with whatever the processes print.
Fixes the remaining test in test/db/cmd/cmd_c on big endian hosts,
tested on OpenBSD/sparc64.
The mask calculation was removed because it was unnecessary (only
comparing selected bytes anyway) and wrong by a factor of 8 (bits vs.
bytes confusion).
* Print a warning if config for VM was NULL.
* Add warning if register is not added to VM due to overlap.
* Update PPC register profile.
* Add vector and float registers. As well as some control and system registers.
* Enable to write values 4bit registers.
* PPC: Uplift most common instructions.
* Print warning if reserved SPR instruction is encountered.
* Update PC/LR addresses and add ca32 writes.
* Write cache needs to get flushed so load and store tests don't share the same memory.
* Add missing T/F branch mnemonics.
* Add Move to/from CR/CR0-7
* Fix TA address calculation for branch instructions.
* Add branch tests for branch mnemonics.
* Add XNOP
* NOP cache touch instructions.
* Add undocumented ATTN instruction to the not_implemented group.
* Add isel instruction.
* Implement CRCLR, CRSET, CROR.
* Add CNTLZ instructions.
* Add mcrf instructions.
* Add inacive test for cmpb.
* Add Load bytes reverse instructions.
* Add test and add address alignment to dcbz.
* Add eqv test
* Correct DIV and MUL operations
* Add div tests.
* Add tests and correct MT/MFXER
* Remove register ca32, ov32
* Remove explici setting of cr register because QEMU does not do it. Otherwise we get a mismatch in the trace
* Simplify carry set for add and sub. Sub instructions are exclusivly defined with addition. Hence no sub case needed.
* Unify BD and fix branch instructions. Fix: Check the single bit not the cr reg
* Document Conditional branches and replace NOPs with EMPTY
* Add mulli instruction to double word instructions.
* Fix ca set for shift instructions:
* ca value had to be determined before the shift happened. The wrong ca value was calculated if the src and target reg were the same.
* Replace NOP wit EMPTY.
* Use MSB isntead of SLT.
* Brought fixup of ADD and SUB instructions: The add and sub instructions had several issues which let to incorrect execution.
* The carry was incorrectly if three add operations happened (only the last add were checked, not both).
* The carry was incorrectly set if the src and target register matched.
* Same applies for the CR bit.
* It was too complex. Several local variables were introduced for this.
* Set result in local var, since it would change if src and target reg are the same.
* Remove MTMSR and MFMSR since it is too complex and untestable currently.
* Use unsigned int for shift. Otherwise the 0x1c shift produces a runtime error since 0xf is int as default.
* Fix mtxer: Only write flag bits.
* Let NOT_IMPLEMENTED macro return NULL.
* Mark st[wd]cx and l[wd]cx as not implemented.
* Increase dcache_line_size to 128 bytes.
* Fix cntlz for ppc32. m was set incorrectly, since it is 0 not 32 for 32bit cpus.
* Fix isel: Use op.crx reg instead of imm.
* Unify helper function names: Prependnig `ppc_` mark as IPI
* Add more "Move to SPR" cases.
* MULLI opeartes only on double word on 64bit CPUs.
* Most registers are now assigned the control register type and no longer show up in the ar command.
* Fix xor if dest and src registers match by saving result in local var.
* Fix BE/LE issue for Load BRX instructions.
* Fix shifts: Use only lower 6bits of n.
* User Pure local variables for ROT macros.
* Fix rldimi instructions.
* n was not inverted.
* more than 6 bits of n could be used
* Add 32bit emulateme tests.
* Add 64bit emulateme tests.
* Determine lg(v) in inline function.
* Calculate CR bit in C not in the VM.
* Check if `~mask = 0` and skip mask calculation if yes.
* Check for `sh == 0` and skip rotations where possible.
* Remove `la` instruction. `la` is a mnemonic for `addi`.
* Remove SPR instructions which are not supported by QEMU or not traced yet. For most set/read SPR instructions QEMU segfaults.
In case of SPR 1 (xer), 8 (lr) and 9 (ctr) the assembler resolves them to their mnemonics (mtxer, mtlr etc.). This means the code here is never reached.
To test the get_xer code MFXER was added again. The rz-tracetests will fail for this instructions (due to missing ca32, ov32). But this case is covert in an issue.
* Chane HexInsn -> HexInsnContainer in asm plugin.
* Make HexInsn -> HexInsnContainer in hexagon_disas.c
* Add function to set concatinate textual disassembly parts.
* Switch expressions, fix segfault.
* Print number of unhandle ops, if they can not be added to RzAnalysisOp.
* Add lookup function to determine sub-instructions for duplex iclass.
* Fix asm tests.
* Add memcpy for instruction containers
* Fix off by one.
* Fix segfault if the packet held only the instruction which is currently resolved. `hic->bin.insn` would be NULL in this case.
* Remove duplicate index name.
* Set instruction address.
* Clean up hic->parse_bits use
* Fix: .new registers were not resolved dur to too early return.
* Fix rebase mistakes.
* Add missing code
* Fix build error: Use sizeof(`text_infix`) since it is way smaller than `text`
* Initialize `HexInsn` inside `setup_new_hic`. Fixes NULL access.
* Mark RzAsm/RzAnalysis members as deprecated and fix typo.
ARM_THREAD_STATE64 fetches arm_thread_state64_t but our struct was
arm_unified_thread_state_t, which adds a header. The pc in the register
profile was hacked to account for the shift, but all the x-regs were
not, so x7 had the value of x8 for example. Now we just use the specific
state for 32 or 64.
This fixes basic single-stepping on macOS/arm64.
dbg->bits is affected by all kinds of things going on in rizin, in
particular it currently randomly switches to 16 when calling pd during
an arm64 debug session, causing the stepping code to break.
We now fetch the cpu type of the process once during attach, similar to
how lldb does it. They also use 0 for the "invalid" value, so we do that
as well.
The non-libuv tcp server implementation had several issues:
* Relied on EINTR or self-connect through global vars for breaking,
which did not work during recv() for example.
* Never retried on EINTR
* Called recv() only once, thus subject to fragmentation issues
To support breaking otherwise blocking socket calls, RzStopPipe has been
imported from chiaki (I am the sole author so I can relicense).
rz_socket_block_time() was also detected to be doing the exact opposite
than it should on Windows through the tests, and fixed.
* Replace on s390x failing tms320 c55x test with c55x+ test.
* Fix resource leaks of parse paramters.
* Revive unreachable code.
* Raplace test with one which has not a different asm string in Capstone v3.
* Rename `new` to `newt` since `new` is a keyword and should not be used as var name.
The warning about non-existent register types is useful for register
profile parsing errors, but this function is also used by e.g.
`ar`/`dr`, so the overloaded `ar` would print a warning when given a
specific register name for example.
This fixes the following minor issues:
* Errors like "Cannot create flag (...) because there is already (...)
flag" during debug startup
* Unnecessary seeks during .dm* causing io reads and e.g. delays of up
to 5s on x86 macOS during debug startup
* Flags are now created in the "maps" flagspace, rather than globally,
making them easier to clean up later
* Flags now have the same size as the maps and not +1
This introduces the ability to split asm strings into tokens and assign a type to them.
This means:
* Each asm strings is assigned a list of tokens. Each token points to a sub-string.
* A token has a type attached which describes it (e.g. register token, operator token, mnemonic token etc.)
* A generic parsing method is introduced which splits an asm string into tokens. The pattern it parses is `<mnem> <op> <op> ...`.
* Instead of the generic method, plugins can create the token strings on their own.
* An API method was introduced to help plugins implement their own parsing methods via regex patterns.
* Custom token strings are stored in `RzAsmOp.asm_toks`
Additionally the coloring of asm strings is now exclusively done via the token strings.
* For this a function was added to colorize tokenized asm strings according to their token types.
Apart from obvious big endian parsing bugs, this also tests `il` with
xbe, since that had endianness issues too and was previously untested.
To be able to have meaningful tests there, extra entries for the
explicit kernel and xapi pointers found in the xbe header have been
removed, since these always seem to point into the regular library array
anyway and would thus result in duplicates.
This fixes test/db/formats/xbe on OpenBSD/sparc64.
* Fix some UTF8 not being considered characters
* Fix broken selection between `smatcher()` and `lmatcher()`
* Fix broken regex maching for `lmatcher()` (big state machines) engine
* Use `long long` for short state machine type
* Add tests
* Close files before trying to delete them
* Add `-N` to not load the rizinrc in `read from stdin` test to avoid overwriting the color setting
* Make sure `.tmp` exists for `idpx` test
* Add utility functions to search for UTF-8 chars in strings.
* Break loop if number of read bytes goes beyond str.
* Add doxygen to `rz_mem_cmp`
* Add NULL checks to functions.
* Drop sdb_json in `librz/cons/grep.c`
* Do not print the JSON key in grep
* Use proper JSON paths in tests
* Use signed number in case of RzJson in tests
* Do not add quotes for sole RZ_JSON_STRING
Allowing converting the primitive json values null, number, string and
bool to be converted to strings is useful, simplifies the code and fixes
crashes when converting otherwise valid RzJson objects.
* Add RzBinStrDb and api to interact with it via rz_bin_object_string_*
* use update instead of insert
* check for null deref in rz_analysis_set_stringrefs
* Never delete strings just add them and fixed comments
* CI: install sigdb in static version of rizin
* RzCore: use bool2status whenever possible
* RzCore: add flirt.sigdb.load vars to select which signatures to load
* test: adjust testing to not consider system/home flirt db
* sign: use a RzSigDb structure instead of a raw list to store signatures
* sign: switch RzSigDb to use a hashtable and avoid duplicates
**Register profile parsing**
- Types, sizes, and offsets get now parsed separately in their own functions.
- More logging added.
- The string is first parsed into a list of register definitions and alias (`RzRegProfileDef`, `RzRegProfileAlias`) and afterward applied to `RzReg.regset`.
- Documentation was added on how the register profile needs to be formatted.
- Functions are more consistent with return types (most return `bool` now instead of `Error Msg` + `NULL`).
**New register profile structs**
The register profile of `RzReg` consists of two lists with alias and definitions.
I chose lists over NULL-terminated arrays because they were a little easier to handle during parsing (no memory management).
Please let me know if you would strongly prefer the NULL-terminated version.
* Kill the thread only if is alive.
* Remove rz_th_lock_guard.
* Use rz_sys_usleep since pthread_yield/sched_yield are not portable.
* Kill threads only on error.
* Avoid killing already-dead threads.
* RzThreadFunction now returns void* and added test_thread_queue.
* Removed pthread_exit
* Improved false-positive detection in str_search
This commit adds the following features:
- Extend the false-positive check on ASCII frequencies to all UTF strings
- Add a global option to activate/deactivate di check
- Improve the false-positive heuristic by adding a special case for extended-ASCII strings
Before this patch the function was using RZ_PREFIX even when IS_PORTABLE
was set, making the function not working well because RZ_PREFIX might
not exist in the system where the portable build is used (the path was
passed to realpath(), which requires an existing path).
This patch fixes the problem by searching for the BINDIR without using
realpath() on the RZ_PREFIX. Moreover, it avoid retrieving the same
prefix every time by saving it in a global variable.
* Add rz_constructor.h to allow defining constructors/destructors on
multiple compilers.
Range is now explicitly inclusive/inclusive to allow generating stats
for the entire 64bit range.
This especially also removes the obscure "Cannot alloc for this range",
which could be observed in Cutter, and adds a more meaningful error to
the p- command whenever no range is available.
* RzBin/ELF: avoid the creation of useless flags at 0
Use UT64_MAX as an error value for addresses like symbols, imports,
sections, etc., and do not create flags for those because they are not
meaningful. This is useful for imports that do not have an address in
the virtual/file space and for non-alloc sections (e.g. .comment).
Entire files are now avoided to be built by checking in meson rather
than using #if DEBUGGER. This now also includes bsd_debug.c, which
is currently slightly broken on OpenBSD/sparc64 and caused compile
errors even with -Ddebugger=false.
* Removed eprintf from librz/bin/format/pyc/marshal.c
* Removed eprintf from librz/bin/format/pyc/pyc_magic.c
* Removed eprintf from librz/bin/format/te/te.c
* Removed eprintf from librz/bin/format/wasm/wasm.c
* Removed eprintf from librz/bin/format/xnu/rz_cf_dict.c
* Removed eprintf from librz/bin/p/bin_bf.c
* Removed eprintf from librz/bin/p/bin_cgc.c
* Removed eprintf from librz/bin/p/bin_elf.inc
* Removed eprintf from librz/bin/p/bin_luac.c
* Removed eprintf from librz/bin/p/bin_mach0.c
* Removed eprintf from librz/bin/p/bin_mdmp.c
* Removed eprintf from librz/bin/p/bin_menuet.c
* Removed eprintf from librz/bin/p/bin_nes.c
* Removed eprintf from librz/bin/p/bin_pe.c
* Removed eprintf from librz/bin/p/bin_pebble.c
* Removed eprintf from librz/bin/p/bin_psxexe.c
* Removed eprintf from librz/bin/p/bin_sfc.c
* Removed eprintfs from librz/bin/bfile.c
* Removed rz_bin_package and rz-bin -X because untested and not implemented
* Removed eprintfs from librz/bin/bin.c
* Be verbose on clang-format and not on command execution
* Removed eprintfs from librz/bin/bobj.c
* Removed eprintfs from librz/bin/dwarf.c
* Removed eprintfs from librz/bin/format/coff/coff.c
* Removed eprintfs from librz/bin/format/dmp/dmp64.c
* Removed eprintfs from librz/bin/format/luac/luac_bin.c
* Removed eprintf from librz/bin/format/luac/luac_common.c and refactoring
* Removed eprintf from librz/bin/format/luac/v53/parse_53.c
* Removed eprintf from librz/bin/format/mach0/coresymbolication.c
* Removed eprintf from librz/bin/format/mach0/dyldcache.c
* Removed eprintf from librz/bin/format/luac/v54/parse_54.c
* Removed eprintf from librz/bin/format/mach0/dyldcache_rebase.c
* Removed eprintf from librz/bin/format/mach0/fatmach0.c
* Removed eprintf from librz/bin/format/mdmp/mdmp.c
* Removed eprintf from librz/bin/format/mz/mz.c
* Removed eprintf from nso/nxo bin format
* Removed eprintf from librz/bin/format/objc/mach0_classes.c
* Removed eprintf from librz/bin/format/omf/omf.c
* Removed eprintf & bprintf from librz/bin/format/pe/pe.c
* Removed eprintf from librz/bin/format/pe/pemixed.c
* Fixed luac test
* pd + pda[jJqi=]
* Added pdb/pdC/pdf/pdk/pdl/pdp/pdr and removed pdi in favour of pdq
* removed pdx since is an alias for 2 other commands.
* Added pdR and pds[fb]
* Added example and fixed bug when cw
* Removed pdx test file since it is the same as cmd_pix
This this an Out of Bound read caused by missing sanitization of the
parsed dyldcache parsed structure.
Introduced via c19eaa8843
Original r2 commit: 09e20cd53d00a1497bf50349fe6eb812b4f54ac5
All uses of rz_str_filter already passed either strlen(..) or -1 or 0,
which was the same as computing strlen(...). The only case where a
different value was passed was in astr.c, where the string was anyway
allocated with rz_str_ndup(). Thus the string passed to rz_str_filter()
is always zero-terminated and we don't need to compute the length in
rz_str_filter() to traverse it all, but we can just stop at the first
NULL byte.
Although the reg profile was not really used yet it will be relevant for the RZIL uplifting.
Introduces:
* Double/quadruple registers now properly overlap with their counterparts.
* Registers have correct types assigned (vec, ctr, sys etc. Before, all regs were of type gbr)
* Consistent register names (asm name style: c3:2, r31 etc, alias style: lc1:sa1, lr etc.)
* Adds _tmp registers in the profile. Later used by uplifted instructions (think of .new registers).
* Add function to add config node to config by `RzConfigNode*`
* Move `SET<NODE>` macros to `rz_config.h`
* Add un/set functions for plugin specific configs.
* Add RzConfig and node setter to hexagon plugin.
* Use plugins RzConfig to determine sdk syntax setting.
* Use plugins RzConfig to determine if numbers should be signed.
* Use plugins RzConfig to determine if immediate hash should be printed.
* Remove unused parameter `rz_asm`.
* Remove unused variable.
* Remove unused callback.
* Remove unused flags from `RzAsm`
* Fix asm tests: Add NULL check; Set correct sign in asm test.
* Moved `rz_analysis_var_list_show()` under rz_core as static function
* Removed `cb_printf()` from RzAnalysis and refactoring zignatures
* Removed newline from matches
* Use `stdout` for ESIL traps
* Converted commands to use RzCmdStateOutput
This has the same meaning as rz_analysis_get_address_bits(), i.e. the
size of an address, which is often, but not always the same as the
"asm.bits" bitness, especiall in thumb where it has to be 32, rather
than 16. This fixes pointer size calculation for such cases.
Fix#1835
* Remove `cb_printf()` from RzBin
* Removed `bin.rawstr` from `rz-bin`
* Dropping `izzz` since does the same as `izz`
* Fix PE one section executables
* Fix behaviour of BFLT, LE, and SMD bin plugins
* Added rz_lang_byte_array and removed rz_print_code + refactoring cmds
* Added test for big endian
* Removed vlang and sync oldshell with newshell cmd changes
* Fixed in a more portable way conversion on signed byte for Java/Kotlin
* Resolved comments and added examples to help users
Due to some commands/tests still using the legacy "..." commands, we
can't fully remove oldshell code yet, but with this patch we just hide
it from the end users. Rzshell is now the default and only shell and
rzshell autocompletion is the default and only autocompletion engine.
* Create `rz_windows.h`
* Move `RzThread` structure definitions to internal header
* Create and use `rz_th_get_user()` API to avoid exposing `RzThread` layout in external header
* Make `rz_th_self()` an IPI
* Use `_wmkdir()` in `rz_sys_mkdir()` on Windows
* Remove unused `winkd_break_read` function
* Remove `RzIORap` definition from `rz_io.h`
* Remove include of `Windows.h` from `rz_types.h`
* Use `rz_windows.h` when necessary
* Remove `RzCoreRtrHost` definition from `rz_core.h`
* Remove `rz_socket.h` include from `rz_core.h`
* `!!`: Fix extra `\r` on Windows
* Use more efficient algorithm
* "Fix" cmd_pipe test
* "Fix" shared_memory.py
* Move feat_redirect test to archos/not-windows-any
This allows tracing an io map back to the RzCoreFile and thus also its
bin file and other info. This is used in rz-ghidra to determine which
memory ranges can be considered readonly to enable constant propagation.
There may be multiple maps containing the current seek. This indicates
only the highest-priority one, which is also the one whose data is
visible at the seek address.
Leaks detected by running rz-test with asan-built rz-asm. The sbc
instructions didn't do the carry subtraction right and threw away half
of the computation, which is why it was detected through leaks.
* Moved rz_print_2bpp_* under rz_core
* Moved rz_print_string to rz_str_stringify_raw_buffer
* Added test for print urlencode and added pu0
* Removed RzCoreString and implemented utf32be/utf16be
So far all the help sections were statically generated from the YAML
files, however sometimes you may want to list things in the help
messages that depend on what is loaded in Rizin (e.g. plugins, etc.).
This patch adds support for a details_cb that provides a dynamically
created RzCmdDescDetail array.
The code was duplicated in both asm and anylsis, so now it's in
arm_it.c.
Previously, conditions were applied to N offsets after an IT instruction
using a fixed stride of 2 bytes. This obviously breaks for 4-byte thumb2
instructions, but whenever we find such an instruction, we can check if
there is a condition inside of it and readjust if necessary.
* Add option to print instruction packets syntax in objdump style.
* Add the config `plugins.hexagon.sdk` to toggle syntax.
* Remove unused trimming.
* Trim only newlines from the assembly string.
* Add new trim functions to allow trimming of certain chars.
Problems fixed:
* Division by repeated addition is horribly slow, now it's classic
shift+sub
* a / a = 1, not a
* rz_bv_set_all() was setting too many bits if len % 8 != 0
* Removed print on div by 0 because we define it to all 1s (like BAP)
mod just uses div and we use regular ut64 division when possible.
Previously, bits were stored in the big endian in reversed order, such
that bit 0 of byte 0 would contain the msb. This requires reversing them
for any calculation and becomes even more confusing when the size is not
a multiple of 8, so let's just store them in classic little endian with
bit 0 of byte 0 containing the lsb.
This also fixes rz_bv_as_hex_string() for bvs that have a size not
divisiable by 0.
* Fix missing SEH scopes in x64 PE
* Fix multiple exception scopes in same address not adding flags
* Also add flags for the exception filter if it exists
* Fix assertion in `rz_analysis_block_update_hash()`
* Add call reference for exception filter
* Update old block's hash when creating a new block in trycatch analysis
* Fix try-catch control flow analysis
* Fix test
This fixes a bug that showed up after 2bdec10f98
because the bits value was set after cpu, but pcalign was already 4 then
and not updated by bits for example in:
rz-asm -a arm -b 16 -o 0x1059e -A dff81000
* Rename functions properly and ensure to remove the old flags.
* Rename flag instead of deleting it
* When renamed, the flags are not set in the right flag space.
This reverts commit bf39802452b7545551f52c4c32f8c01764ab2a75.
