Commit graph

472 commits

Author SHA1 Message Date
Florian Märkl
9a379f345d
Make dyld_chained_ptr_* parsing endian-independent (#3113)
Fixes test/db/cmd/cmd_tc and test/db/formats/mach0/arm64e on big endian
2022-10-22 01:03:24 +02:00
wargio
fbad0b4859 Remove agd and add rz_analysis_similarity/match + refactoring
- Introduce the following new methods:
  * `rz_analysis_similarity_basic_block()`
  * `rz_analysis_similarity_function()`
  * `rz_analysis_similarity_basic_block_2()`
  * `rz_analysis_similarity_function_2()`
  * `rz_analysis_match_basic_blocks()`
  * `rz_analysis_match_functions()`
  * `rz_analysis_match_basic_blocks_2()`
  * `rz_analysis_match_functions_2()`
- Remove `RzAnalysisDiff` and its usages
- Remove `librz/analysis/diff.c` entirely to and adds `librz/analysis/similarity.c`
- Remove a lot of unused structures and simplified the code by a lot
- Fix typo in levenshtein
- Remove `librz/core/gdiff.c` and `gdiff` functions since they were unused or only used in rz-diff
- Remove `difftype` (or "color") from `RzANode` which lead to dead code
- Decrease serialized data in projects.
- Add `RzAnalysisVarKind` and `RzAnalysisVarType` and replaces all the hardcoded values
- Remove command `agd`
- Remove all `rz_core_gdiff` usages from `rz_core.h`
- Refactor `rz_analysis_var_list` and `rz_analysis_var_count` due unused argument (`RzAnalysis` was not used)
- Add new method `rz_analysis_var_count_total` to simplify some usages
- Remove `rz_core_graph_diff`
- Add `typedef RzAnalysisFcnType` on `enum``used for function types
- Remove unused `RZ_ANALYSIS_FCN_VARKIND_LOCAL`
- Remove from `RzAnalysisFunction` the following variables:
  * `fingerprint`
  * `fingerprint_size`
  * `diff`
  * `diff_ops`
  * `diff_thbb`
  * `diff_thfcn`
- Remove from `RzAnalysisBlock` the following variables:
  * `fingerprint`
  * `diff`
- Remove `RZ_ANALYSIS_THRESHOLDBB` and `RZ_ANALYSIS_THRESHOLDFCN`
- Remove the following callbacks from `RzAnalysisPlugin`
  * `RzAnalysisDiffBBCallback`
  * `RzAnalysisDiffFcnCallback`
  * `RzAnalysisDiffEvalCallback`
- Fix fancy table columns/rows when a color string is in a cell
- Bumped project version to 9
2022-10-04 20:02:35 +02:00
wargio
8f40517990 Refactor rz_float_new_from_utXX_as_fXX 2022-09-16 21:55:58 +08:00
Heersin
b13202b934
RzUtil: softfloat support based on bitvector (#2784) 2022-09-14 14:38:22 +02:00
Riccardo Schirone
9ba89d8116 core/cmd: rewrite rz_cmd_macro API 2022-08-26 07:53:43 +08:00
Riccardo Schirone
02a0b7a1ad core/cmd: just store RzCore reference instead of void *data 2022-08-26 07:53:43 +08:00
Riccardo Schirone
ac1b0e4d94
core/cmd: fix CHOICES argument selection between choices_cb/choices (#2932) 2022-08-18 20:10:13 +08:00
karliss
32ef171354
Fix reset followed by color change in rz_cons_html_filter (#2912) 2022-08-13 09:43:18 +02:00
Heersin
a9c59ce1dd
Add rz_bv_range_set and rz_bv_is_all_one to bitvector lib (#2891) 2022-08-10 08:20:16 +08:00
Florian Märkl
5f97d93327 Rewrite and fix non-libuv tcp server (Rt command)
The non-libuv tcp server implementation had several issues:
* Relied on EINTR or self-connect through global vars for breaking,
  which did not work during recv() for example.
* Never retried on EINTR
* Called recv() only once, thus subject to fragmentation issues

To support breaking otherwise blocking socket calls, RzStopPipe has been
imported from chiaki (I am the sole author so I can relicense).

rz_socket_block_time() was also detected to be doing the exact opposite
than it should on Windows through the tests, and fixed.
2022-08-06 13:20:52 +02:00
Rot127
b9fa2be193
Asm tokens fixes (#2855)
* Replace on s390x failing tms320 c55x test with c55x+ test.

* Fix resource leaks of parse paramters.

* Revive unreachable code.

* Raplace test with one which has not a different asm string in Capstone v3.

* Rename `new` to `newt` since `new` is a keyword and should not be used as var name.
2022-07-30 21:24:49 +08:00
Rot127
c009f8d7a3
Tokenize and colorize asm strings (#2417)
This introduces the ability to split asm strings into tokens and assign a type to them.

This means:
* Each asm strings is assigned a list of tokens. Each token points to a sub-string.
* A token has a type attached which describes it (e.g. register token, operator token, mnemonic token etc.)
* A generic parsing method is introduced which splits an asm string into tokens. The pattern it parses is `<mnem> <op> <op> ...`.
* Instead of the generic method, plugins can create the token strings on their own.
* An API method was introduced to help plugins implement their own parsing methods via regex patterns.
* Custom token strings are stored in `RzAsmOp.asm_toks`

Additionally the coloring of asm strings is now exclusively done via the token strings.
* For this a function was added to colorize tokenized asm strings according to their token types.
2022-07-27 08:52:01 -04:00
GustavoLCR
0f69192562
Fixes for the regex engine (#2839)
* Fix some UTF8 not being considered characters
* Fix broken selection between `smatcher()` and `lmatcher()`
* Fix broken regex maching for `lmatcher()` (big state machines) engine
* Use `long long` for short state machine type
* Add tests
2022-07-25 10:29:32 +08:00
Riccardo Schirone
503b3baeeb reg: remove x86-linux.regs unused file 2022-07-20 22:04:02 +02:00
Riccardo Schirone
faac2436b1 Move SDB into RzUtil
* Make SDB code SPDX compliant
* Use some RzUtil functions inside SDB

util: compile natively without any extra dependency
2022-07-19 08:45:20 +02:00
Khairul Azhar Kasmiran
a58538d387
Fix rz_str_split_list_regex() (#2815) 2022-07-18 22:26:55 +08:00
Rot127
5a38df5294
Fix: register profile comments parsing (#2806)
Parse comments in register profile definitions correctly.
2022-07-18 19:36:56 +08:00
Rot127
ccdd9f2848
Add UTF-8 friendly strchr (#2809)
* Add utility functions to search for UTF-8 chars in strings.

* Break loop if number of read bytes goes beyond str.

* Add doxygen to `rz_mem_cmp`

* Add NULL checks to functions.
2022-07-18 17:53:58 +08:00
Riccardo Schirone
8939fdbd7d
Use add_global_arguments instead of passing cflags/ldflags all around (#2812)
See also 66a0e22293/docs/TESTING_WITH_SANITIZERS.md (clang)
2022-07-17 21:33:51 +08:00
Riccardo Schirone
6c665c6098 Use Rizin specific macros instead of the ones from SDB 2022-07-17 12:11:13 +02:00
Riccardo Schirone
470e030f19
util: provide rz_time_gettimeofday API (#2811) 2022-07-17 09:57:00 +08:00
Anton Kochkov
b2d9429129
Remove sdb json (#1118)
* Drop sdb_json in `librz/cons/grep.c`
* Do not print the JSON key in grep
* Use proper JSON paths in tests
* Use signed number in case of RzJson in tests
* Do not add quotes for sole RZ_JSON_STRING
2022-07-12 22:48:33 +08:00
Florian Märkl
c3e9c19109
Fix and test rz_json_as_string() for primitives (#2796)
Allowing converting the primitive json values null, number, string and
bool to be converted to strings is useful, simplifies the code and fixes
crashes when converting otherwise valid RzJson objects.
2022-07-09 15:40:50 +02:00
billow
40ca22202b
Refactor rz_core_print_disasm_json (#2746)
* Split `rz_core_print_disasm_json` as: `rz_core_print_disasm_json` and `rz_core_disasm`
* Use `rz_str_enc_string_as_type` in `ds_init`
* Refactor `rz_core_print_disasm` and `rz_core_handle_backwards_disasm`
* Add `rz_core_asm_bb_middle`
* Move global `disasm_texts` to `RzDisasmState`
* Rewrite `core_analysis_bytes_standard` with `rz_core_analysis_bytes`
* Add `test_rz_core_print_disasm` and `test_rz_core_analysis_bytes`
* Rewrite `rz_core_print_disasm` with `RzCmdStateOutput`
* `pdi` -> `pdq` in tests
* `pdi` -> `pDq` in tests
* `pid` -> `pdq` in tests
* `pId` -> `pDq` in tests
* `pij` -> `pdj` in tests
* `pIj` -> `pDj` in tests
* `pdeqq` -> `pdeQ` in tests
* Remove `pDJ` from `test/db/json/`
2022-07-09 08:10:06 +08:00
Riccardo Schirone
2f1e39818e Add liblzma subproject 2022-07-02 10:12:26 +02:00
Giovanni
ea02b0d25f
Remove zignatures and move aaF[l] -> F[al] , zf[sdc] -> F[sdc] (#2682) 2022-06-29 07:49:43 +08:00
Riccardo Schirone
2987e035da hash: use RzHash in most hash APIs 2022-06-28 21:55:26 +08:00
Riccardo Schirone
9ea7c2fa5a RzHash: rename everything in librz/hash to RzHash prefix 2022-06-28 21:55:26 +08:00
Giovanni
a2d23a8e11
Remove RQueue (#2720) 2022-06-22 22:41:05 +08:00
Rot127
f108873520
Rework register profile parsing
**Register profile parsing**

- Types, sizes, and offsets get now parsed separately in their own functions.
- More logging added.
- The string is first parsed into a list of register definitions and alias (`RzRegProfileDef`, `RzRegProfileAlias`) and afterward applied to `RzReg.regset`.
- Documentation was added on how the register profile needs to be formatted.
- Functions are more consistent with return types (most return `bool` now instead of `Error Msg` + `NULL`).

**New register profile structs**

The register profile of `RzReg` consists of two lists with alias and definitions.
I chose lists over NULL-terminated arrays because they were a little easier to handle during parsing (no memory management).
Please let me know if you would strongly prefer the NULL-terminated version.
2022-06-20 20:54:42 +08:00
Giovanni
a988941bf2
Refactor thread code and add RzThreadQueue (#2683)
* Kill the thread only if is alive.
* Remove rz_th_lock_guard.
* Use rz_sys_usleep since pthread_yield/sched_yield are not portable.
* Kill threads only on error.
* Avoid killing already-dead threads.
* RzThreadFunction now returns void* and added test_thread_queue.
* Removed pthread_exit
2022-06-19 21:07:13 +08:00
Dhruv Maroo
3c16afeeb0
Implement empty instruction for RzIL (#2697) 2022-06-14 23:07:57 +04:00
Luca Borzacchiello
2d69ecbb22
Better false-positives detection in rz_scan_strings (#2691)
* Improved false-positive detection in str_search

This commit adds the following features:
- Extend the false-positive check on ASCII frequencies to all UTF strings
- Add a global option to activate/deactivate di check
- Improve the false-positive heuristic by adding a special case for extended-ASCII strings
2022-06-14 22:14:13 +08:00
Riccardo Schirone
72c29e83d5 RzUtil: fix rz_path_prefix in case of portable build
Before this patch the function was using RZ_PREFIX even when IS_PORTABLE
was set, making the function not working well because RZ_PREFIX might
not exist in the system where the portable build is used (the path was
passed to realpath(), which requires an existing path).

This patch fixes the problem by searching for the BINDIR without using
realpath() on the RZ_PREFIX. Moreover, it avoid retrieving the same
prefix every time by saving it in a global variable.

* Add rz_constructor.h to allow defining constructors/destructors on
  multiple compilers.
2022-05-31 23:32:52 +08:00
Florian Märkl
0950e823a4
Refactor rz_core_analysis_get_stats() and fix edge cases (#2645)
Range is now explicitly inclusive/inclusive to allow generating stats
for the entire 64bit range.
This especially also removes the obscure "Cannot alloc for this range",
which could be observed in Cutter, and adds a more meaningful error to
the p- command whenever no range is available.
2022-05-30 04:59:47 +00:00
Paul I
9db3ed0670 Rename RzAnalysisXRefType enum members 2022-05-12 09:24:12 +02:00
Riccardo Schirone
6f8a59072c test: allow to run unit tests without installation 2022-05-10 16:57:44 +02:00
GustavoLCR
71e12433ac
Fix rz_buf_fwd_scan unit test for big endian archs (#2589) 2022-05-05 12:39:25 +08:00
Rot127
b65ef05e96
Fix: Allow to setup big endian bitvectors with less than 8bit values. (#2571) 2022-04-29 16:03:42 +02:00
Heersin
2fdb8ac07e
Fix copied bit count in rz_copy_nbits and add unit test (#2547) 2022-04-18 18:47:08 +02:00
GustavoLCR
55fc4dfac1
Implement new rz_buf_fwd_scan() API (#2530) 2022-04-17 15:08:08 +02:00
Peiwei Hu
5fa722efab
Refactor RzTable: change the implementation to RzVector (#2536) 2022-04-13 00:57:52 +08:00
Anton Kochkov
d6d214572f
Revert "Refactor RzTable: change the internal storage to RzVector (#2421)" (#2517)
This reverts commit 1748b38ce1.
2022-04-08 18:06:20 +08:00
Peiwei Hu
1748b38ce1
Refactor RzTable: change the internal storage to RzVector (#2421) 2022-04-07 21:13:34 +08:00
Riccardo Schirone
47b73ca7fa RzUtil/str: remove len argument from rz_str_filter
All uses of rz_str_filter already passed either strlen(..) or -1 or 0,
which was the same as computing strlen(...). The only case where a
different value was passed was in astr.c, where the string was anyway
allocated with rz_str_ndup(). Thus the string passed to rz_str_filter()
is always zero-terminated and we don't need to compute the length in
rz_str_filter() to traverse it all, but we can just stop at the first
NULL byte.
2022-04-06 01:46:18 +08:00
Rot127
10bedc2b00
Add Hexagon register profile (#2490)
Although the reg profile was not really used yet it will be relevant for the RZIL uplifting.
Introduces:
* Double/quadruple registers now properly overlap with their counterparts.
* Registers have correct types assigned (vec, ctr, sys etc. Before, all regs were of type gbr)
* Consistent register names (asm name style: c3:2, r31 etc, alias style: lc1:sa1, lr etc.)
* Adds _tmp registers in the profile. Later used by uplifted instructions (think of .new registers).
2022-04-03 18:26:17 +02:00
GustavoLCR
e945fb0e31 Introduce rz_file_dos_basename() API
Use it whenever we need to interact with paths that can contain `\` as a separator as well
2022-03-29 09:27:51 +08:00
GustavoLCR
77c46e1c9d Fix rz_file_basename on Windows 2022-03-29 09:27:51 +08:00
Florian Märkl
1cbd1d7b31
Introduce addr_bits to type db for e.g. thumb (#2457)
This has the same meaning as rz_analysis_get_address_bits(), i.e. the
size of an address, which is often, but not always the same as the
"asm.bits" bitness, especiall in thumb where it has to be 32, rather
than 16. This fixes pointer size calculation for such cases.

Fix #1835
2022-03-27 15:53:42 +02:00
Riccardo Schirone
7f19c8f867 Split unit and integration tests 2022-03-23 10:34:53 +01:00