Commit graph

163 commits

Author SHA1 Message Date
wargio
fbad0b4859 Remove agd and add rz_analysis_similarity/match + refactoring
- Introduce the following new methods:
  * `rz_analysis_similarity_basic_block()`
  * `rz_analysis_similarity_function()`
  * `rz_analysis_similarity_basic_block_2()`
  * `rz_analysis_similarity_function_2()`
  * `rz_analysis_match_basic_blocks()`
  * `rz_analysis_match_functions()`
  * `rz_analysis_match_basic_blocks_2()`
  * `rz_analysis_match_functions_2()`
- Remove `RzAnalysisDiff` and its usages
- Remove `librz/analysis/diff.c` entirely to and adds `librz/analysis/similarity.c`
- Remove a lot of unused structures and simplified the code by a lot
- Fix typo in levenshtein
- Remove `librz/core/gdiff.c` and `gdiff` functions since they were unused or only used in rz-diff
- Remove `difftype` (or "color") from `RzANode` which lead to dead code
- Decrease serialized data in projects.
- Add `RzAnalysisVarKind` and `RzAnalysisVarType` and replaces all the hardcoded values
- Remove command `agd`
- Remove all `rz_core_gdiff` usages from `rz_core.h`
- Refactor `rz_analysis_var_list` and `rz_analysis_var_count` due unused argument (`RzAnalysis` was not used)
- Add new method `rz_analysis_var_count_total` to simplify some usages
- Remove `rz_core_graph_diff`
- Add `typedef RzAnalysisFcnType` on `enum``used for function types
- Remove unused `RZ_ANALYSIS_FCN_VARKIND_LOCAL`
- Remove from `RzAnalysisFunction` the following variables:
  * `fingerprint`
  * `fingerprint_size`
  * `diff`
  * `diff_ops`
  * `diff_thbb`
  * `diff_thfcn`
- Remove from `RzAnalysisBlock` the following variables:
  * `fingerprint`
  * `diff`
- Remove `RZ_ANALYSIS_THRESHOLDBB` and `RZ_ANALYSIS_THRESHOLDFCN`
- Remove the following callbacks from `RzAnalysisPlugin`
  * `RzAnalysisDiffBBCallback`
  * `RzAnalysisDiffFcnCallback`
  * `RzAnalysisDiffEvalCallback`
- Fix fancy table columns/rows when a color string is in a cell
- Bumped project version to 9
2022-10-04 20:02:35 +02:00
Anton Kochkov
52260857b5
Convert some of the a commands to the rzshell (#3025)
* Remove `ap` command
* Convert `ad` commands to rzshell
* Convert `aF` command to the rzshell
2022-09-15 21:09:15 +02:00
Giovanni
008363fcc4
Fix #2851 - wrong AVR analysis jump value for rjmp/rcall (#3012) 2022-09-07 23:00:23 +02:00
billow
46a1929397
Refactor rz_core_analysis_graph*() functions (#2949)
- fix `ags` and `agl` commands
- add more tests for the `ag` commands
- move graph related code to cgraph.c
- remove APIs `rz_core_print_bb_custom` `rz_core_print_bb_gml`
- exports
    - `rz_graph_drawable_to_json_str`
    - `rz_graph_drawable_to_cmd`
    - `rz_graph_drawable_to_gml`
    - `rz_core_graph`
    - `rz_core_graph_diff`
    - `rz_core_graph_format_from_string`
    - `rz_core_graph_type_from_string`
    - `rz_core_graph_write`
    - `create_agraph_from_graph_at`
- add unit/integration tests for the graph API
2022-08-29 22:48:23 +08:00
Giovanni
3752e24b98
Update rz-libdemangle (#2984) 2022-08-27 02:35:07 +02:00
Riccardo Schirone
9ba89d8116 core/cmd: rewrite rz_cmd_macro API 2022-08-26 07:53:43 +08:00
wargio
a73e8ecd4d Add references and flags to constant pointers to data maps 2022-08-13 22:52:24 +02:00
Giovanni
5655350535
Resolve golang string structures in data section (#2878) 2022-08-11 18:00:38 +08:00
Rot127
b46e7bd4f2
PPC uplifting to RzIL (#2823)
* Print a warning if config for VM was NULL.
* Add warning if register is not added to VM due to overlap.
* Update PPC register profile.
* Add vector and float registers. As well as some control and system registers.
* Enable to write values 4bit registers.
* PPC: Uplift most common instructions.
* Print warning if reserved SPR instruction is encountered.
* Update PC/LR addresses and add ca32 writes.
* Write cache needs to get flushed so load and store tests don't share the same memory.
* Add missing T/F branch mnemonics.
* Add Move to/from CR/CR0-7
* Fix TA address calculation for branch instructions.
* Add branch tests for branch mnemonics.
* Add XNOP
* NOP cache touch instructions.
* Add undocumented ATTN instruction to the not_implemented group.
* Add isel instruction.
* Implement CRCLR, CRSET, CROR.
* Add CNTLZ instructions.
* Add mcrf instructions.
* Add inacive test for cmpb.
* Add Load bytes reverse instructions.
* Add test and add address alignment to dcbz.
* Add eqv test
* Correct DIV and MUL operations
* Add div tests.
* Add tests and correct MT/MFXER
* Remove register ca32, ov32
* Remove explici setting of cr register because QEMU does not do it. Otherwise we get a mismatch in the trace
* Simplify carry set for add and sub. Sub instructions are exclusivly defined with addition. Hence no sub case needed.
* Unify BD and fix branch instructions. Fix: Check the single bit not the cr reg
* Document Conditional branches and replace NOPs with EMPTY
* Add mulli instruction to double word instructions.
* Fix ca set for shift instructions:
* ca value had to be determined before the shift happened. The wrong ca value was calculated if the src and target reg were the same.
* Replace NOP wit EMPTY.
* Use MSB isntead of SLT.
* Brought fixup of ADD and SUB instructions: The add and sub instructions had several issues which let to incorrect execution.
* The carry was incorrectly if three add operations happened (only the last add were checked, not both).
* The carry was incorrectly set if the src and target register matched.
* Same applies for the CR bit.
* It was too complex. Several local variables were introduced for this.
* Set result in local var, since it would change if src and target reg are the same.
* Remove MTMSR and MFMSR since it is too complex and untestable currently.
* Use unsigned int for shift. Otherwise the 0x1c shift produces a runtime error since 0xf is int as default.
* Fix mtxer: Only write flag bits.
* Let NOT_IMPLEMENTED macro return NULL.
* Mark st[wd]cx and l[wd]cx as not implemented.
* Increase dcache_line_size to 128 bytes.
* Fix cntlz for ppc32. m was set incorrectly, since it is 0 not 32 for 32bit cpus.
* Fix isel: Use op.crx reg instead of imm.
* Unify helper function names: Prependnig `ppc_` mark as IPI
* Add more "Move to SPR" cases.
* MULLI opeartes only on double word on 64bit CPUs.
* Most registers are now assigned the control register type and no longer show up in the ar command.
* Fix xor if dest and src registers match by saving result in local var.
* Fix BE/LE issue for Load BRX instructions.
* Fix shifts: Use only lower 6bits of n.
* User Pure local variables for ROT macros.
* Fix rldimi instructions.
* n was not inverted.
* more than 6 bits of n could be used
* Add 32bit emulateme tests.
* Add 64bit emulateme tests.
* Determine lg(v) in inline function.
* Calculate CR bit in C not in the VM.
* Check if `~mask = 0` and skip mask calculation if yes.
* Check for `sh == 0` and skip rotations where possible.
* Remove `la` instruction. `la` is a mnemonic for `addi`.
* Remove SPR instructions which are not supported by QEMU or not traced yet. For most set/read SPR instructions QEMU segfaults.


In case of SPR 1 (xer), 8 (lr) and 9 (ctr) the assembler resolves them to their mnemonics (mtxer, mtlr etc.). This means the code here is never reached.
To test the get_xer code MFXER was added again. The rz-tracetests will fail for this instructions (due to missing ca32, ov32). But this case is covert in an issue.
2022-08-11 08:41:15 +08:00
Rot127
720598f1f0
[Hexagon] Resolve duplex instructions dynamically (#2882)
* Chane HexInsn -> HexInsnContainer in asm plugin.

* Make HexInsn -> HexInsnContainer in hexagon_disas.c

* Add function to set concatinate textual disassembly parts.

* Switch expressions, fix segfault.

* Print number of unhandle ops, if they can not be added to RzAnalysisOp.

* Add lookup function to determine sub-instructions for duplex iclass.

* Fix asm tests.

* Add memcpy for instruction containers

* Fix off by one.

* Fix segfault if the packet held only the instruction which is currently resolved. `hic->bin.insn` would be NULL in this case.

* Remove duplicate index name.

* Set instruction address.

* Clean up hic->parse_bits use

* Fix: .new registers were not resolved dur to too early return.

* Fix rebase mistakes.
* Add missing code

* Fix build error: Use sizeof(`text_infix`) since it is way smaller than `text`

* Initialize `HexInsn` inside `setup_new_hic`. Fixes NULL access.

* Mark RzAsm/RzAnalysis members as deprecated and fix typo.
2022-08-08 08:12:53 +08:00
Dhruv Maroo
d44b5748eb
[RzIL] IL lifting of SuperH ISA (#2518)
* Add an assembler for the SuperH ISA
	* Rewrite GPL GNU disassembler code
2022-08-02 02:37:03 +05:30
Rot127
c009f8d7a3
Tokenize and colorize asm strings (#2417)
This introduces the ability to split asm strings into tokens and assign a type to them.

This means:
* Each asm strings is assigned a list of tokens. Each token points to a sub-string.
* A token has a type attached which describes it (e.g. register token, operator token, mnemonic token etc.)
* A generic parsing method is introduced which splits an asm string into tokens. The pattern it parses is `<mnem> <op> <op> ...`.
* Instead of the generic method, plugins can create the token strings on their own.
* An API method was introduced to help plugins implement their own parsing methods via regex patterns.
* Custom token strings are stored in `RzAsmOp.asm_toks`

Additionally the coloring of asm strings is now exclusively done via the token strings.
* For this a function was added to colorize tokenized asm strings according to their token types.
2022-07-27 08:52:01 -04:00
Zhang Zengbo
3b9e823db7 ARM pseudo subvar: don't substitude [pc, reg] 2022-07-21 22:24:06 +02:00
Rot127
2e48303443
TMS320 and TMS320c64x register profile (#2805) 2022-07-19 18:08:43 +08:00
Anton Kochkov
b2d9429129
Remove sdb json (#1118)
* Drop sdb_json in `librz/cons/grep.c`
* Do not print the JSON key in grep
* Use proper JSON paths in tests
* Use signed number in case of RzJson in tests
* Do not add quotes for sole RZ_JSON_STRING
2022-07-12 22:48:33 +08:00
billow
40ca22202b
Refactor rz_core_print_disasm_json (#2746)
* Split `rz_core_print_disasm_json` as: `rz_core_print_disasm_json` and `rz_core_disasm`
* Use `rz_str_enc_string_as_type` in `ds_init`
* Refactor `rz_core_print_disasm` and `rz_core_handle_backwards_disasm`
* Add `rz_core_asm_bb_middle`
* Move global `disasm_texts` to `RzDisasmState`
* Rewrite `core_analysis_bytes_standard` with `rz_core_analysis_bytes`
* Add `test_rz_core_print_disasm` and `test_rz_core_analysis_bytes`
* Rewrite `rz_core_print_disasm` with `RzCmdStateOutput`
* `pdi` -> `pdq` in tests
* `pdi` -> `pDq` in tests
* `pid` -> `pdq` in tests
* `pId` -> `pDq` in tests
* `pij` -> `pdj` in tests
* `pIj` -> `pDj` in tests
* `pdeqq` -> `pdeQ` in tests
* Remove `pDJ` from `test/db/json/`
2022-07-09 08:10:06 +08:00
Giovanni
b1f5b5b454
Remove usage of rz_bin_is_big_endian (#2761)
* Remove usage of rz_bin_is_big_endian which changes the endianness set by the user
2022-07-02 18:36:16 +00:00
Riccardo Schirone
32274f6975 Fix tests for .gnu_debugdata 2022-07-02 10:12:26 +02:00
Giovanni
f355dcf557
Add RzBinStrDb and api to interact with it via rz_bin_object_string_* (#2754)
* Add RzBinStrDb and api to interact with it via rz_bin_object_string_*
* use update instead of insert
* check for null deref in rz_analysis_set_stringrefs
* Never delete strings just add them and fixed comments
2022-07-01 06:34:41 +08:00
Riccardo Schirone
6f8aec2b3b
CI: install sigdb in static version of rizin (#2689)
* CI: install sigdb in static version of rizin
* RzCore: use bool2status whenever possible
* RzCore: add flirt.sigdb.load vars to select which signatures to load
* test: adjust testing to not consider system/home flirt db
* sign: use a RzSigDb structure instead of a raw list to store signatures
* sign: switch RzSigDb to use a hashtable and avoid duplicates
2022-06-23 10:29:46 +02:00
Luca Borzacchiello
2d69ecbb22
Better false-positives detection in rz_scan_strings (#2691)
* Improved false-positive detection in str_search

This commit adds the following features:
- Extend the false-positive check on ASCII frequencies to all UTF strings
- Add a global option to activate/deactivate di check
- Improve the false-positive heuristic by adding a special case for extended-ASCII strings
2022-06-14 22:14:13 +08:00
wargio
df0664b736 Add support for riscv64 string recovery 2022-05-18 19:24:53 +02:00
wargio
0187442d69 Add support for ppc64 string recovery 2022-05-18 19:24:53 +02:00
wargio
53de6e47a1 Add support for mips32/64 BE/LE string recovery 2022-05-18 19:24:53 +02:00
wargio
99985c89dc Refactoring using sigs for x64/x86/arm32/arm64
- initial support for mips32/mips64/riscv64/ppc64
2022-05-18 19:24:53 +02:00
Riccardo Schirone
e36d66abe7 Remove analysis.endsize config
Commit taken from radare2 project:
03afa631ff37ecb7585d3d4bb052f7208e68c25dA ("Kill anal.endsize ##anal")
2022-05-18 10:25:01 +02:00
Rot127
fb665274f4
Fix: Hexagon relocations (#2612)
* Add more relocation types.
* Replace flawed patching algorithm with the `llvm-llds` algorithm.
* Add macros for bitmasks.
* Remove code duplicates.
* Add relocation tests.
2022-05-13 20:26:31 +08:00
Giovanni
423617930c
[dalvik] handle move-exception after gotos/returns and mid op jumps (#2603)
* Fix dalvik analysis by looking for move-exception after gotos/returns
* Fixed Dalvik analysis due compiler optimizations
2022-05-12 13:16:54 +02:00
Giovanni
0d4d8f5892
Ported aa* commands under newshell (#2594) 2022-05-10 19:07:20 +02:00
Riccardo Schirone
27e0d7ae82
RzBin: improve handling of addresses in ELF (#2591)
* RzBin/ELF: avoid the creation of useless flags at 0

Use UT64_MAX as an error value for addresses like symbols, imports,
sections, etc., and do not create flags for those because they are not
meaningful. This is useful for imports that do not have an address in
the virtual/file space and for non-alloc sections (e.g. .comment).
2022-05-10 10:17:35 +08:00
Paul I
df354a4cec
Convert af commands to rzshell (#2586) 2022-05-05 18:19:17 +08:00
wargio
4a06141377 Fix 2128 - Parse go.buildinfo and set compiler value under iI 2022-05-03 12:13:22 +02:00
wargio
0750370104 Added string checking on golang parsed tests. 2022-05-03 12:13:22 +02:00
wargio
01643ab3e0 Use rz_str_len_utf8_ansi for detecting valid string length 2022-05-03 12:13:22 +02:00
wargio
f9d74c9115 Added tests and fixed some others 2022-05-03 12:13:22 +02:00
Florian Märkl
3cbde10260
Update Hexagon for compact struct conversion (#2548)
This is updating with rz-hexagon 1ca78d073861b6ce66b682d200c7a3fc2734a6a3
2022-04-22 17:50:39 +02:00
Paul I
d24a325df1
Remove afC commands (#2555) 2022-04-22 12:57:11 +02:00
Martin M
74c071b071
Fix disassembly of LDD instruction on AVR (#2519) 2022-04-08 21:23:15 +02:00
Rot127
10bedc2b00
Add Hexagon register profile (#2490)
Although the reg profile was not really used yet it will be relevant for the RZIL uplifting.
Introduces:
* Double/quadruple registers now properly overlap with their counterparts.
* Registers have correct types assigned (vec, ctr, sys etc. Before, all regs were of type gbr)
* Consistent register names (asm name style: c3:2, r31 etc, alias style: lc1:sa1, lr etc.)
* Adds _tmp registers in the profile. Later used by uplifted instructions (think of .new registers).
2022-04-03 18:26:17 +02:00
Giovanni
348ed19ad6
Dex add support for some imports within direct_methods via super calls (#2491)
* Dex add support for some imports within direct_methods via super calls
* Fixed tests and added test
2022-04-03 20:54:55 +08:00
Rot127
628d04f713
Add RzConfig for asm plugins (#2488)
* Add function to add config node to config by `RzConfigNode*`
* Move `SET<NODE>` macros to `rz_config.h`
* Add un/set functions for plugin specific configs.
* Add RzConfig and node setter to hexagon plugin.
* Use plugins RzConfig to determine sdk syntax setting.
* Use plugins RzConfig to determine if numbers should be signed.
* Use plugins RzConfig to determine if immediate hash should be printed.
* Remove unused parameter `rz_asm`.
* Remove unused variable.
* Remove unused callback.
* Remove unused flags from `RzAsm`
* Fix asm tests: Add NULL check; Set correct sign in asm test.
2022-04-03 20:44:50 +08:00
Giovanni
4fb8c83e94
[DEX] Add relocation support 2022-04-02 16:23:43 +02:00
Paul I
5ea3630c35
Convert ab commands to rzshell (#2395) 2022-04-02 21:27:55 +08:00
Riccardo Schirone
c2dd9df426 RzCore: print "lib" field in 'is' command also in JSON mode 2022-04-01 11:57:50 +02:00
Giovanni
63622fa9e6
[RzIL] AVR lifter (#1890)
Added all the ops excluding des and spm due their complexity
Added tests to verify output of each execution cycle.
2022-02-22 18:54:55 +01:00
Florian Märkl
bfb98ed20c
[RzIL] Add lifter for 32-bit Arm, including thumb (#2241) 2022-02-22 12:23:28 +00:00
Florian Märkl
102549102d
Refactor Arm IT handling and fix for 4-byte ops (#2336)
The code was duplicated in both asm and anylsis, so now it's in
arm_it.c.
Previously, conditions were applied to N offsets after an IT instruction
using a fixed stride of 2 bytes. This obviously breaks for 4-byte thumb2
instructions, but whenever we find such an instruction, we can check if
there is a condition inside of it and readjust if necessary.
2022-02-19 17:16:15 +01:00
Rot127
3d71687d5e
Add Hexagon IDE conforming packet syntax (#2209)
* Add option to print instruction packets syntax in objdump style.
* Add the config `plugins.hexagon.sdk` to toggle syntax.
* Remove unused trimming.
* Trim only newlines from the assembly string.
* Add new trim functions to allow trimming of certain chars.
2022-02-14 02:27:55 +08:00
Florian Märkl
f729512817
Fix pcalign after changing analysis bits (#2306)
This fixes a bug that showed up after 2bdec10f98
because the bits value was set after cpu, but pcalign was already 4 then
and not updated by bits for example in:

rz-asm -a arm -b 16 -o 0x1059e -A dff81000
2022-02-13 09:28:33 +01:00
Riccardo Schirone
dea0918afd
Convert wa commands to rzshell (#2293) 2022-02-09 18:49:11 +08:00