Adds the core implementation of the new search.
The rough architecture is the following:
A search for a certain type of information (strings, bytes, keys etc.)
creates a collections of items to search for (byte patterns, regular expressions etc.).
Then specifies some settings how the search (number of threads, maxum hits...)
and the finding is performed (string length, inverse match etc.).
It also defines a search space, which is currently only the IO buffer.
But can be anything in the future, like a graphs or the knowledge base.
The search splits up the search space into windows (for IO: address ranges)
and dispatches each search window into a 'find()' thread.
The 'find()' handler (provided by a specific search implementation)
checks the given window and produces search hits matching the elements in the search collection.
The main search handler collects the hits of the dispatched workers
and returns them to the user.
Note: The byte and string search implementations are added in the next two commits.
Part 6/9. Likely won't build in between parts.
Co-authored-by: wargio <deroad@kumo.xn--q9jyb4c>
As due to https://github.com/mesonbuild/meson/issues/9702 we are not
using the expansion of @PACKAGE_INIT@ anyway, the cmake module provides
little additional value over a plain configure_file().
More importantly, by not using it, we get full support for muon, which
does not implement the cmake module yet.
Duplicated code for pkg-config and cmake configuration is avoided by
letting every rizin module register itself in a global dictionary
called "modules" which is then iterated in one place to perform any
shared logic, inspired by how qemu handles multiple targets in its meson
build system.
This is an almost pure refactor, so the resulting .pc/.cmake files
should be identical with the following exceptions:
- rz_reg.pc does not have plugindir anymore as rz_reg has no plugins.
- Some other modules have their dependencies in .pc files altered to be
consistent with cmake.
* Move binrz meson directives to binrz/ dir
* Move rizin-shell-parser to a subproject
* Move bochs to a meson subproject
* Move rzqnx to meson subproject
* Move winkd to a meson subproject
* Move rzar to a meson subproject
* Move rzw32dbg_wrap to a meson subproject
* Move ptrace-wrap to a meson subproject
* Move rzgdb to a meson subproject
* Rename w32dbg_wrap to rzw32dbg_wrap
* Update CODEOWNERS
* Remove old references to shlr
* Move shlr/heap stuff in core and remove shlr/arm
* Move spp to a meson subproject
* Remove last references to shlr
* Remove use_webui option
* Move include files directives to librz/include/meson.build
* Move librz meson directives into librz/meson.build
* Handle d/ directories inside the specific module meson.build
* Move plugins listing to specific module meson.build
* Move rzheap to its own subproject
* Fix js linter and licenses
* Use meson.override_dependency for all rz_ modules
This brings our concept of effects in line with BAP and fixes issues
like values being evaluated prematurely in seq. The main point is that
ops themselves are the effects, rather than returning them to be
evaluated later.
* Remove wrong librz.pc.in file
* Add plugindir variable to .pc files to allow easy plugin configuration
* Add CMake config files to find Rizin in CMake projects
* Add CI jobs to check cmake files
* Provide workaround for mesonbuild/meson#9702
* Use .lib suffix and no prefix on MSVC to make CMake files work
* Disable cmake files on static libraries build
* Remove `RIZIN_EXTRAS` because useless
Extras plugin should be treated as any other regular plugin, they should
not have their own directory. I believe we did not have anyway any
plugin using that directory.
* Do not use `<version>` in `/usr/share`
Users can only install one version of Rizin at a time anyway, thus
splitting the `/usr/share/rizin` directories does not really bring
multi-version install possible. At the same time, we stick to what's
common for other tools, which do not use version in `/usr/share`
* Move plugin directory to `/usr/lib64/rizin/plugins`
* Remove `RZ_HOME_BIN` env var because not useful anymore
* Introduce `rz_util/rz_path` to provide rizin paths
* Use `rz_path_system_sdb_types` whenever it makes sense
* Use some others `rz_path_*` functions instead of custom ones
* Use `rz_path` APIs for themes, fortunes and signatures
* Remove `rz_sys_prefix`
* Reduce the number of defines needed to handle home and system paths
* Reduce the number of `rz_path` functions
* Make the lib debug depending on log level and `RZ_LOG_DEBUG`
* Refactor a bit how directory of plugins are loaded
* Keep supporting old plugins directory for at least 0.4.0
The new intermediate language is based on the BAP (Binary Analysis Platform) Core Theory.
It operates on SMT-like bitvectors and their arrays to represent values and memory.
Apart from that, it has also representation for the data and code side effects.
* Added demangling as plugins and libdemangle as external dependency
* Added iDl to return the list of demanglers
* moved demangling tests into rz-libdemangle and keep only bin tests
* Refactored string search
string search logic moved from librz/bin/bfile.c to
librz/util/str_search.c
* UTF-16BE and UTF-32BE support
* Decoupled rz_str_search with rz_bin
* Changed parameter order in rz_scan_strings
* Fixed needle reset in rz_string_search
Now when the string search fails, needle is restored to original_needle+1,
i.e., we continue the searching process starting from the next byte.
* Changed heuristic for UTF BE detection
one more byte in for the detection (6 instead of 5)
* Fixed unit test in str_search
* Refactored rz_scan_strings and better UTF16-LE and UTF16-BE detection
Now if there is an ambiguity between UTF16-LE and UTF16-BE, we take the
longest one.
* Changed UTF BE heuristic
If there is any ambiguity about big-endian or little-endian, always
choose little endian
* Small format fixes in str_search
* Added preferred endianess switch in RzUtilStrScanOptions
* Fixed str_search regression tests
* Added API calls of windows heap for Cutter
* Added header to meson build
* Separate windows specific code
* Move API calls to `cheap.c`
* Add documentation and RZ_OWN
* Introduce `RzArchProfile` and add the CPU profiles
* Create `rz_arch.h` and `arch_profile.c`
* Add the CPU Profile's SDB files
* Add the `meson.build` for compiling SDB files
* Add the hashtables and the flagspaces
* Add a test and compare the `type` in `rz_sysreg_get`
* Update the list of AVR CPUs in `asm_avr`
* Run clang-format and include rz_arch.h in rz_analysis.h
* Fix the install directory of the compiled SDB files
* Introduce `RzArchTarget` which houses `RzArchProfile`
* Support for loading a new profile when `asm.cpu` is changed
* Change the variables names to lowercase and add tests
* Add a test for loading a new CPU profile when CPU is changed
* Add Doxygen documentation, run `clang-format` and correct the requested changes
* Remove unnecessary zero-initializations and other improvements
* Reversed a condition while creating the path in `rz_arch_profiles_init`
* Fixed four tests: three regarding printing the flagspaces and one from printing flags after analysis in AVR
* Free the required pointers and check for `NULL`
* Free the pointers that was allocated previously, before returning `NULL`
* Free the pointer in `rz_sysregs_db_new`
* Support for adding a fallback profile for unsupported CPUs
* switched to a new branch cause the old one was messy
* add test for srec reading
* finished SRecord PR
* 32 bit records require S7 terminator
* added srec with arm elf
Co-authored-by: wargio <wargio@libero.it>
Co-authored-by: Anton Kochkov <anton.kochkov@gmail.com>
* Added authors and licenses and plugin access to crypto related files
* removed `rz_hash` and related files
* introduced `rz_msg_digest_*` with plugins
* updated all old methods with newer ones.
* added HMAC support to `rz-hash` (moved key option under `-K`)
* removed wrong algorithms on `woD`/`woE`
* fixed parity digest size
* fixed the behaviour on all hash functions
* added unit test for hash
* optimized code for HMAC key calculation
* removed hash legacy tests
* RzType introduction
* Move format processing to RzTypes from RzUtil
* Rename RzType to RzTypeDB
* Join RzType and RzAnalysis noreturn functions
* Add noreturn column in `aflt` output
* Remove D (disassembly) pf tests
* Fix `tn` test
* Add function noreturn serialization
* Modify meaningless type test
* Fix wrongly written `Cf` test
* Fix `za t` (zignature type) command
* Fix signature type test
* ##bin add bin plugin for luac format
* ##build Update meson scripts and some headers for luac bin plugin
* Update luac specified macros
* Update luac struct according to packed ines_hdr
* Update info function in luac plugin
* Reconstruct bin_luac for version switching
* Update meson script
* Add {-I} function for luac bin plugin
* Add some load utils of lua bin plugin
* Add src file info as guid info in lua bin plugin
* Define INNER_BUFFER_SIZE as macro
* Fix stolen byte in luac plugin and update test script
* Add asm plugin to target and enable it
* Implement a quick and dirty simple lua 5.4 analysis plugin
* Implement luac bin plugin sections, symbols, entry
* Test luac bin format {is, iz, iS, ie, pd}
* Add luac disasm test
* Add luac analysis test
Removed unused code and comments.
Removed dead code
Moved print method outside class.c
Moved json method outside class.c
Removed yc_w_refs and i_mref which never worked.
Fixed set_flags method to work as intended.
Fixed various parsing errors.
Fixed many memory leaks.
Fixed "insecure" rz_cons_printf warning.
removed `#define V`, `#define U` and `#define _`
Implemented newshell calls.
Optimized code.
Refactoring rz_bin_java_get_field_offsets
Avoid loading static core plugins when not needed.
Removed useless `if 0`
