* arch/tms320: add TMS320C54x disassembly support
Add a C54x instruction decoder that reuses the shared C55x decode engine
(c55_decode/c55_format) via the C55ArchDesc plug-in interface, rather than
duplicating the matcher/formatter. Disassembly only for now (.lift = NULL).
Engine changes (c55_ir.c/.h):
- add C55ArchDesc.words_le so the decoder can byte-swap the little-endian
16-bit instruction words used by the C54x COFF object format;
- add a self-contained C54x memory-operand renderer (direct @dma, MMR,
indirect *ARx with all post-modify modes, *ARx(lk) const-index, *(lk)
ABS16 absolute and circular '%' addressing) and bare-hex immediates;
- add C55Operand.circular for the '%' suffix and C55Operand.space_join
for the space-separated second half of a C54x parallel instruction;
- extend the data-memory operand-field analysis (register, base pointer,
displacement, direction, referenced size) to the LOAD/STORE op types the
C54x ld/st family uses, in addition to the C55x MOV form.
The C54x decoder (isa/tms320/c54x/c54x.c) covers the complete documented
instruction set - all 117 mnemonics of the SPRU172 opcode map, in every
documented encoding form:
- load/store/move, integer and logical ALU ops in every addressing form
(Smem, #lk, dual-accumulator, Xmem/Ymem, TS/ASM/SHIFT-shifted, the
shift-by-16 and #lk,16 long-immediate forms, and the two-word
Smem,SHIFT form whose operation selector lives in the second word);
- the full multiply/MAC family: Smem, #lk, program-memory, squaring,
multiply-by-A, signed-unsigned and the dual-operand MAC[R]/MAS[R]
Xmem,Ymem forms;
- the parallel (dual-operation) class rendered "op1 .. || op2 .." -
ST||ADD/SUB/LD/MPY/MAC[R]/MAS[R], ST||LD T and LD||MAC[R]/MAS[R];
- double/long-word (Lmem) add/subtract, the unary accumulator ops
(exp/norm/abs/neg/rnd/sat/min/max/rol/ror/sftc/cmpl/...);
- control flow with the separate delayed (bd/calld/bcd/banzd/fcalad/...)
variants, conditional return/execute (rc[d]/xc) and the multi-condition
"tc, c"-style combinable condition fields, repeats (incl. rpt #lk),
conditional stores, I/O port access, status-bit set/clear and the
non-linear idle encoding.
Operands resolve to their architectural names - the full memory-mapped
register file (AR0-AR7, the accumulator AL/AH/AG/BL/BH/BG halves, T, TRN,
SP, BK, BRC/RSA/REA, IMR/IFR, PMST, XPC), the ST0/ST1 status bits and the
named condition codes; the memory-mapped-register operand is kept single
word (its long-offset modes are not legal). The analyzer classifies every
instruction (op->type, op->id), resolves branch/call targets and the stack
effect of calls/returns/pushes, and exposes operand details: the register,
base pointer, displacement and access direction of data-memory loads and
stores, and the target register of indirect branches/calls.
All encodings were verified byte-exact against the TI asm500 assembler,
and every decoded instruction re-assembles to an identical encoding (a
full-opcode-space disassemble/reassemble round-trip is stable). A 297-case
disasm test suite and an analysis test suite (opcode classification, branch
and call targets, stack effects, memory-operand fields, data-immediate values, the register
profile, named instruction ids and COFF binary-fixture function discovery)
are added, and the real-world emulateme C54x .text decodes cleanly.
* arch/tms320: add TMS320C54x RzIL lifting
Lift the C54x integer core to RzIL so emulation and IL-based analysis work
for C54x as they already do for C55x/C55x+.
- Register profile: C54x previously fell through to the C64x profile
(a0-a31, =PC pce1), wrong for the A/B accumulator core. Add a proper
C54x profile: the two 40-bit accumulators A/B (with the L/H 16-bit and
G 8-bit guard slices overlapping their parent), AR0-AR7, T/TRN, SP, DP,
BK, ST0/ST1/PMST, BRC/RSA/REA, IMR/IFR, XPC and a 24-bit PC.
- IL VM config: tms320_c54x_il_config() binds the canonical registers; the
accumulator slices stay unbound, the lifter expresses them as bit-slices
of A/B so they never desynchronise.
- Lifter (C55ArchDesc::lift hook, dispatched by c55_lift): the no-shift
forms of LD/LDU/LDR/LDM, ADD/SUB/AND/OR/XOR, STL/STH/STLM/STM, the mvd*
memory-to-memory moves, the DLD/DST 32-bit double-word load/store (high
word at the lower address), PSHM/POPM and RET. Shift/round/saturate
variants are left unlifted (their shift count is carried only as a
display string); the engine's generic EA/read/write/post-modify helpers
are reused for the addressing modes.
Tested via two new RzIL VM blocks in test/db/rzil/tms320: a register/
immediate/memory execute test, and an end-to-end emulation of the
emulateme binary's _decrypt (a UART hex-writer) showing the IL VM emits
the hex digits and advances the write position.
---------
Co-authored-by: Anton Kochkov <anton.kochkov@gmail.com>
* Cast operands for AND and OR instructions to the correct width
* Add missing operand casts for SBB and ADC.
* Add flawed instructions to asm tests
---------
Co-authored-by: Dhruv Maroo <dhruvmaru007@gmail.com>
* Add reliable http:// test
* REUSE.toml: Add `test/www/**` entry
* Use `cwd` instead to work around old http.server in Python 3.6
* Move test to `not-windows-any`
* NetBSD: Add `python3` symbolic link
* Prevent test from running on woodpecker
Verified against the TI dis55 disassembler and the C55x+ Algebraic
Instruction Set (SWPU104) encoding tables.
Co-authored-by: Anton Kochkov <anton.kochkov@gmail.com>
Builds on the shared decode-IR engine to lift a broad set of common C55x and
C55x+ (Ryujin) instruction forms that were previously left without RzIL,
deriving the exact semantics from the TI C55x+ references (SWPU104 / SWPU086).
Control flow and addressing:
- ret / reti / retcc: return address read from the top of stack, SP popped by
two words, control transferred to it (retcc guarded by its predicate).
- 24-bit XAR correctness on C55x+ (XAR0-15, XSP, XSSP, XDP, XCDP) in the
register table, the IL-VM profile and the pointer-arithmetic width, enabling
the 24-bit amov address immediate; classic C55x keeps its 23-bit file.
- long constant-index and 16-bit-absolute addressing modes, memory-to-memory
copy, shifted memory loads (uns()/signed, immediate shift), and the
memory-mapped-register moves mov reg,mmap(@reg) / mov mmap(@reg),reg.
- push/pop of accumulator sub-register halves and dbl(xarN) pointer pairs.
Arithmetic, logical, shift and bit operations:
- sub-register add/sub on a 16-bit accumulator slice, for the immediate and
register forms: a .L destination updates [15:0], a .H destination updates
[39:16] sign-extended through the guard (SWPU104 1.5.1).
- bitwise and/or/xor with a 16-bit source (half or AR/T) into a full
accumulator, zero-extending the operands to 40 bits (SWPU104 6.6.1); the
memory-source bitwise forms into full and half destinations; and the
shift-ALU forms <op> ACx.<sub> << #S6, ACy.<sub> on the 16-bit slice.
- memory-source add (including a 16-bit half addend and the reverse-subtract
sub ACx.<sub>, Smem, ACy.<sub>), the 32-bit dbl(Lmem) add/sub forms
(ACy = ACx +/- dbl(Lmem) and the reversed ACy = dbl(Lmem) - ACx), and the
memory-destination immediate RMW add/sub #k, Smem.
- register bit ops bclr / bset / bnot @#k, ACx[.h/.l] / ARx: clear, set or
toggle bit k of the register, the bit number taken relative to the
addressed sub-register (a .h operand targets bit k+16, the guard k+32).
- bitwise not ACx.<sub>/ARx into an accumulator half or a 16-bit register
(not ACx.l, ARy), the half-register and short (0x7b #1/#-1) sftl/sfts
including register-count shifts, and btst @#k, ACx.l/.h, TCy.
This also corrects an op_type-fallback mis-lift: on C55x+ several non-move
instructions (round, sat/satr, the mant/nexp helper) and the bit-field
extract/expand bfxtr/bfxpa were typed as a move and so were lifted as a plain
register copy. round and sat/satr now carry their lops and reach the existing
rounding / saturation handlers (matching C55x); mant/nexp, bfxtr/bfxpa, the
operand-less sat and the register-indexed memory bit ops (bclr/bset/bnot Baddr)
are marked decode-only (no modelled data effect) so the fallback can no longer
guess at them.
Also fixes a FIRSADD/FIRSSUB lifting bug: the Cmem operand was not converted
from AR to XAR before emitting its pointer post-modify, so a post-modified Cmem
(e.g. firssub *ar3-, *ar5-, *ar6-, ...) produced an invalid 16-bit-vs-24-bit
subtract that failed IL validation; Cmem is now widened like Xmem/Ymem.
The multiply/MAC family stays unlifted by design, as do the flag-predicate
retcc forms (whose condition register field is not represented by the decoder)
and the software-interrupt intr (which would need the interrupt-vector base);
the correct-or-NULL contract test asserts a representative deferred form.
The disassembler tests for both variants are extended with the expected IL for
every form that lifts -- previously the optional IL field was omitted on many
lines whose lift already existed, leaving the lifting unchecked -- so the asm
suite now validates RzIL for all lifted C55x / C55x+ instructions, not just a
subset.
Measured on a 5 MB C55x+ firmware image (16k-instruction sample) RzIL coverage
rises from 82% to 99.4% of decoded instructions, the remaining tail being the
multiply/MAC family and the deferred forms above.
Replace the per-variant, table/token-driven disassembly and lifting for the
TMS320C55x and C55x+ DSPs with a single shared decode-IR layer, c55_ir, that
both variants drive through a C55ArchDesc descriptor. The shared engine
table-walks an instruction (c55_decode), formats it (c55_format), fills the
analysis op (c55_fill_analysis), and lifts to RzIL (c55_lift) arch-agnostically,
with the opcode tables, register files, operand extractors, and op-type/lift
mappings supplied per variant (c55x and c55x_plus).
Both test corpora decode and lift entirely through the shared engine: forcing
the shared-only decode path and, separately, the shared-only lift path each pass
the full suite -- 1288 tests including the three RzIL VM emulation tests, whose
decrypt loops execute every instruction through the shared IL. The legacy
decoder and lifter are retained only as a fallback for a complex long-tail (the
parallel dual-MAC group, dual-memory addsub/subadd, the absolute-k24 and dbl
multi-operand loads/stores, and the bit-field forms); the shared path is used
when it can decode an instruction and control falls through otherwise, so
behaviour is preserved at every step.
Several forms are extended beyond the old engine: the full C55x+ register file
(ac0-31, ar/xar0-15, CPU-gated in the profile and IL VM) so extended-register
forms lift where the legacy lifter bound only the low eight; pc-relative and
compare-and-branch control transfer; the bcc flag predicate; and the
single-data-memory bitwise and address-arithmetic forms (and/or #k16,Smem and
register-mode amar Smem,xar) decoded cleanly with RzIL rather than inherited from
the legacy tables.
Squash of rizinorg/rizin PR #6434 ("improve TMS320C55x+ analysis and RzIL")
rebased onto dev, with the PR head's doubled c55plus_il.c (every symbol defined
twice, failing to compile) de-duplicated to a single clean copy.
Substantially extends the C55x/C55x+ RzIL lifter over the existing structured-
operand helpers: mov/copy (immediate, register, memory load/store, half-register
read-modify-write), the full addressing-mode set with post-modify side effects,
control and system-register moves, 40-bit accumulator ALU with shifted sources,
16-bit and dual-memory add/sub, the ST0_55 status-flag model (cmp/cmpand/rol/ror,
named-bit bset/bclr), a documented psh/pop stack model, address-unit amov/aadd/
asub and amar, and bcc/callcc control transfer. The multiply/MAC family and satr
are lifted with explicit, documented integer-mode approximations (not verified
DSP semantics); irreducibly multi-output primitives (bit counts, Viterbi, FIR,
distance) are left correct-or-NULL. The register file covers ac0-7 and xar, found
by validating on real Motorola Wrigley C55x+ firmware whose prologues save 40-bit
accumulators as dbl(acN)+acN.g pairs.
Adds RzIL-VM emulation tests (including the C55x and C55x+ _decrypt emulateme
binaries), per-instruction IL assertions, and ~95% instruction-class disassembly
coverage per corpus; pins little-endian in the VM tests for big-endian hosts; and
regenerates the analysis expectations against current dev.
For a fully linked TI COFF executable the section s_vaddr fields hold the real
load addresses (e.g. .text at 0x100, vectors high) rather than a packed
sequential layout. Map sections at s_vaddr for F_EXEC objects (keeping the
sequential 16-aligned fallback only for relocatable objects), treat a defined
symbol's n_value as an already-absolute address instead of re-basing it onto the
section VA, and give non-loadable sections (DWARF/debug, build attributes,
.pinit) no loadable vaddr so their code-valued debug entries no longer split
instructions mid-stream or shadow the real low-addressed loadable sections. The
formats/coff test expectations are updated accordingly.
Add a new LGPL-3.0 VAX-11 architecture plugin that replaces the removed
binutils-derived GPL one. It is written from scratch from the documented
VAX operand-specifier encoding and does not reuse any GPL code.
location_by_biggest_range() computed each location-list entry's PC-range
size as (begin - end). For a normal [begin, end) range (begin < end) this
underflows and wraps to a huge ut64, so the entry with the *smallest* span
was always chosen as a variable's single representative storage instead of
the largest.
This breaks functions whose register arguments and locals are described by
location lists, e.g.
item: [low, X): DW_OP_reg0 ; [X, high): DW_OP_reg8
input_buffer: [low, Y): DW_OP_reg1 ; [Y, high): DW_OP_reg10
with DW_AT_frame_base = DW_OP_call_frame_cfa (.debug_loc + DW_AT_GNU_locviews,
no .debug_loclists). The wrongly-picked short entry is frequently one that
does not resolve to a valid RzAnalysisVarStorage (e.g. an implicit
DW_OP_stack_value piece), leaving the variable with EVAL_PENDING storage. The
affected variables then fail to materialize and 'afv'/'afvl' reports nothing
for the whole function -- even though the arguments live plainly in registers
and need no CFA computation.
Computing the span as (end - begin) selects the genuinely largest range, so
each variable resolves to the register it occupies for most of the function
and the register arguments load correctly.
Co-authored-by: Anton Kochkov <anton.kochkov@gmail.com>
* util/vector: hoist quicksort scratch buffers out of the recursion
vector_quick_sort allocated its two element-sized scratch buffers (t and
pivot) with malloc/free on every recursive call. For a vector of n elements
the sort makes O(n) recursive calls, i.e. O(n) malloc/free pairs purely for
scratch space, and each call could also fail half-way through the sort.
Split the function into a small entry point that allocates the two buffers
once and a recursive worker that receives them as scratch. The buffers are
reused across the whole recursion (each partition step finishes using them
before recursing, and the recursion is sequential, so sharing one pair is
safe). Small elements -- the common case, including every RzPVector-backed
sort -- use stack buffers and allocate nothing at all; only elements larger
than 256 bytes fall back to a single heap allocation for the whole sort.
The element movement and rand()-based pivot selection are unchanged, so the
result is identical for any input (verified byte-for-byte against the previous
implementation for ascending and descending orders over many random arrays).
* util/vector: evaluate the comparator once per element in the quicksort
The partition loop tested the element against the pivot with two separate
calls to the comparator:
if ((cmp(VEC_INDEX(a, i), pivot, user) < 0 && !reverse) ||
(cmp(VEC_INDEX(a, i), pivot, user) > 0 && reverse)) {
Because cmp is an opaque function pointer the compiler cannot common up the
two calls, so depending on the result and the reverse flag the comparator was
invoked up to twice per element. Compute the result once into a local and test
that:
int c = cmp(VEC_INDEX(a, i), pivot, user);
if ((c < 0 && !reverse) || (c > 0 && reverse)) {
This halves comparator calls in the worst case and is a clear win whenever the
comparator is non-trivial (the common case for struct elements). Measured on a
shared host: ~12-14% faster for int sorting and ~30% faster with a moderately
expensive comparator. The ordering is unchanged (verified byte-for-byte).
* util/vector: simplify rz_pvector_remove_data index computation
The index of the located slot was computed as
size_t index = (el - (void **)vec->v.a) * sizeof(void **) / vec->v.elem_size;
For an RzPVector the element size is always sizeof(void *), so the
`* sizeof(void **) / vec->v.elem_size` factor is identically 1 and the pointer
difference `el - (void **)vec->v.a` already yields the index directly. Drop the
redundant scaling, which removes a multiply and a divide and makes the intent
clear. Behaviour is unchanged.
* test/unit: add RzVector sort and rz_pvector_remove_data regression tests
The existing sort tests only sort 4-5 small elements and there was no test for
rz_pvector_remove_data. Add coverage for the code paths exercised by the sort
changes and the remove_data cleanup:
- test_vector_sort_large sort 2000 heavily-duplicated ut32 values
ascending and descending, verifying the result
is ordered and a permutation of the input (vs a
reference qsort). Drives the recursion deeply
and the shared scratch buffers.
- test_vector_sort_large_elem sort 400 elements of 304 bytes each, taking the
heap-allocated scratch fallback, and check the
full payload (not just the key) stays consistent
through all the element moves.
- test_pvector_remove_data remove interior, first and last elements by
value while preserving order, and confirm
removing an absent value is a no-op.
All pass on both the previous and the optimized implementation (the sort and
remove_data changes are behaviour-preserving).
* test/bench: benchmark rz_vector_sort and rz_pvector_sort
bench_vector.c benchmarked only remove_at and swap. Add sort benchmarks so the
suite covers the functions touched by the sort optimizations and can be run
against the old and new librz for before/after numbers:
- rz_vector_sort over 4k ut64 with a cheap comparator
- rz_vector_sort over 4k ut64 with a deliberately expensive comparator
(shows the effect of evaluating the comparator once per element)
- rz_pvector_sort over 4k pointers (reference; pvector sort is unchanged)
Each iteration refills the buffer from an unsorted master copy via a single
memcpy before sorting; that overhead is identical across builds so the measured
delta reflects the sort.
---------
Co-authored-by: Anton Kochkov <anton.kochkov@gmail.com>
MIPS PIC code calls a function by loading its address from the GOT into
$t9 and then doing `jalr t9` (or `jr t9` for a tail call), e.g.:
lw v0, -sym._MIPS_STUBS(gp) ; v0 = *(gp + %call16(puts))
move t9, v0
jalr t9 ; -> puts
rizin did not turn this into a call to the imported function, so the
target was lost: rz-ghidra rendered it as an indirect `(*_data.XXXX)()`
instead of `puts(...)`, whereas a direct `jal sym.dummy` decompiled fine.
Root cause (two independent gaps):
1. For (R|U)CALL and RJMP ops the core creates the CALL/CODE xref from
op->ptr, not op->jump (see core_analysis_followptr() and the op-type
switch in librz/core/canalysis.c, RZ_ANALYSIS_OP_TYPE_RCALL/RJMP). The
MIPS plugin only ever set op->jump for `jalr`/`jr`, so no call xref was
produced even when $t9 was tracked, and the decompiler never saw a call
target.
2. $t9 was only tracked when it was the *direct* destination of a
gp-relative load (`lw t9, ...(gp)`). The very common sequence that loads
into another register first and then `move t9, vX` was not tracked, so
even op->jump was left unset there (this is the issue's binary). Note
that capstone emits `move t9, vX` as the 2-operand alias of `or` (and on
some toolchains `addu`/`daddu`), i.e. `or t9, vX, $zero`, so the move
must be recognised across MIPS_INS_MOVE *and* the 2-operand OR/ADDU
forms.
Additionally, the tracked value was the GOT *slot* address, while the call
target is the function the slot points to, so the slot has to be
dereferenced.
This commit:
- tracks the destination register and slot of every gp-relative load
(gp_load_reg/gp_load_ptr in MIPSContext), and propagates it to $t9 on a
register move (MIPS_INS_MOVE, or the 2-operand OR/ADDU alias), so the
PIC sequence above is recognised;
- adds mips_pic_call_target(), which dereferences the GOT slot via the
analysis IO bind (honouring word size and endianness) to obtain the
actual callee;
- sets op->ptr (and op->jump) to that resolved address for `jalr t9`
(RCALL) and `jr t9` (RJMP tail call), so the core emits the proper
CALL/CODE xref and the decompiler resolves the callee like it does for
`jal`.
The resolution is best-effort and fully guarded: if gp is unknown, the
slot cannot be read, or it holds 0, op->ptr is left unset and behaviour is
exactly as before. The core additionally validates the target
(is_valid_xref) before creating the xref, so a stale/garbage slot cannot
introduce a bogus call, and the propagation only fires when $t9 is the
destination, so ordinary moves (`move fp, sp`, ...) are unaffected.
Co-authored-by: Anton Kochkov <anton.kochkov@gmail.com>
Use rz_read_le32() in the x86/x64 egg emitter so 4-byte string immediates are
no longer byte-reversed on big-endian hosts (System Z/s390); little-endian
output is unchanged. Regenerate the stale 32-bit x86 (#1889) and 32-bit arm
rz-gg goldens to match the tool's actual output, drop BROKEN markers from the
simple_cmp tests that already pass, and annotate the remaining broken tests
(no AArch64 egg backend; compiler-dependent C output) with the reason.
Closes#3486Closes#1889
Co-authored-by: Anton Kochkov <anton.kochkov@gmail.com>
* hash: add chi-square (vs uniform) rz-hash plugin
Adds a chi-square goodness-of-fit (vs a uniform byte distribution)
statistic as an rz-hash plugin and the rz_hash_chisquare() API.
Unlike Shannon entropy, chi-square separates high-entropy data that is
truly uniform (encryption/CSPRNG, ~255) from high-entropy data that is
merely compressed or packed (much larger values), which is a common
question when triaging firmware blobs.
* hash: add index-of-coincidence rz-hash plugin
Adds the index of coincidence as an rz-hash plugin and the
rz_hash_ioc() API. IoC is ~1/256 for uniform data and markedly higher
for text, padding, single-byte-XOR'd data and repeating-key regions;
computed at several strides it is the Friedman/Kasiski test for a
repeating-XOR period.
* hash: add min-entropy rz-hash plugin
Adds min-entropy H_inf = -log2(max_i p_i) as an rz-hash plugin and the
rz_hash_min_entropy() API. This is the conservative worst-case entropy
used by NIST SP 800-90B: 8.0 for a uniform block, dropping as soon as a
single byte value dominates.
* hash: add serial-correlation rz-hash plugin
Adds the lag-1 serial correlation coefficient (with wrap-around, as in
the classic `ent` tool) as an rz-hash plugin and the
rz_hash_serial_correlation() API. Near 0 for compressed/encrypted data
but clearly non-zero for executable code, counters and gradients - the
order-aware axis that the histogram-only metrics cannot see.
---------
Co-authored-by: Anton Kochkov <anton.kochkov@gmail.com>
This rewrites the look of the interactive horizontal histogram in Rizin -
the output of `p==v`, `p==ev`, `p==mv`, `p==0v`, `p==fv`, `p==pv`, `p==zv`,
`p==Sv` and the analysis-based interactive variants. It closes#6365 and
#4431 in full, and now sits on top of the recently merged static-histogram
PR (#6427) so it reuses the new RzHistogramOptions struct and its helpers.
Five sub-tasks from #6365:
- Context-aware vertical ruler with up to 5 anchor labels (top, bottom,
quarter, half, three-quarter) honouring opts->value_min..value_max,
value_precision, value_scale and value_unit. Reuses the static
helpers compute_ruler_gutter, render_ruler_gutter, label_value_at and
select_ruler_label_rows so visual and static stay consistent.
- X-axis byte-offset ruler at the bottom: `^` ticks every (addr_w + 3)
cols followed by absolute byte offsets computed from opts->offpos +
realj * blocksize.
- Cursor offset + percentage shown right-aligned on the status line, with
"Index N data V" on the left.
- Top minimap: Unicode block characters (▁▂▃▄▅▆▇█) for the density of
each slice of the whole data, plus a ┏━━━┓ window indicator showing the
visible slice. Always rendered when the new `scr.hist.minimap` config
is true. When the terminal is wide (hist->w > 200) the minimap shrinks
by 43 columns to make room for a two-line `px 0x20`-style hex preview
panel on the right showing the 32 bytes at the cursor's file offset.
Each byte is wrapped in its px-style colour code (b0x00 / b0x7f /
b0xff / btext / other) when opts->color is set, matching rz_print's
rz_print_byte_color exactly. A trailing safety pad keeps the canvas
from clipping the last byte of the first hex row.
- Missing-half bug on `p==v` when the cursor is at offset 0 fixed by
clamping `adder` to [0, histogramwidth - span]. The old expression
`barnumber + 1 - histogramwidth/(zoom*2)` was always negative for the
default barnumber=0, causing the rendering loop to read `data[-N]`
(segfault on large files, missing left half on small ones).
Cursor visibility:
- The cursor column is drawn as a CONTINUOUS vertical line connecting two
plain markers at the top and bottom, ALWAYS exactly one character wide.
The line itself uses the dedicated `wordhl` palette colour (default
red background, configurable via `ec wordhl ...`), drawn on every
chart row so the cursor is always a full-height vertical strip. The
markers (`▼` at the top and `▲` at the bottom, or ASCII `v` / `^`
when scr.utf8=false) are intentionally left un-highlighted so they
read as a clean pair of arrows pointing at the cursor column.
- The cursor screen column is computed up-front (j_cursor) by inverting
the data-to-column map (rel * zoom * width / histogramwidth). Two
distinct widening bugs are avoided this way:
1. sizeofonebar > 1 (high zoom) - each data index spans several
screen columns; only j == j_cursor && kbar == 0 renders as the
cursor, the remaining kbar columns fall through to the gradient.
2. histogramwidth < width (chart much wider than data) - several
adjacent screen columns map to the same data index via integer
truncation; only the j_cursor column may render as the cursor.
Interactive keybindings & live config:
- The `:` hotkey drops into rz_core_visual_prompt_input, matching the
rest of Rizin's visual modes. Lets the user run arbitrary rizin
commands without leaving the histogram.
- The `?` help text now uses the same colour-coded format as the
visual / visual-bit-editor modes (rz_core_visual_append_help with
pal.args for keys and pal.help for descriptions), shown via
rz_cons_less_str.
- The config-driven opts (scr.hist.minimap, scr.hist.block, scr.utf8,
scr.color, hex.offset) are re-read on every redraw via
refresh_visual_opts_from_config, so `:` `e scr.hist.minimap=true`
<Enter> takes effect immediately without having to quit and re-enter.
The canvas's `color` field is refreshed alongside so `scr.color`
changes take effect on the same redraw.
Hex preview panel:
- When the terminal is wide (hist->w > 200) and the minimap is enabled,
the visual mode shows a two-line hex preview on the right of the
minimap rows: 32 bytes at the cursor's file offset, formatted as 8
pairs of 2 bytes separated by spaces (`abcd ef00 1234 5678 ...`),
matching `px 0x20` minus the header / offsets / ASCII column. The
bytes are fetched live via rz_io_read_at_mapped each redraw, so
moving the cursor (`h` / `l`) updates the preview.
- Each byte gets its px-style colour code: green for 0x00, red for
0xff, yellow for 0x7f, btext (white) for printable ASCII, "other"
(magenta) for non-printable. Mirrors rz_print_byte_color so the
histogram preview reads consistently with `px`.
- A trailing safety pad keeps the canvas from clipping the last hex
byte of the first row (a side effect of UTF-8 minimap glyphs
interacting with the canvas's width tracking when the row fills
the canvas exactly).
- Implemented via two new fields on RzHistogramInteractive
(`cursor_bytes`, `cursor_bytes_len`) that the caller fills in just
before the render call and clears right after. The minimap helper
grows two extra parameters that the visual function passes through;
when the panel is disabled (narrow terminal, no cursor_bytes, or
shrinking the minimap would leave it < 40 cols) the helper falls
back to the previous full-width minimap.
Closes#4431 in full:
- The negative-offset crash above is the immediate segfault from the
bug report.
- `print_histogram_bytes` now samples one byte per block instead of
reading nblocks contiguous bytes from core->offset. For an 8 GB file
shown across 80 bars the original code rendered the first 80 bytes
of the file; the new code samples at offsets brange->from + i *
blocksize so the chart represents the full span.
- The inner `int i` in the column-aggregation loop is renamed to `k`
to drop the shadow over the outer `size_t i`.
Refactor on the cmd_print.c side:
- New default_visual_opts(core, offset) returns an RZ_OWN
RzHistogramOptions* pre-populated for the visual commands (ruler=true,
minimap from scr.hist.minimap, offpos from caller, palette and
screen-mode toggles from config via refresh_visual_opts_from_config).
The nine print_visual_bytes call sites now build opts via this helper,
then pass it to print_visual_bytes which takes ownership. Entropy
sets value_max=8 / value_precision=1 / data_f=fdata so the visual
histogram shows the Shannon range matching the static side.
- `print_visual_bytes(core, opts, data, brange)` now propagates opts
cleanup along every error path; rz_histogram_interactive_new no
longer leaves a heap-allocated opts pointer dangling. The redraw loop
fetches 32 cursor bytes via rz_io_read_at_mapped, points
hist->cursor_bytes at a stack buffer for the call, then NULLs it
back so the next iteration's fetch is independent.
- RzHistogramInteractive gains `blocksize`, `cursor_bytes` and
`cursor_bytes_len` fields.
New config option:
- `scr.hist.minimap` (bool, default true) controls whether the top
minimap is shown for p==v / p==ev. Surfaces as `opts->minimap` and
is honoured by `rz_histogram_interactive_horizontal`. When true,
the minimap is ALWAYS rendered (provided there's room) - even when
the chart already shows the full data, in which case the window
indicator spans the whole map. Changes via `:` `e scr.hist.minimap=...`
<Enter> are picked up on the very next redraw.
Tests (33 total, 10 new for the visual side):
- test_histogram_interactive_horizontal_basic - smoke test with
barnumber=0 (pins the #4431 crash regression).
- test_histogram_interactive_horizontal_ruler_percent - fractional
labels with value_max=100 / value_scale=0.01 / unit="%".
- test_histogram_interactive_horizontal_ruler_default - the legacy
0..255 byte ruler.
- test_histogram_interactive_horizontal_no_negative_adder - covers
`p==v` at offset 0 on a small data set.
- test_histogram_interactive_horizontal_percent - status-line percent
indicator present.
- test_histogram_interactive_horizontal_cursor_markers - the ▼/▲
cursor markers (and ASCII v/^ fallback) appear on the cursor column,
left un-highlighted.
- test_histogram_interactive_horizontal_cursor_full_line - the cursor
bar is rendered on every chart row between the markers regardless
of the data threshold (continuous vertical line).
- test_histogram_interactive_horizontal_cursor_width - pins single-
char width across BOTH cursor-widening bugs: high zoom (sizeofonebar
> 1) AND chart wider than data (histogramwidth < width).
- test_histogram_interactive_horizontal_minimap_toggle - pins the
scr.hist.minimap gating across {zoomed, not zoomed} when
opts->minimap=true / =false.
- test_histogram_interactive_horizontal_hex_preview - 4 cases: wide
terminal + cursor_bytes shows the hex panel; narrow terminal
suppresses it; missing cursor_bytes suppresses it; colour mode
emits ANSI escape sequences for the bytes.
Both p== integration tests in test/db/cmd/cmd_print pass with their
regenerated EXPECT blocks (the per-block sampling change moves the
visible bars for small buffers).
Co-authored-by: Anton Kochkov <anton.kochkov@gmail.com>
Follow-up to the C bitfield support added in #6439 (#1240, #314): the PDB
type parser dropped struct/union bitfield members because an LF_BITFIELD
member type resolved to NULL in pdb_type_parse(), so "tc"/"ts"/"tp" on a
PDB-loaded type lost every bitfield field.
member_parse() now detects a TpiKind_BITFIELD field type, resolves the
member to the bitfield's underlying integer (base_type) and reports the
bit width (length) via an out-parameter. class_member_parse() and
union_member_parse() store it in RzTypeStructMember.size /
RzTypeUnionMember.size (the bitfield width in bits, 0 if not a bitfield),
matching the convention used by the C and DWARF member paths.
Update the PDB type expectations that previously asserted bitfield_typedef
had no members: db/cmd/cmd_idp (idpij) and db/tools/rz_bin (rz-bin -Pj)
now expect the three resolved "unsigned char" members of minimal.pdb.
Co-authored-by: Anton Kochkov <anton.kochkov@gmail.com>
