Rewrites the C55x and C55x+ analysis classifiers as pure byte-level
dispatch -- no mnemonic-string matching, no round-trip through the
disassembler -- and adds the supporting infrastructure they need to
produce useful RzAnalysisOp metadata.
What lands
==========
* librz/arch/isa/tms320/c55x/c55x_analysis.{c,h} -- C55x baseline
classifier, ~360 lines, 256-entry size table extracted from the
decoder's table.h.
* librz/arch/isa/tms320/c55x_plus/c55plus_analysis.c -- C55x+
classifier rewritten in the same shape, ~470 lines covering 90+
opcodes with byte-level disambiguation for 0x02 / 0x03 / 0x74 /
0x76 / 0x7B / 0xC5.
* librz/arch/isa/tms320/tms320_dwarf_regnum_table.h plus a hook in
librz/arch/dwarf_process.c -- TI cgt55 ABI DWARF register-number
mapping, so the cl55 compiler's .debug_info variable locations
resolve into rizin register names instead of returning the dummy
"?" placeholder.
* librz/arch/p/analysis/analysis_tms320.c -- thin dispatcher that
picks the per-cpu classifier and stops carrying the tms320_dasm_t
engine in analysis state.
Why a byte-driven classifier
============================
The old classifier round-tripped through the disassembler and did
strncasecmp() on the mnemonic string. Three problems:
1. It kept a tms320_dasm_t engine alive in the analysis context
just to read its 'syntax' buffer after every classify call.
Removing it shrinks the per-analysis state and removes a
tms320_dasm_init/_fini pair from the analysis_init/_fini path.
2. It only set op->type -- never op->jump, op->fail, op->stackop,
op->stackptr, op->val, op->eob. Basic-block formation followed
only the most obvious control flow, and call/ret/push/pop
semantics were invisible to higher-level analysis.
3. It couldn't disambiguate predicated versus unconditional calls:
the disassembler emits 'callcc' vs 'call', but the substring
match missed the conditional fail-path for CALLCC.
The new classifiers fix all three:
- Read the leading byte (and second-byte refinements where the
encoding family is shared) directly from buf.
- Resolve jump and call targets from BE-stored displacement and
absolute fields, with correct sign extension for the 8-bit and
16-bit relative forms.
- Read 24-bit absolute targets via rz_read_at_be24().
- Set op->fail = addr + size for every conditional jump/call,
op->eob = true for unconditional branches and RET so basic-block
walkers terminate correctly.
- Track the stack: PSH/POP per ISA cluster, CALL/CALLCC +2,
RET/RETI -2.
- Capture INTR/TRAP immediates in op->val via set_imm().
- Disambiguate sub-opcodes that share a leading byte by reading
the relevant bits of the second byte. For C55x, the most
notable case is 0x48 (RPT/RPTADD/RPTSUB/RET/RETI) which uses
bits 0-2 of byte 1; for C55x+ the disambiguations are 0x02,
0x03, 0x74, 0x76, 0x7B and 0xC5.
- Handle parallel-prefix bytes (odd-valued leading bytes below
0x80 in C55x like 0x03, 0x05, 0x07, 0x11, ...) by treating
them as a 1-byte prefix and dispatching on byte 1 so paired
'|| retcc', '|| bcc', etc. classify correctly.
Both classifiers ship analyzer helpers (set_cjmp, set_call, set_jmp,
set_ret, set_cret, set_push, set_pop, set_imm, set_mem_width,
set_dst_reg, set_ireg, set_dir, set_disp) so each opcode entry fills
the RzAnalysisOp ptr / val / stackop / stackptr / fail / eob fields
uniformly across both architectures.
DWARF register mapping
======================
Loading any cl55-compiled TI COFF v2 with debug info (every
emulateme*.ticoff2.dbg.coff in rizin-testbins) used to fire:
ERROR: No DWARF register mapping function defined for tms320 32 bits
per variable, because dwarf_process.c had no entry for arch=tms320.
The new tms320_dwarf_regnum_table.h covers the cgt55 ABI numbering:
AC0-AC3, T0-T3, AR0-AR7, SP/SSP/CDP, BK03/BK47/BKC, DP/PDP, CSR,
BRC0/BRC1, TRN0/TRN1, RPTC, IER0/IER1, IFR0/IFR1, DBIER0/DBIER1,
IVPD/IVPH, ST0_55..ST3_55 (42 entries). Reach into the table is
guarded; out-of-range numbers fall back to NULL so the caller
surfaces the dummy "?" instead of confidently picking the wrong
register.
Wrigley3G coverage
==================
Validation against a 3.1 MB Wrigley3G baseband firmware (Motorola
Droid A855, MSG39UPEU_A1.19_1.80, partition CG45.img) found 31
leading-byte values producing real instructions classified as NULL.
The c55x+ classifier here covers those:
0x50-0x5F MOV memory/register cluster
0x88, 0x8A MOV ACx <-> mem high/low halves
0x8C ADD with carry, mem -> ACx
0x97 Dual-memory MOV (parallel)
0xA0 MOV with parallel dual addressing
0xAC, 0xAD MOV #k16, ACx (long immediate)
0xB4, 0xB5 MOV with rounding and shift
0xB6, 0xB7 ADD with shift (T-register or immediate)
0xC0, 0xC2, 0xC4 ADD #k16 with shift slots
0xCC Packed ADD :: MOV dual-instruction encoding
0xD0 MOV ACx, dbl(*(#abs24))
0x2E, 0x2F XCCPART predicated execute
0x0B, 0x23 Wrigley silicon pseudo-ops (TRAP)
0xC6 BFXTR / BFXPA bit-field extract (MOV)
The 0x03 family classifier extends from a 4-bit (0xF0) to a 6-bit
(0xC0) mask so the full encoded range resolves:
0x03 0x00-0x3F INTR #k5
0x03 0x40-0x7F TRAP #k5
0x03 0x80-0xBF SWAP register pairs
0x03 0xC0-0xFF SIM_TRIG (Wrigley-specific simulator trigger)
Coverage on Wrigley3G rises from 94.4% to 97.4% (2000-sample
random survey).
Tests
=====
Two new test suites land alongside the classifiers:
test/db/analysis/tms320.c55x_32 11 tests (batched)
test/db/analysis/tms320.c55x+_32 13 tests (batched + binary
fixtures)
Tests are intentionally batched -- each test bundles 10-12 opcode
checks behind one rizin process spawn instead of one per check.
That brings both suites down to under 0.5 seconds combined.
The c55x+ suite includes six binary-fixture tests against the
companion rizin-testbins drop-in tms320/coff2/*.obj corpus,
covering function discovery (afl), stack-pointer tracking
(afvs / afS), data-section walk (iS), and globals enumeration
(is). The c55x suite covers tms320/emulateme_nostd.ccsv5.c55x
.ticoff2.dbg.coff from the existing rizin-testbins tree.
Cross-reference
===============
TI SPRU374 'TMS320C55x DSP Mnemonic Instruction Set Reference
Guide' (publicly available) -- C55x baseline.
TI SWPU086 'TMS320C55x+ DSP Algebraic Instruction Set Reference
Guide' (May 2005) -- C55x+ instruction encodings.
TI SWPU104 'TMS320C55x+ DSP Mnemonic Instruction Set Reference
Guide' (December 2006) -- C55x+ mnemonic forms.
The c55x_plus disassembler glue layer (c55plus.c) is rewritten as a
thin shim over the th0rpe c55plus_decode() walker:
- Drop the ad-hoc ctype tolower() loop; use rz_str_case() to
lower-case the walker's mixed-case mnemonics in one call.
- Reorder the includes; drop the unused ones.
- Scope local variables to where they are actually used.
- Move the global setup (ins_buff / ins_buff_len) into the same
block as the c55plus_decode() call so the dataflow is obvious.
While here, fix a pre-existing off-by-one in utils.c. The hex-digit
lookup table was declared as a 17-character string with a duplicated
leading '0':
static char hex_str[] = "01234567890abcdef";
This shifted every nibble >= 0xA by one position in the table:
hex_str[10] = '0' (should be 'a')
hex_str[11] = 'a' (should be 'b')
...
hex_str[15] = 'e' (should be 'f')
hex_str[16] = 'f' (unreachable)
Result: get_hex_str(0xff) returned "ee", get_hex_str(0xab) returned
"0a", and every disassembled '.byte 0xNN' for an unknown opcode with
nibbles >= A came out wrong. The function is used from
c55plus_decode.c on the unknown-opcode fallback path
(hash_code == 0x223), so the bug surfaces whenever the decoder bails
out and emits a raw byte.
Fix: use the correct 16-character table "0123456789abcdef" and rename
the static to hex_digits to make the role obvious. While here, tidy
strcat_dup() to use bitwise tests on the n_free bitmask so the
'3 = free both' contract is enforced uniformly, add docstrings to both
helpers, and drop the redundant memcpy length guard that was a no-op
for non-NULL length-zero strings.
No behavioural change for either piece beyond the bug fix above.
The c55x_plus decoder shipped with two private string helpers in
utils.c:
strcat_dup(s1, s2, n_free) - allocate s1+s2 and optionally free
inputs, with a bitmask controlling
which of s1/s2 are released
get_hex_str(n) - format the low 8 bits of n as a
two-character lowercase hex string
Both have direct equivalents in rz_util:
strcat_dup(s, lit, 1) -> rz_str_append(s, lit)
strcat_dup(lit, s, 2) -> rz_str_prepend(s, lit)
strcat_dup(s1, s2, 3) -> rz_str_append_owned(s1, s2)
strcat_dup(s1, s2, 1) where
s2 is also owned and freed
manually right afterwards -> rz_str_append_owned(s1, s2)
get_hex_str(n) -> rz_str_newf("%02x", n & 0xff)
This commit converts all 56 strcat_dup call sites in
c55plus_decode.c and decode_funcs.c plus the single get_hex_str
site, then deletes utils.c and utils.h entirely.
While here, replace several local sprintf-into-stack-buffer +
rz_str_dup patterns with direct rz_str_newf calls:
- get_AR_regs_class1: was malloc(50) + sprintf per case, now a
single rz_str_newf per case returning the result directly.
The function is reduced from 34 lines to 14.
- get_AR_regs_class2: same pattern, reduced from 130 lines to
79 with no allocation needed at the top.
- get_token_decoded case 40/48, 70/72/80, 41/73: sprintf into
a 512-byte stack buffer then rz_str_dup -> single rz_str_newf.
- decode_funcs.c case 2 of get_status_regs_and_bits: was
calloc(50) + sprintf, now rz_str_newf.
The 512-byte stack buffer 'buff_aux' in get_token_decoded becomes
unused and is removed.
C55PLUS_DEBUG, the only useful symbol that used to live in utils.h,
moves to ins.h (which all c55x_plus translation units transitively
include). ins.h gains a direct <rz_util.h> include so the rest of
the headers don't need to pull it in indirectly.
No behavioural change. Full c55x_plus test regression passes:
asm/tms320_c55x+_32 104/104, analysis/tms320.c55x+_32 45/45,
parity against TI dis55.exe v4.3.6 on the 13-source testbins corpus
remains 140/140.
COFF C_EXT (external/global) symbols carry their function-or-not
status in two places: the DTYPE field of n_type (the ISFCN bit), and
implicitly by the section they live in. The existing code only
honoured the DTYPE bit:
ptr->type = DTYPE_IS_FUNCTION(s->n_type) || !strcmp(name, "main")
? RZ_BIN_TYPE_FUNC_STR : RZ_BIN_TYPE_UNKNOWN_STR;
C and C++ compilers set the ISFCN bit when emitting object code, so
this works for compiled output. But assembler-emitted globals -- TI's
asm55p / cl55, the GNU GAS COFF backend on legacy targets, and any
hand-written .s -- often leave n_type at zero. Every assembly symbol
then comes back as RZ_BIN_TYPE_UNKNOWN_STR, and 'aaa' has no way to
distinguish a function from a data label. Concrete example: with TI
asm55p output for the C55x+ test corpus, none of the eight .global
labels in 03_branches_calls.s (_short_ret, _short_branch, _short_call,
...) were promoted to functions, so analysis only found those
reachable by following control flow from a hard-coded entry.
Add a section-flag fallback: if the symbol's containing section has
COFF_SCN_CNT_CODE set (i.e. it's a .text / code section), the symbol
is a function. This keeps the DTYPE check as the primary signal but
fills the gap on assembler output.
Verified with TI C55x+ .obj files: all .global labels now appear as
RZ_BIN_TYPE_FUNC_STR and are picked up by 'aaa'. No regression on
the standard COFF test suite (Windows i386/amd64 .obj from compiled
C output).
The COFF section table walker assumed every COFF section header is
40 bytes -- the standard COFF1 form. The TI Common Object File Format
(SPRAAO8) extends this for TI COFF v2 (file magic 0x00C2) used by the
TI tools (asm55, cl55, cl6x, cl28, etc.):
------------------------------------------------------------------
offset size field standard COFF1 TI COFF v2
------------------------------------------------------------------
0x00 8 s_name char[8] char[8]
0x08 4 s_paddr ut32 ut32
0x0c 4 s_vaddr ut32 ut32
0x10 4 s_size ut32 ut32
0x14 4 s_scnptr ut32 ut32
0x18 4 s_relptr ut32 ut32
0x1c 4 s_lnnoptr ut32 ut32
0x20 2/4 s_nreloc ut16 ut32 <-- widened
0x22 2/4 s_nlnno ut16 ut32 <-- widened
0x24 4 s_flags ut32 ut32
0x28 - reserved - ut16 <-- new
0x2a - mempage - ut16 <-- new
------------------------------------------------------------------
TOTAL 40 48
Reading TI COFF v2 with the standard 40-byte stride walks the section
table off-by-8 per section, producing nonsense values for every
section after the first. In practice, vaddr and size for the second
section spill into the next section's name field, surfacing as
attention-grabbing decimal values like 0x7461642e (ASCII '.dat'
reversed -- bytes from the upcoming '.data' name being interpreted as
a size).
Concrete reproducer with TI asm55p output:
$ wine asm55p.exe -v5505 03_branches_calls.s
$ rz-bin -S 03_branches_calls.obj # before this fix
...
0x00000061 0x7461642e 0x00000040 0x7461642e ... # garbage size
...
$ rz-bin -S 03_branches_calls.obj # after this fix
0x000000fa 0x34 0x00000030 0x34 -r-x .text
...
Fix: add a parallel coff_init_scn_hdr_ti() that reads the 48-byte
layout, with nreloc/nlnno as ut32 (clamped to ut16 because the rest
of the COFF code keeps them as 16-bit and counts above 64K are not
encountered in practice). bin_coff_init_scn_hdr() picks the variant
based on coff_is_ti_machine() -- the same predicate already used in
the file-header parser to consume TI's f_target_id field.
Tested with TI asm55p output for C55x, C55x+, and C5500 (machine ids
0x9c, 0xa1, and TI_1/TI_2 file magics) -- sections now parse with the
correct sizes, vaddrs, and flags, and the resulting binaries open
cleanly under 'rizin -A' for analysis.
The c64x branch of tms320_reg_profile() declared '=PC pc' but only
ever registered pce1 -- pc was never one of c64x's declared registers.
This raised:
WARNING: Invalid alias given in register profile: pc.
on every rz-asm / rizin invocation of any tms320 cpu, including c55x
and c55x+, because the warning fires during analysis init before the
cpu-specific branch of the profile is selected.
The C64x ISA uses pce1 (Program Counter Extension 1, .32 at index
545) as its program counter -- the only PC-named control register
declared in the c64x profile -- so '=PC pce1' is the correct alias.
The companion test/db/analysis/tms320.c64x_32 'arp' expectation is
updated to match.
The c55x / c55x+ branch (is_c5000) is unchanged -- those profiles do
declare 'pc' as a 24-bit program counter, so '=PC pc' stays valid
there.
This change is independent of the c55x_plus work in the rest of this
series; it would be a useful cleanup on its own. Included here because
it surfaces immediately whenever any of the new c55x+ analysis tests
are run.
Detected with valgrind, some element assignments could memcpy with
identical addresses. This is usually a no-op in practice, but
theoretically undefined behavior.
* Add option to replace (geometric) invalid values with another value.
* Add geometric Mean and Std Dev to benchmarks.
* Add Doxygen documentation
* Add a README to the bench dir as intro.
empty_cond was never signalled and the termination depended only on
the timeout in rz_th_queue_close_when_empty() causing a re-check of
emptiness.
However, the rz_th_cond_timed_wait() implementation, which was used
there, was flawed because it expected a relative timeout but passed that
directly to pthread_cond_timedwait() which expected an absolute time
value, practically causing it to time out immediately. Depending on the
pthread_cond implementation, this possibly created a situation where the
mutex could never be acquired by another thread, effectively causing a
deadlock. This behavior was observed on Mac OS X 10.5 (ppc) when running
the test_core_bin test.
We solve this by not using a timeout at all and signalling the condition
variable for all waiting threads at the appropriate time.
* Add Welfords square of sums algorithm for variance and std deviation calculations.
* Add standard deviation to benchmarks
* Simplify Welford
* Add geometric mean and standard deviation to Welford Sums
* Add order ignoring remove_at version with better performance.
* Optimize rz_vector_swap by using stack memory for small elements.
* Add benchmark for rz_vector_swap
* Refactor del_edges to use RzGraphStatus.
* Refactor del_edge() to use RzGraphStatus.
* Refactor update_edge() to use RzGraphStatus.
* Refactor has_edge() to use RzGraphStatus.
* Refactor add_edge() to use RzGraphStatus.
* Fix leak of b
* Fix leak of xref list
* Fix leaks of analysis ops
* Address review comments
* Inlcude clean up
* Fix invalid free
* Add tests with node and edge data.
* Extend tests
Various issues related to pointer size and RzVector behavior.
Testing rz_pvector_shrink() at the place removed here is not necessary
as it has its own dedicated tests.
* GRAPH: Add rz_graph_add_get_node() and refine API.
* GRAPH: Remove unused, and rename parameters.
* Improve doxygen.
* Implement rz_graph_del_edges()
* Rename rz_graph_node_get_hash_id() to rz_graph_node_get_hash_id() to make clear what identifier is returned.
* Simplify node identification.
Removes the option to have two sources of identifiers (node data or other identifier data).
Changes the API to use the hash id instead of a pointer to data.
* Fix type annotations.
* Remove duplicate function.
* Fix invalid asserts.
* Set flag if node was present.
* Grow by a factor of 1.25. Exponential growth quickly leads to OOM.
* Remove the edge index again to not remove reduce the main advantage of an adjacency matrix
* Refactor list based graph to use vectors instead of hash maps for edges.
* Fix heap-use-after-free
* Add an rz_graph_update_edge function.
* Add function to print graph as dot graph.
* Add warning about del_edges runtime.
* Refactor add_node to use RzGraphStatus.
* Refactor del_node to use RzGraphStatus.
* Use cast-macro to prevent ASAN issues.
* Add support to various MIPS reloc conversion.
R_MIPS_26, R_MIPS_HI16, R_MIPS_LO16, R_MIPS_GOT16, R_MIPS_PC16,
R_MIPS_CALL16, R_MIPS_64, R_MIPS_GOT_HI16, R_MIPS_GOT_LO16,
R_MIPS_CALL_HI16, R_MIPS_CALL_LO16, R_MIPS_REL16
* Fix test missing RUN at the end.
init and fini are called on regular rizin start, even if not debugging.
These errors were distracting and not very meaningful there. They do
however make sense when executing any actual debug operation.
