RECON 2026 - Deobfuscation in the Age of Agentic Reverse Engineering

### Overview Presented at RECON 2026 by long-term reverse engineers Tim and Nicolo, this talk explores how agentic reverse engineering is reshaping the deobfuscation landscape. Building upon observations from a 2022 presentation where obfuscation was deemed far ahead of deobfuscation due to global transformations, lack of scalable tools, and reliance on custom tooling, the speakers demonstrate that autonomous agentic loops change the economics of software analysis. By pairing frontier LLMs (such as Claude and Codex) with programmatic access to analysis tools within a sandboxed Docker environment, agents act as tool orchestrators capable of iterating through complex reverse engineering workflows.

### Key Topics - **Agentic Reverse Engineering Architecture**: Structuring LLMs with tool access (Binary Ninja, Unicorn, Miasm, msynth, Z3) inside sandboxes, operating in an autonomous goal-driven loop with human-in-the-loop guidance. - **Virtual Machine (VM) Deobfuscation**: Scaling deobfuscation across simple custom VMs, commercial protectors (e.g., Themida, VMProtect), kernel-level anti-cheat engines, and game DRMs. - **Control Flow Flattening Decompilation Passes**: Developing automated Binary Ninja Medium Level Intermediate Representation (MLIL) passes to resolve state dispatchers and reconstruct clean control flow graphs in malware such as Emotet and FinFisher. - **Global Code Obfuscation & Call Graph Recovery**: Reconstructing indirect call tables protected by mixed boolean-arithmetic (MBA) expressions through constant propagation and SMT solvers. - **Anti-Agentic Obfuscation Dimensions**: Developing defensive counter-strategies aimed specifically at disrupting agentic workflows.

### Technical Details - **VM-Based Obfuscation Case Studies**: - *Simple Function-Level VMs*: Given an obfuscated function, the agent autonomously discovered the architecture, recovered handler semantics, authored a custom disassembler, and reconstructed high-level code within minutes ("oneshot"). - *Commercial Obfuscators*: Facing decentralized dispatching, handler duplication (10k–50k handlers), and rolling control-flow-sensitive bytecode encryption, unguided agents failed. When provided with a trace-based pipeline and templates, agents performed emulation (Unicorn) from entry to exit to establish ground truth, conducted symbolic execution (Miasm) with concrete inputs to match execution step-by-step, handled runtime errors/unmapped bytecode autonomously, and synthesized a VM disassembler in 1–2 hours (a task often taking weeks manually). - *Kernel-Level Anti-Cheat*: Identified a shared VM interpreter powering over 1,000 bytecode entries. The agent wrote a disassembler for the single VM instance and devirtualized entries in parallel along uncovered concrete execution paths. - *Game DRM*: Successful on titles from 5–7 years prior, recovering license crypto routines. However, modern titles resisted static agentic analysis due to runtime-dependent gates and anti-emulation loops targeting QEMU modeling discrepancies and multi-threaded register writes. - **Control Flow Flattening & In-the-Wild Malware**: - *Emotet*: The agent authored a Binary Ninja MLIL workflow that located the central dispatcher, enumerated states, identified successor basic blocks, and patched edges across multiple functions. - *FinFisher*: Contained mixed boolean-arithmetic (MBA) opaque predicates. While the agent rewrote the flattening workflow, opaque MBA predicates resisted the unassisted pass. - **Global Call Graph Obfuscation**: - In a DRM system utilizing MBA-indexed global tables for indirect calls, an unassisted agent wasted time on brute force and invalid crypto identification. Guided by analysts, the agent identified table arrays, propagated entries, solved MBAs using Z3 and msynth, and patched indirect calls into direct calls, discovering hidden function boundaries. - **Anti-Agentic Defensive Dimensions**: - *Budget & Search Traps*: Spaghetti code, state explosion, and computationally expensive oracle checks designed to exhaust token/time limits. - *Weaknesses of Classical Techniques*: Enforcing global semantics and runtime binding requiring analysis-time unavailable inputs. - *Tool Friction*: Anti-debugging and anti-emulation (e.g., exploiting hardware modeling discrepancies in QEMU). - *Poisoning the Narrative*: Planted strings, prompt injections, and red herrings.

### Takeaways Agents act primarily as powerful tool orchestrators rather than fully autonomous binary understanders. While commercial-grade protections still require human reverse engineers to provide procedural workflows and domain guidance, agentic automation drastically lowers the cost and turnaround time of iterative deobfuscation from weeks to hours. In response, modern obfuscation designs must evolve toward anti-agentic principles that exploit tool friction, runtime dependencies, and search space limits.