Clue-Driven Reverse Engineering by LLM in Real-World Malware Analysis

Overview: Teni and Wei-J from CyCraft address the critical challenge of hallucinations and cascading error propagation when utilizing Large Language Models (LLMs) for automated malware reverse engineering. The speakers demonstrate how ungrounded token predictions in stripped, decompiled binaries contaminate downstream analysis across entire call graphs. The presentation introduces "Celebi," a clue-driven reverse engineering framework that inspects internal transformer attention heads and softmax token probability distributions to detect when an LLM is guessing, significantly improving symbol recovery and binary comprehension.

Key Topics: - LLM hallucination and the "single-source trust" dilemma in malware triage - Error cascade and snowballing effects in decompiled binary call graphs - Internal transformer interpretability: identifying "clue-focus attention heads" - Softmax probability distribution analysis as an LLM "lie detector" - The Celebi automated reverse engineering framework and APT41 malware case study

Technical Details: Decompiled C code generated by tools like IDA Pro is stripped of variable names, type structures, and function symbols, forcing analysts to infer intent from low-level operations. While state-of-the-art LLMs can generate plausible names for stripped symbols, they suffer from "single-source trust": models prioritize generating syntactically fluent answers even when contextual evidence is absent. The speakers illustrated how a localized misnaming—such as an LLM guessing that a boolean check `v3` represents an `area` calculation—cascades into misidentified function signatures, corrupts caller functions, and creates a snowballing misunderstanding across the binary. Standard mitigation strategies like prompt engineering and RAG fail to prevent this because they do not verify whether generated tokens are grounded in actual binary evidence.

To solve this, the authors drew inspiration from investigative interrogation methods combining "reference checks" and "lie detectors." Translating this to transformer architectures, the researchers analyzed multi-head self-attention mechanisms to discover specialized "clue-focus attention heads." Unlike syntactic or grammatical heads, clue-focus heads actively attend to high-entropy binary clues (such as Windows API invocations, unique crypto constants, and string literals) when predicting identifier tokens. Ablation experiments confirmed that pruning clue-focus heads caused a drastic drop in naming accuracy, proving their causal role. To implement the "lie detector," the framework inspects token probability distributions at the final softmax layer: sharp, unimodal distributions indicate high confidence, while broad, high-entropy distributions indicate guessing, triggering immediate proposal rejection.

These mechanisms were integrated into an end-to-end framework named "Celebi" (inspired by the time-traveling Pokémon). Celebi ingests decompiled code from IDA Pro, extracts internal clues (strings, API calls) and external clues (resolved via static emulation using tools like Speakeasy to unpack dynamic Windows API hashes and locate crypto constants like AES S-boxes), prioritizes functions with high clue density, prompts the LLM for symbol renaming, filters suggestions through attention and softmax validation, and recursively feeds validated symbols back as new clues. The authors demonstrated Celebi against a real-world APT41 malware sample containing over 800 stripped functions and dynamically resolved API calls, successfully unpacking EDR process injection routines without human-assisted verification.

Key Takeaways: Automated binary reverse engineering workflows must avoid single-source trust in LLMs by validating generated symbols against verifiable binary artifacts before updating call graphs. Inspecting transformer internal interpretability signals—specifically clue-focus attention heads and softmax output entropy—provides a reliable mechanism to detect and suppress hallucinations. Malware analysis automation pipelines must incorporate static binary emulation to resolve obfuscated API hashes and crypto primitives, providing the rich contextual clues required for grounded LLM reasoning.

(AI summary — Gemini Flash)