Beyond Manual Hunting: OSINT Copilot with LLMs & Evidence-Based Collection, Enrichment ...

### Overview Presented by Victor and Matos, this talk examines the intersection of cybersecurity threat research and data engineering. The speakers address the primary bottleneck facing threat analysts: sifting through massive volumes of raw, noisy data from forums, marketplaces, and other sources to produce actionable intelligence for decision-makers. Rather than relying solely on manual hunting or external proprietary LLMs, the speakers propose an end-to-end pipeline that combines distributed data collection, deterministic entity extraction, local small language model (SLM) semantic enrichment, vector search, and agent-driven contextual investigation.

### Key Topics - **Hunting Challenges & Pipeline Philosophy**: Threat research requires moving beyond raw data collection to actionable intelligence. The speakers emphasize that "data engineering comes first" and "the pipeline is the product"—language models alone cannot fix unstructured or unorganized data. - **Data Collection Trade-Offs**: Gathering data from diverse sources involves choosing between lightweight HTTP clients (scalable and low resource, but unable to execute JavaScript or manage browser footprints) and browser automation (handles JavaScript, bypasses barriers, and manages device fingerprints, but incurs heavy resource and complexity overhead). - **Dual-Layer Enrichment**: Processing unstructured post text requires both deterministic extraction (regex/pattern parsing for IPs, emails, domains, URLs, and cryptocurrency wallets) and semantic analysis via language models (identifying intent, company mentions, communication context, and risk/revenue tags). - **Local/Small Language Models vs. Commercial APIs**: Processing millions of events and billions of tokens daily makes external APIs (such as Entropic or Openi) cost-prohibitive. The team deploys self-hosted smaller models using optimization tools such as VLM (for GPU KV cache management) and MTP (multi-token prediction) to guarantee privacy, predictable costs, and steady output.

### Technical Details - **Extraction & Normalization**: Data pipelines handle session management, fingerprint rotation, and deduplication. Collected posts are parsed into metadata fields (forum name, topic ID, post ID, creation date, collection date, language, raw text). - **Embedding & Search Architecture**: Enrichment generates event summaries, risk indicators, and vector embeddings. Embeddings are stored in a dedicated index separate from original documents to prevent data loss. Documents are indexed in a distributed search engine (referenced as Galaxy/logical search engine) supporting both BM25 keyword matching and vector similarity. - **Contextual Graph & Agentic Investigation**: When an analyst queries an entity (e.g., an actor name like Gordon Freeman), hundreds or thousands of matching events may appear. To make this manageable, an AI agent equipped with database access tools navigates the schema, groups relevant fields, and constructs connection graphs across multiple sources. In an illustrated case study, this approach linked actors ("Cipher 403", "Home Home Hoot One") and a threat group ("F") targeting South American countries (Venezuela, Argentina, Guatemala, Chile, Ecuador) and Canada. - **Benchmarking**: Tests across more than 10,000 events demonstrated that smaller local models performed comparably to large frontier models for risk indexing and entity dimensions, with only slight differences observed in text summarization.

### Takeaways - Language models cannot compensate for poor data engineering; structured collection, parsing, and normalization must precede AI integration. - Self-hosted smaller models running in-cluster provide superior data privacy, latency control, and cost predictability for high-volume threat data processing. - Combining deterministic indicators with semantic risk tags allows automated filtering, enabling analysts and investigative agents to focus exclusively on high-priority events and relationship mapping.