From OSINT to Detection: Building an Agentic CTI Pipeline

Detection teams face a constant flood of threat reports, and only a small share contain command lines, registry persistence, or other details worth writing rules for. Andrew Skatoff explains that the steps from finding a useful report to deploying a rule are slow and rely on expertise that lives in analysts' heads, and sets out to automate part of that workflow.

The talk walks through Huntable CTI Studio, an open-source prototype that ingests articles from configured sources and scores them with weighted keyword patterns. A random forest classifier then filters paragraph-sized chunks, and analysts can give it feedback and retrain it. Extraction agents pull artifacts such as command lines, process trees, registry keys, services and scheduled tasks, guided by tightly written prompts designed to fail closed rather than invent content. A generation step produces Sigma rules, validates them with pySigma, and checks novelty by comparing matched fields instead of using embeddings. Every run keeps a full trace, and an evaluation system scores prompt and model changes against gold-standard articles.

A human reviews and approves every rule before anything ships. The main takeaways are design choices rather than the tool itself: cheap deterministic filters ahead of costly LLM stages, narrow low-temperature extraction contracts, structured similarity checks, and auditable traces. The prototype is Windows-focused and not hardened for enterprise use, so it should not be pointed at sensitive intelligence on hostile networks.