Automating Dark Web CTI Reports With RAG Insight For MISP Sharing

The talk starts from the problem that personal data sold on dark web forums gives criminals material for targeted fraud, and that the gap between a sale and victim awareness is too long. The speaker explains how crawlers reach Tor hidden sites through a SOCKS proxy, notes the operational security risks, and outlines a three-stage pipeline: collecting forum posts, classifying them, and feeding the results into a retrieval-augmented generation system.

The collection targets a marketplace-style forum where leaked databases, stealer logs and account lists are traded. Roughly 10,900 posts across several boards were labelled into five categories, including combo lists, cracked accounts, stealer logs, databases and other. A BERT encoder with a softmax head reached about 94 percent accuracy, and the confusion matrix showed that most errors fall in the rarer classes. A reference table keeps posts that mention Taiwan-related keywords along with their predicted labels, and retrieval uses that table to answer analyst questions. The speaker also compared answers with and without retrieval, using an LLM as a judge, and reported better average results with retrieval.

The later part covers running a small local model, Llama 3.2 3B, through Ollama. The speaker argues that both the embedding and inference models should run locally when the data is sensitive, while noting that the smaller model is slower but workable on a laptop. The final step uses PyMISP to convert generated findings into objects, attributes and tags, then uploads them as a MISP event. The takeaway is that this kind of pipeline can shorten the time from leak to shared report, with privacy and evaluation as the main trade-offs to weigh.