Overcoming Barriers in Security DSLs with BabbelPhish

Bobby Filar, head of data science at the email security company Sublime Security, opens with the onboarding problem. Detection-as-code platforms rely on query languages with unfamiliar syntax, and new detection engineers often spend weeks learning several of them. He argues that experienced analysts already reason through a checklist of questions about a suspicious email, and each answer maps to a piece of detection logic that could be written as a query.

He then presents BabbelPhish, an open-source project that uses a fine-tuned GPT-3 model to convert natural language requests into MQL. The team built a training set of about 3,000 examples by mining its documentation, public rule repositories, a community Slack channel, and annotations supplied by detection engineers, then published the set on Hugging Face. The model was chosen for low cost, fast inference, and reproducibility. Benchmarks reported a correct answer in roughly 98 percent of cases within three attempts, and generated queries were checked with the MQL interpreter for syntax. A VS Code extension offers completion-style suggestions, and a recorded demo showed English questions turning into queries.

Filar closes with limitations and next steps. Planned user studies will compare the tool against built-in autocomplete on task completion and context switching. Future work includes tighter tokenization through the product's own interpreter, fill-in-the-middle prompting, vector stores of common snippets, and explanations of existing rules. He stresses that natural language should be an optional entry point to a query language rather than something forced on experienced users.