A Little Less Malware, A Little More Context: Using LLMs to Detect Malicious macOS Activity

The speakers, both from OpenAI's threat intelligence and detection teams, start from the idea that process command lines are language a model can read. They built a macOS lab that logged endpoint security events into an Elastic stack, then sent process groups to a model through an API and requested a structured verdict, a suspicion score, ATT&CK tags and a short analysis. Test families included info stealers, a loader, a remote access tool driven by a Caldera agent, admin scripts that mimicked insider activity, and ransomware. Most malicious cases scored high, while routine administrator work stayed benign. Ransomware was missed because it spawned no extra processes and left little command-line context.

The middle of the talk focused on context and prompting. Analyzing commands one at a time produced far more false positives than grouping them by process group, since the model could not judge whether an action was expected. Adding the user's role to the grouped data cut false positives further and helped the model judge intent. Sequences also exposed masquerading and encode-decode chaining that single commands hid. A detailed system prompt, which breaks the task into tagging, scoring and explanation steps, gave much better precision than a short prompt at similar recall.

The takeaways were that LLM detections should augment EDR rather than replace it, and that one prompt can cover several attack types without depending on exact syntax the way signatures do. Prompt edits change output, so versioning and evaluation matter, and structured outputs make it easier to feed results into SIEM or SOAR tooling. Future ideas included larger context windows, retrieval from internal repositories, fine-tuning, and applying the approach to application logs and network data.