Enhancing Command Line Classification with Benign Anomalous Data

## Overview

In the presentation *"Enhancing Command Line Classification with Benign Anomalous Data,"* Sophos data scientists Ben Gelman and Sean Bergeron propose a method to break the stagnation of machine learning models used for detecting malicious command lines. Relying on human analysts to label incoming threats is unscalable and exhausting.

To create an automated, persistent labeling pipeline, the researchers initially attempted to run unsupervised anomaly detection to identify candidate malicious commands and verify them using a large language model (LLM). However, when evaluated against expert labels, the LLM achieved an unusable 36% precision on malicious classifications due to mistaking benign atypical commands for malicious behavior. In contrast, the LLM was virtually perfect when identifying anomalous commands as **benign**. Rather than discarding the system, they repurposed it to source rare, high-confidence benign samples from the distribution's "long tail" to enhance supervised classifiers.

---

## The Core Problem: Alert Fatigue & The Benign Long Tail

The speakers outline fundamental trade-offs between unsupervised anomaly detection and supervised learning:

* **Anomaly Detection Challenges:** Unsupervised methods scale cost-effectively across CPUs and distributed environments without training labels. However, they flag anything deviating from the distribution rather than determining malicious intent. Even with context (parent/child graphs or layered local/global context) reducing false positive rates (FPR) to 2–3%, a 2% FPR on a typical daily volume of 100 million commands yields 2 million false alarms, overwhelming SOC analysts. * **Supervised Learning & The Benign Long Tail:** Supervised models control false positives but require curated datasets. While malicious data can be aggregated from sandboxes, customer investigations, and VirusTotal, benign training sets typically rely on approved, non-malicious enterprise hosts. This restricts benign training data to frequent, simple commands (e.g., directory changes). Supervised models fail to generalize to the "long tail" of rare benign actions (e.g., complex sysadmin commands), causing false alarms in production.

---

## System Architecture & End-to-End Pipeline

The proposed architecture surfaces diverse anomalies, filters them through a small reasoning LLM, and injects verified benign samples back into the primary classifier. The pipeline operates across two scales:

1. **Data Ingestion Scales:** * **Full Scale:** Processes the entire stream of roughly 50 million command lines per day via distributed computing. * **Reduced Scale:** Samples 4 million commands daily, enabling processing on cheap AWS CPU/GPU instances. 2. **Feature Extraction:** * **Full Scale:** Utilizes hand-engineered expert features designed to detect hallmarks of attacks (e.g., case obfuscation such as alternating cases in `powershell.exe`) and standard NLP features (character statistics, counts, and entropy). * **Reduced Scale:** Uses a transformer encoder—specifically `jina-embeddings-v2-base-code` (pretrained on code including PowerShell)—to convert command lines into semantic latent vector representations. 3. **Multi-Algorithm Anomaly Detection:** * **Full Scale:** Uses **Isolation Forest**, which isolates outliers by measuring the average number of random splits required to separate a data point. * **Reduced Scale:** Employs modified implementations of **K-Means** and **PCA** alongside Isolation Forest to capture different structural anomalies and maximize data diversity. 4. **LLM Labeling:** * The anomalous subsets are piped to **OpenAI o3-mini**, a small reasoning LLM. Because anomaly detection filters down the input volume, running the LLM remains fast and low-cost. The LLM labels candidate anomalies, specifically isolating high-confidence benign samples. 5. **Target Classifier Retraining:** * The newly confirmed benign anomalies from the long tail are appended to the training corpus of an **XGBoost** binary classifier.

---

## Experimental Baselines & Takeaways

The authors evaluate their approach on two operational baselines: * **Regex Baseline (RB):** A starting-point pipeline using static regex rules for rudimentary labeling. * **Aggregated Baseline (AB):** A mature labeling setup incorporating regexes, sandboxes, customer telemetry, and active investigations.

The methodology demonstrated that anomaly detection does not need to serve as an imperfect front-line alert mechanism; instead, it can be paired with LLMs to continuously capture complex benign behavior, plugging critical visibility gaps in supervised security models.