08. Toppling The Stack: Outlier Detection For Threat Hunters by David J. Bianco

### Overview In this presentation, David J. Bianco discusses data analysis methods for threat hunters aiming to detect malicious behavior in large data sets. While reviewing recipes on his website, threat hunting.net, Bianco observed that threat hunters overwhelmingly rely on a single anomaly detection technique: least frequency of occurrence (LFO) analysis, also known as stat counting or stacking. Bianco argues that relying solely on this method is limiting and presents three additional outlier detection techniques—scatter plots, box-and-whisker plots, and isolation forests—to help hunters analyze numerical and multidimensional data more effectively.

### Key Topics - **Limitations of Stacking (Least Frequency of Occurrence):** While simple and well-supported across SQL, Splunk (stats module), and Python (pandas), LFO is primarily suited for categorical or enumerated data with few fields per record. Its core assumption—that rarity implies suspiciousness—often produces a long tail of benign results, lacks effectiveness for continuous numerical data, and makes automated threshold alerting difficult. - **Visualizing Correlations with Scatter Plots:** Visual methods leverage the human brain's pattern recognition capabilities. Scatter plots allow hunters to compare two numeric dimensions against an expected baseline or correlation line, such as analyzing changes in network traffic patterns. - **Understanding Univariate Distributions with Box-and-Whisker Plots:** Box plots present a five-number summary (minimum, 25th percentile, median/50th percentile, 75th percentile, and maximum) to visualize data distribution. They explicitly plot outliers outside the whiskers, making it straightforward to derive numeric thresholds for SIEM detection signatures. - **Multidimensional Anomaly Detection with Isolation Forests:** When data spans many dimensions where visual inspection fails, isolation forests provide an unsupervised machine learning approach to isolate anomalous records through random recursive partitioning. Fewer splits to isolate a point indicate an outlier.

### Technical Details - **Scatter Plot Application:** Bianco demonstrates identifying data exfiltration in a test lab using a producer-consumer ratio (bytes sent vs. bytes received over time). Plotting week one against week two in Python using Plotly highlights hosts that deviate significantly above the "no change" line. - **Box Plot Application:** Bianco attempts to identify malicious PowerShell executions by comparing command-line character lengths. While the specific hypothesis failed due to automated administrative scripts having large command lines, the plot clearly identified outliers beyond ~400 characters. Bianco also notes the alternative statistical heuristic of flagging values 2 to 3 standard deviations above the mean. - **Isolation Forest Implementation:** Bianco describes the tree partitioning process where points needing fewer random splits are scored as outliers. Implemented in Python using scikit-learn, this method was applied alongside collaborator Chris McCubbin in an open-source project called Clearcut to analyze Bro HTTP logs by converting strings/categoricals into numeric values.

### Takeaways Bianco urges the threat hunting community to expand their analytical toolkit beyond basic frequency stacking. Hunters should select outlier detection methods tailored to their data structure: LFO for low-cardinality enumerated data, scatter plots for bivariate numerical correlations, box plots for single-variable distributions and signature thresholding, and isolation forests for complex, high-dimensional datasets.