Practitioners' Perception of ML-Based Security Tools and Explanations

Study Design

Jaron Mink (UIUC) presents interview findings on SOC practitioners’ views of ML-based security tools and explanations. Eighteen professionals with at least one year using classification tools participated in hour-long interviews covering backgrounds, ML usage, and three explanation types.

ML Plus Rules in Practice

Most SOCs combine ML with rule-based detection rather than using either alone. Rules anchor detection for known behaviors with lower false positives; ML supplements detection of novel activity but alone draws skepticism due to false-positive load.

Benefits and Pain Points

Practitioners weigh usability, effectiveness, adaptability, efficiency, and security. ML’s black-box outputs hinder trust, yet complex third-party rules are also opaque—“find who wrote the signal.” Hybrid approaches balance coverage and alert volume.

Explanations and Needs

Added ML explanations are evaluated against whether they resolve operational challenges. Correct classification under non-adversarial conditions remains the paramount unmet need; practitioners doubt ML alone meets SOC requirements despite industry AI hype.