Smart Threats, Smarter Defenses: Leveraging Machine Learning to Shrink and Sharpen Cyber Threat...

## Overview Wilson Chua presents a machine learning approach to reducing cyber threat analyst workload by 60–90% through intelligent IP blocklisting at the network edge. Using data analytics and AI, his team compressed 3–4 million known malicious IPs into a compact, optimized blocklist of 70–80k IPs while maintaining effective attack prevention. On a typical day, this blocks 60–70% of attacks; on good days, up to 90%.

## Key Topics **Attack Volume Reduction & Impact**: Using the example of a reported 1.4 million attacks in one day, the team demonstrated that reducing attack volume from 1.4 million to 140k (via 90% blocking) dramatically decreases forensics workload. Blocking attacks at the edge prevents them from reaching internal servers and internal security teams.

**Cyber Kill Chain Defense**: By blocking known malicious IPs, the approach disrupts all seven stages of the cyber kill chain—from reconnaissance (blocking IPs from Shodan, Census, Binary Edge) through to C2 callbacks and malware delivery. Attackers often rely on known compromised infrastructure.

**Machine Learning Model**: The team used XG Boost to build a predictive model that identifies the top 70k IPs most likely to attack in the coming week based on current week traffic patterns. A confusion matrix showed strong prediction accuracy.

## Technical Details **Pareto Principle in Action**: Analysis revealed that 1.6 million of the monitored IPs never actually attacked any networks, while the bulk of attacks came from a small, high-volume subset. The three key predictors of persistent threat were: (1) number of incidents (IPs with >10,000 hits in a week tend to persist), (2) number of destination IPs targeted, and (3) number of ports attacked. Highly aggressive IPs that scan many destinations and ports are most predictive of future attacks.

**Data Sources & Consensus Approach**: IP data was gathered from AbuseIPDB, VirusTotal, Snort, Apenic, and other blocklist sources. The team applied an algorithm requiring consensus from two or more sources to minimize false positives—though early versions mistakenly included legitimate IPs from Google, Azure, and AWS.

**Firewall Impact**: Testing on a MicroTik RB3011 (a basic, low-cost firewall common in the Philippines and Asia) showed that loading 70k IPs caused a CPU spike during script execution but returned to normal afterward, with no degradation in throughput (tested at ~124 Mbps with A+ buffer bloat rating). Forinet and pfSense were also tested successfully.

**Weekly Updates & Community Model**: Pre-formatted blocklists are published free on GitHub for MicroTik, Forinet, and pfSense, updated every seven days. The team uses an open-source, community-driven model, asking users to report false positives or missed IPs to improve the lists.

## Key Takeaways - Machine learning can significantly reduce threat analyst workload by identifying and prioritizing the highest-risk IPs from massive datasets. - Edge-based blocking (at firewalls and routers) prevents attacks from reaching internal networks, enabling triage and focus on the 20–30% of attacks that do penetrate. - Smaller, intelligently selected IP blocklists are more practical and performant than exhaustive lists; basic firewalls cannot handle millions of entries. - The Pareto principle applies strongly to cybersecurity: most attacks come from a small fraction of threat IPs. - Open-source, community-validated blocklists are feasible and beneficial, particularly for organizations using commodity hardware in developing regions. - Sensitive ports (22, 23, 21, 3389, 3306, 1433) warrant triage focus; blocking at the edge reduces the scope of forensic investigation. - Automated abuse notifications to IP owners encourage remediation of compromised systems.