#HITB2021AMS HATS: Catch up with Ero Carrera and Sergey Lozhkin

# HITB2021AMS: Threat Intelligence, Machine Learning, and Application Security

## Overview Ero Carrera, a 20-year veteran in infosec who spent the last decade at Google on large-scale malware analysis and threat intelligence automation, discusses the intersection of threat intelligence, machine learning, and security practices. He is joined by Dennis, another trainer, to explore both the opportunities and security implications of AI/ML in security operations, as well as application security best practices.

## Key Topics

### Threat Intelligence at Scale Carrera highlights the fundamental difference between threat intelligence operations at Google versus typical enterprises: access to massive infrastructure supporting billions of rows of data across terabytes, enabling instant queries. Google's Threat Analysis Group consisted of approximately 20 people, including former government intelligence analysts, top reverse engineers, vulnerability researchers, and specialized threat analysts. While fresh data is crucial for tracking current attacks, maintaining historical context over years helps security teams understand threat actor evolution and infrastructure reuse patterns.

### Machine Learning Reality vs. Hype Carrera expresses healthy skepticism about ML/AI, emphasizing that these techniques cannot do magic but excel at automating patterns within workflows. Approximately 90% of practical ML work involves debugging data—understanding incorrect labels, acquisition issues, sensor problems, and inconsistencies across tools. He advocates for interpretable algorithms (Random Forest, Logistic Regression, Bayesian networks) over neural networks in security contexts because decisions often require justification for legal compliance and operational action. When stakes are high and actions depend on model verdicts (infrastructure sinkholes, etc.), the reasoning trail is essential.

### Feature Engineering and Expertise Successful ML implementation requires both domain expertise and machine learning expertise working together. Domain experts understand data subtleties—what API calls mean, sandbox outputs, debugging approaches. ML practitioners understand algorithm behavior, overfitting, and parameter tuning. Carrera describes specific applications at Google: ML models for recognizing compiler patterns during disassembly at massive scale, and ranking evidence in threat intelligence workflows.

### AI/ML Security Concerns Multiple attack vectors emerge: - **Model Reversal**: Attackers can reverse-engineer black-box models to extract intellectual property and understand what the model knows, potentially revealing sensitive intelligence collection capabilities. - **Oracle Attacks**: Repeated queries to a model can gradually extract information about its training data, especially problematic if the model trained on private or sensitive information. - **Inference Attacks**: Attackers can construct computationally expensive inputs to cause denial-of-service on inference infrastructure, exploiting expensive code paths in model evaluation. - **Data Poisoning**: False data or bias in training data can mislead models or hide malicious activity below the model's detection threshold.

Google's infrastructure includes options for resource limiting per inference job, though standard frameworks may not explicitly address such attack vectors.

### Application Security and Developer Training Dennis emphasizes that application security repeats the same mistakes across decades—techniques evolve but fundamental vulnerabilities persist. The root cause lies in human factors; developers frequently copy-paste vulnerable code from Stack Overflow, import libraries with unpatched vulnerabilities, and lack security awareness. OWASP and security championship programs address this by targeting developer education rather than creating dedicated security experts. The key is moving vulnerability fixes left in the software development lifecycle (to code review phases) rather than discovering them late in release cycles. Developer security training differs fundamentally from attacker training: security is one feature among many for developers, requiring practical, code-focused education rather than tool mastery.

## Takeaways - Infrastructure scale is a powerful enabler; advanced algorithms matter less than foundational tooling at enterprise scale. - Interpretability and debuggability of ML models are often more valuable than raw accuracy in security contexts where decisions must be justified. - Data quality and understanding dominate ML project timelines; poor data engineering undermines even sophisticated algorithms. - ML models in security operations require protection from reversal, inference attacks, and computational DoS. - Developers are the root cause of most application vulnerabilities; training programs should target development teams, not create isolated security experts. - Moving security left in the development lifecycle through developer education is more cost-effective than late-stage penetration testing or bug bounties.