The 11th Annual Black Hat USA Network Operations Center (NOC) Report

## Overview

At Black Hat USA, Neil "Grifter" Wyler (VP of Defensive Services at Coalfire) and Bart Stump (Managing Principal, Defensive Services at Coalfire) presented the 11th Annual Network Operations Center (NOC) Report. Wyler highlighted that this was his 23rd year running the Black Hat NOC. Starting originally with just a few personnel handling 1,500 attendees and 15 training classes using open-source scripts and off-the-shelf gear, the network operations have scaled to support 20,000 attendees and over 90 training classes. Today, the NOC runs with a crew of roughly 70 people alongside key industry technology partners.

## Infrastructure, Architecture & Partner Integrations

The NOC network is deployed in approximately four days, thoroughly tested during a Friday "burn-in" day, and torn down within hours after the event:

* **Core Routing, Firewalls & Automation:** Palo Alto Networks handles core firewalling and routing, as well as Cortex XSIAM for log ingestion, alert/incident generation, ticketing, and automated playbook resolution. * **Switching & Wireless:** Arista Networks supplies switching infrastructure and access points. The NOC itself is placed on an isolated Intermediate Distribution Frame (IDF) switch to avoid past traffic issues. * **Endpoint Management & DNS:** Cisco provides MDM across iOS devices (used for attendee registration and lead retrieval) and Cisco Umbrella for DNS security. * **Proactive Monitoring:** The team placed custom-labeled Raspberry Pi and Orange Pi appliances throughout the venue running Cisco ThousandEyes agents to proactively identify network degradation before attendees report issues. * **Upstream Connectivity:** Lumen provides two 10 Gbps circuits (20 Gbps aggregate). Bandwidth usage remained relatively light, averaging 800 Mbps and peaking at only 4.2 Gbps during trainings.

## Key Traffic Statistics & Observations

Full packet capture (PCAP) and telemetry across the conference revealed notable network trends:

* **Traffic Volume & Encryption:** The NOC captured over 63 billion packets totaling more than 62 terabytes of data. Encrypted traffic broke the 90% threshold for the first time at Black Hat USA, reaching 91% (contrasting with historical plateaus of 75–80%). * **Connected Hosts:** Telemetry observed just shy of 59,000 unique hosts. The presenters estimated that approximately 50% of attendees actually connect to the conference Wi-Fi. * **Cleartext Exposures:** The NOC captured 62 hosts transmitting unencrypted passwords, recording roughly 2,800 unique plaintext credentials across the wire. * **Application Trends:** App telemetry showed iCloud surpassing Google in traffic, Slack retaining its top workspace spot, NiceHash beating XMRig in cryptocurrency mining traffic, and Hinge surpassing Tinder as the most-used dating application.

## Defensive Operational Constraints & Security Philosophy

Operating the Black Hat network requires balancing aggressive defense of core infrastructure with minimal interference to attendee activity:

* **Crown Jewels Protection:** The attendee registration servers faced diverse active attacks throughout the conference, necessitating strict defensive posture around conference administrative assets. * **Zero-Blocking Policy for Attendees:** Because attendees, speakers, and training classes actively demonstrate exploits, communicate with remote attack labs, and test custom tools, the NOC deliberately does not block anomalous or hostile traffic. Blocking suspicious traffic would break legitimate lab exercises and vendor presentations. * **Needle in a "Needle Stack":** Rather than looking for anomalies in normal business traffic, the NOC monitors a baseline saturated with offensive activity. As a result, the team actively detects actual background malware infections, unauthorized leaks, and existing device compromises that attendees' home enterprise security tools missed, notifying affected users so they leave more secure than when they arrived.