What Do We Learn When We Scan the Internet every hour?

Internet-wide scanning probes hosts on specific port and protocol pairs to collect publicly served data, and research built on that data assumes the hosts stay reachable. The speaker noticed hosts that answered, disappeared, and returned hours later, which raised the question of how often scans need to run. To study this, the team scanned about six million hosts that had any of the 40 most common ports open, repeating the scan every 45 minutes for roughly a week. Server capacity limited the size of the experiment.

Lifespans were defined as contiguous runs of successful scans and plotted as cumulative distribution curves. Common pairs such as HTTP on port 80 and SSH on port 22 had similar medians, yet SSH hosts stayed up longer in the tail of the distribution. Email protocols remained online for nearly the whole experiment, which the speaker linked to what those services are designed to do. Behavior also varied by network: the same protocol looked very different across a content delivery network, a cloud provider and a residential ISP. Redefining a host by certificate fingerprint instead of full body hash lengthened measured HTTPS lifespans, because page content changed on each visit.

The main takeaways are that uptime is not uniform across ports, protocols and networks, so scan frequency should match the target. Researchers also need to state what counts as the same host, since that choice changes results. Repeated measurement matters because the network keeps changing.