Predicting the Lifespans of Internet Services: Falling down the ML Rabbit Hole, and What We Learned From The Thud

Ariana Mirian, a researcher at an internet scanning company, describes why hourly scan data was hard to trust. Many hosts are ephemeral, appearing and vanishing between checks, so the team set out to predict how long each service would stay up, which could guide how often it gets rescanned.

Regression failed first. Port, protocol and autonomous system are categorical, and one-hot encoding left very sparse matrices given roughly 40 ports, 300 protocols and 19,000 autonomous systems. Grouping autonomous systems helped little, R-squared stayed around 0.3 at best, and feature rankings changed from model to model. Seeing that most hosts lived long with a small short-lived tail, the team switched to classifying hosts as under or over 12 hours. Several classifiers agreed on feature importance, reaching about 87 percent accuracy and a 0.94 ROC AUC, with autonomous system ranked highest. Software attributes such as vendor and version ranked low next to network-level features.

The takeaways were as much about process as models. Experts from both networking and machine learning were needed, since dropping autonomous system was rejected on domain grounds. Reframing the question was worth more than tuning, and an actionable answer beat a publishable one. In the Q&A, the speaker explained that scanner speed was tuned against known-live hosts and that a one-hour gap was counted as a false negative, while retries were skipped in the experiment due to scan budget.