Botnets are the Best Way to Measure User-Hostile Behavior on the Internet

The talk starts from a research problem: web measurement studies need to observe how sites treat visitors, including dark patterns such as hard-to-find cancellation flows and misleading consent dialogs. The speaker reviews the two usual approaches. Bot farms are fleets of automated browsers run on servers the researcher controls. They are costly to maintain, behave in ways that sites can detect, and offer little network diversity. Piggybacking uses browser extensions installed by volunteers, which adds a privacy and performance burden for participants and limits measurements to whatever sites those people happen to visit.

The middle of the talk presents an alternative: a measurement botnet in which a small client runs on volunteers' home machines, pulls studies when spare capacity is available, and returns compressed, signed results. The client isolates the browser on Linux using systemd sandboxing with system call filtering, namespaces and control groups, and it claims only a fraction of system resources. Volunteers can schedule ride-alongs to watch what the bot is doing, and studies can ask a human for judgments on pages, which helps with subtle design choices that are hard to model automatically. The speaker also explains the choice of a fork of OpenWPM for browser instrumentation and covers design questions such as Docker versus native isolation and macOS support.

The closing part addresses risks and open problems: poisoning by untrustworthy participants, which may be countered by re-measuring the same sites, and legal uncertainty under computer misuse law and terms of service. The talk ends by placing measurement botnets within a history of volunteer distributed computing and invites contributions on isolation, scheduling, bot detection, packaging and incentives.