PhishDefend: A Reinforcement Learning Framework

The talk argues that internet measurement, the practice of quantifying parts of the internet including its security aspects, can guide better defenses. It is built around three projects. The first examines an underground market offering to break into Gmail accounts for a fee. The researchers posed as buyers and as fictional victims with constructed online footprints and tracked the services. Of 27 services found, five made an attempt, and those relied on highly targeted phishing that adapted when two-factor authentication blocked the first attempt.

The second project compares an internet-wide scanning engine with nmap. The initial overlap was about 87 percent, and many discrepancies came from hosts that appeared and disappeared between scans. A head-to-head rerun showed each scanner finding services the other missed, and the team tracked down the causes by manual verification. They added a "pending removal" status to the data so users can see hosts that are being rechecked before they are dropped, which sharply reduced the apparent false positive rate.

The third project addresses patching at a large university IT organization. A weekly laundry-list email was easy to ignore, so the team rewrote notices to cover one vulnerability family at a time, with instructions and a spreadsheet of details. Patch rates rose from about 3 percent to 78 percent. Interviews with administrators revealed monotony, exceptions tracked through a forgotten form, and patch cycles that did not match the notices. The takeaway is to measure the human workflow as well as the machines, and to talk with the people who do the work.