GPT-3 and me

The talk, given by two data scientists from Sophos, argues that very large self-supervised models are changing what machine learning can do, and that security researchers should work out how such models fit defensive problems. The speakers explain self-supervised training, where a model learns from unlabeled data by solving an artificial prediction task, and show how capability grows with model size. GPT-3 is too large to run on ordinary hardware, so the work relies on API access and its cost.

The middle of the talk turns to experiments. One demonstration translates obfuscated malicious command lines into plain-English descriptions that analysts could use to speed triage. Output improved when signature names from detection rules were added to the prompt, and the team picked the best candidate by generating a command back from each description and comparing it to the original. A second experiment classifies spam messages from only a handful of labeled examples through in-context learning, reaching roughly 90 to 95 percent accuracy, while a random forest baseline barely beat guessing. Fine-tuning on around a thousand samples pushed accuracy higher but made inference too slow for routine use.

The closing message is that few-shot prompting can produce a usable prototype quickly, but cost and latency restrict it to selective use, such as on alerts that already look suspicious. The speakers also floated translating natural-language questions into database queries and suggesting command completions for analysts as areas to explore.