How Generative AI is Shaping Online Threats

Shelby Grossman of the Stanford Cyber Policy Center asks whether generative AI has changed what online threats look like or only how cheaply they are produced. She defines covert influence operations as coordinated efforts to mislead, where the deception often sits in the account rather than the content. Comparing older campaigns with AI-era reports from model providers, she finds familiar narratives. A survey experiment found AI-written propaganda slightly less persuasive than human-written propaganda, but about as persuasive once a person edited it.

The talk then turns to harassment, child safety and fraud. Harassment campaigns using AI so far look crude and draw little engagement, compared with a sophisticated pre-AI operation that abused copyright reporting against activists. Nudify apps and AI-enabled sextortion lower the effort needed, since a scammer no longer needs a genuine first image. Grossman says feared floods of synthetic abuse material into the CyberTipline have not yet materialized, though overload remains a concern for investigators.

She closes with defensive uses and dilemmas. Student-built LLM moderation bots outperformed traditional tools in a course exercise, while dual-use requests, such as a model answering questions that could help either a school administrator or an attacker, make safeguards hard to set. She urges practitioners to build the trust and safety field through peer-reviewed research and community events. The takeaway is that AI mostly makes existing harms cheaper and faster to scale.