Spear-Phishing at Scale using Generative AI

The talk looks at how generative AI changes email-based attacks, which remain a leading route for social engineering. The argument is that attackers are chasing better return on investment, and generative models cut the time and cost of writing targeted lures while making them more convincing. The examples come from attacks seen in the wild, with the caveat that AI authorship can only be judged as likely rather than proven.

The examples move from polished, generic invoice fraud with no spelling mistakes, to benefits-enrollment phishing whose attached PDF carried a QR code, to the most elaborate case: a fabricated email thread with invented replies from real people at the target organization about a real event, sent from a newly registered lookalike domain. The speaker notes that AI-text detectors gave unreliable verdicts, with many false positives and misses, so they should not be relied on as a primary signal. A short exercise showed that scraped profile data and recent posts, fed into a prompt, produced a believable lure after little iteration.

For detection, many existing signals still apply, such as first contact from an unknown sender and malicious documents or links. Defenders can also use language models for intent and entity extraction. Practical advice includes layered defenses, user education about convincing lures, phishing-resistant multi-factor authentication such as hardware keys, and out-of-band approval for large payment changes.