Pentests Using LLMs

The talk opens with a broad look at artificial intelligence in cyber security. The speaker argues that AI is a flexible base for building security tools and that the field is changing quickly enough to matter now. The discussion focuses on offensive use, and the speaker also raises the risks that more pessimistic views of machine intelligence point to.

The core of the talk sorts offensive AI into five areas: social engineering with deepfakes and phishing, evading detection with polymorphic malware, reconnaissance through open-source intelligence and automated target discovery, credential attacks against biometric systems, and automated exploit development, including malware co-pilots. The argument is that a large share of common attacker tactics could be automated or sped up with language models, which would make them cheaper to run. The open-source PentestGPT project was cited as an existing example, alongside a proprietary project still in development.

The speaker then covered the challenges: ethical concerns, false positives, and the reliability of model output. The need for more targeted exploit generation, so defenders can test their own systems and build countermeasures, was also raised. The talk noted that AI models have their own weaknesses, such as prompt injection and training data poisoning. The closing takeaway is that the industry should take a responsible and strategic approach so defenders can prepare for these capabilities.