Prompt Hardener

The talk opens with the rapid spread of generative AI in business applications, where easy API access lets developers add AI features quickly and widens the attack surface. A real case showed a crafted prompt in a text-to-SQL library leading to arbitrary Python execution on a server. The presenters stress that no single control is enough, so prompt hardening sits alongside keeping secrets out of prompts, runtime output checks and least-privilege access.

The core of the talk is Prompt Hardener, an open-source tool that uses an LLM to evaluate and rewrite a system prompt in repeated feedback loops until scores pass a threshold. It applies four techniques: spotlighting, which marks untrusted input by replacing spaces with a special Unicode character; random sequence enclosure, which wraps instructions in unpredictable tags; instruction defense, which defines responses to detected attacks; and role consistency, which keeps user-supplied text in the user role. Attack testing uses payloads grouped by the OWASP Top 10 for LLM applications. Two demos, a comment summarizer and an internal FAQ bot, showed the command-line and web interfaces.

Benchmarking with a separate red-teaming tool across 258 payloads per prompt raised the defense rate of the FAQ bot from about 67 percent to 94 percent, using an older GPT-3.5 model. Some categories, such as over-reliance, improved little. The takeaway is that changing the system prompt alone can meaningfully strengthen defenses without changing the model, though it is not a complete fix. Future work targets agent-based applications and newer hardening research.