Designing and Participating in AI Bug Bounty Programs
Deep dives into adversarial prompt injection, context poisoning, and RAG manipulation flaws to help researchers find and demonstrate high-impact AI vulnerabilities.
Summary
The Emerging AI Vulnerability Landscape
Integrating Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) into applications introduces novel vulnerability classes that bypass traditional web defenses. Understanding how AI models process context and instructions is essential for uncovering high-impact flaws.
Adversarial Prompting and Indirect Prompt Injection
Attackers craft indirect prompt injection payloads embedded within external data sources, web pages, or documents. When an AI agent ingests this untrusted content, the embedded prompt hijacks the model's instruction flow to execute unauthorized actions or exfiltrate private data.
Context Poisoning and RAG Manipulation
Retrieval pipelines can be manipulated by injecting malicious documents into vector knowledge bases. By poisoning indexed context, attackers alter model reasoning, force incorrect data retrieval, and trigger unauthorized API executions.
Demonstrating Business Impact in Bug Bounties
Successfully reporting AI vulnerabilities requires researchers to articulate clear exploit chains and business consequences. Documenting reproducible multi-step prompt injection chains helps organizations understand severity and implement robust guardrails.
Related talks
Crypto and Privacy Village·48 min
Patrick Walsh
Demonstrates model inversion and embedding inversion attacks against LLMs and RAG systems, showing how sensitive personal data can be extracted from vector databases.
Main Stage·19 min
Tobias Diehl
Demonstrates how adversaries exploit data voids in LLM training corpora and search indexes to manipulate Microsoft Copilot answers and inject untrusted content.
Creator Stage·22 min
D. Jurado · J. Nogue
Chronicles the deployment of XBOW, an autonomous AI offensive agent that discovered over 200 open-source vulnerabilities and submitted 1,000 bug bounty reports.