Builders & Breakers: A Collab Look at Securing LLM-Integrated Apps

The session pairs an application builder with a penetration tester who specialises in AI red teaming. It starts by contrasting traditional software with LLM-integrated applications, where the infrastructure looks much the same but meaning now matters as well as syntax. Two entries from the OWASP Top 10 for LLM applications, prompt injection and data poisoning, frame the rest of the talk.

The offensive part uses a demo of a refund chatbot for a web shop, where a model decides whether refunds are granted. Simple rewording and fake formatting can flip the decision, and a poisoned training example can force approvals. The tester also fuzzes the chatbot's endpoint with thousands of injection payloads through a Burp extension, and some nonsense answers get accepted. Bug bounty data shows vendors downgrading jailbreaks and prompt leaks, so researchers are pushed toward issues with real impact such as data access.

The defensive part covers moderation services, classifiers and open-source AI firewalls, which can still be bypassed, along with trust boundaries and access control on the functions a model can call. A GitLab chatbot case shows how a planted comment led to private code leaking through rendered HTML and an image request; a strict content security policy and sanitised output would have blocked it. The takeaways: classic appsec controls still apply, testing must be repeated because outputs vary and models change, and the application builder shares responsibility for defences.