Unlocking Generative AI: Balancing Innovation with Security

Jason Ross opens by framing generative AI as a technology that produces new content rather than retrieving or predicting it. He highlights two traits that matter for organizations: outputs are non-deterministic, so the same input can yield different results, and the content is persuasive enough that people tend to accept it without checking. He walks through common corporate uses such as marketing, internal search, meeting summaries, build reporting and incident reporting, and explains how hallucinations and data exposure can creep into each.

The middle of the talk follows the OWASP Top 10 for LLM applications. Ross covers prompt injection, arguing that the real concern is data exposure rather than offensive text, along with model theft, data poisoning and excessive agency. He notes that poisoning increasingly happens through retrieval-augmented generation data stores and internal wikis rather than the model itself, and that model files can behave like code, so supply chain risk spans the whole stack. He also raises the difficulty of honoring deletion requests once data has been trained into a model.

For defenders, the takeaway is that general application security still applies, but training environments should be treated as production, training data needs vetting, and model artifacts should be compared against known good versions. Legal contracts with model providers and ethics review become part of the security program. Ross closes by noting that model output can change between versions and that the OWASP list is being revised with community input.