And what if it was hacked? Tactics and Impacts of Adversarial Machine Learning

The talk opens with a business concern: generative AI is speeding up attackers, and a cited industry survey found many executives worried about adversarial capabilities such as phishing, malware and deepfakes. The speaker gives a short primer on how artificial intelligence, machine learning, deep learning and language models relate, then defines adversarial attacks as inputs crafted to make a model behave against its design. A well-known image misclassification example illustrates the idea before the focus moves to jailbreak prompts for chat assistants. These are grouped into categories including prompt injection, prompt leaking, role-play, developer-mode framing, token manipulation and switching to other languages.

The middle of the session is a live demonstration against a commercial chatbot. The speaker first makes a request the bot refuses, then adds a jailbreak persona taken from a public group and gradually builds a simple keylogger, a data-collection server and a phishing email with HTML through conversation alone. Other real-world examples cover a sales bot talked into selling a product for under a dollar, and a prompt that caused a model to reveal its training data. The point is that guardrails can be sidestepped without conventional exploit code.

To close, the speaker recommends educating staff and developers about LLM risks, improving guardrail hardening, and running AI red teaming to see how systems respond to users. Several published AI security frameworks are listed as starting points. The takeaway for practitioners is that any AI feature exposed to users needs monitoring and testing, since new jailbreaks appear constantly.