Tim Grogan - 2025

## Pentesting Custom AI Models

Tim Grogan, a cloud security engineer with a pen-testing background, shares findings from recent engagements where clients built proprietary on-prem AI models—not off-the-shelf ChatGPT—for specific business services trained on private datasets.

## Threat Modeling AI Pipelines

For each client, the team modeled risks around user submissions, file uploads, and integration APIs. Primary test goals included data poisoning, fuzzing training inputs, and manipulating model decisions to produce attacker-favorable outcomes.

## Adversarial Techniques

Testers flooded models with junk or misleading data to shift classification behavior and embedded prompt-injection strings—jailbreak-style instructions—in metadata and images to override safety guardrails, similar to tricks used against public LLMs.

## Lessons Learned

Custom AI expands the pen-test scope beyond classic web flaws. Organizations shipping ML features need explicit abuse cases, input validation, output monitoring, and red-team exercises focused on model integrity—not just API authentication.