In Need of 'Pair' Review: Vulnerable Code Contributions by GitHub Copilot

## Overview Presented at Black Hat by researchers Hammond Pierce (New York University) and Benjamin Tan (University of Calgary)—alongside collaborators Brendan Dolan-Gavitt (NYU) and Ramesh Karri—this talk examines the cybersecurity implications of AI-assisted code generation. Following the launch of GitHub Copilot in June 2021 as the first commercial AI "pair programmer," developers gained the ability to generate multi-line code completions in real time directly inside their development environments. However, the speakers observed that Copilot frequently suggests insecure code patterns, prompting an empirical, systematic academic investigation into the security posture of Copilot's generated outputs.

## Motivation and Initial Observations The shift from manual code search (such as copying solutions from Stack Overflow) to context-aware generative AI tools changes software development dynamics. While exploring Copilot shortly after its release, the researchers tested Python web server development using Flask. During login form creation and database querying, Copilot automatically generated strings that introduced direct SQL injection vulnerabilities. A recreation demonstrated that Copilot generated vulnerable SQL string concatenations for authentication workflows, demonstrating that the model reproduces common coding flaws despite high developer productivity.

## Research Scope and Core Questions To assess these security risks, the research team structured their inquiry around three central objectives: - **Evaluation Methodology**: Defining an empirical approach to test the security of code produced by Copilot across various coding contexts. - **Experimental Findings**: Measuring the prevalence and types of vulnerabilities emitted by the model under test conditions. - **Security Implications and Mitigations**: Assessing the broader industry impact of integrating large deep neural network models (such as GPT-3 derivatives) into active software development lifecycles and outlining defenses against vulnerable AI contributions.