Security researchers at Noma Security have demonstrated a vulnerability in GitHub Copilot that allowed them to extract private repository code through prompt manipulation techniques.

The attack, dubbed "GitLost," exploited how GitHub's AI coding assistant processes context from private repositories when generating code suggestions. The researchers found they could craft specific prompts that caused Copilot to inadvertently leak snippets of private code it had been trained on or had access to.

How the Attack Worked

The vulnerability centered on Copilot's context-awareness feature, which analyzes nearby code to provide relevant suggestions. By carefully structuring prompts and code comments, the researchers could manipulate the AI into revealing private repository contents that should have remained confidential.

The team tested various prompt injection techniques, finding that certain patterns of comments and incomplete code blocks could trigger Copilot to complete suggestions with private code fragments. The leaked information included function names, variable declarations, and in some cases, entire code snippets from private repositories.

"We were able to consistently extract private code by exploiting how Copilot interprets context," the researchers wrote in their disclosure. "The AI assistant treated our crafted prompts as legitimate requests for code completion."

GitHub's Response

GitHub acknowledged the vulnerability and has implemented mitigations to prevent similar attacks. The company said it has updated Copilot's filtering mechanisms to better detect and block prompt injection attempts that could lead to private code exposure.

The vulnerability highlights broader security concerns around AI coding assistants that have access to vast amounts of source code, including potentially sensitive private repositories. Similar issues have been reported with other AI development tools that train on or access private codebases.

Noma Security said it followed responsible disclosure practices, reporting the vulnerability to GitHub before publishing their findings. The company has not disclosed whether the vulnerability was actively exploited in the wild before the fix was implemented.