Skip to main content

Incidents

Study: GitHub Copilot and Amazon CodeWhisperer emit hard-coded credentials seen in training (Huang et al., FSE 2024)

Researchers built 900 prompts from GitHub files with the credential removed. Copilot returned 2,702 well-formed credentials and CodeWhisperer 129; two were confirmed to be working keys.

This record is a measurement reported by a paper or a vendor, not a single dated incident.

Study finding 2023-09-14 plausible

Description

"Your Code Secret Belongs to Me: Neural Code Completion Tools Can Memorize Hard-Coded Credentials" (Huang, Li, Wu, Zhang and Lyu; four of the five authors at the Chinese University of Hong Kong; arXiv 2309.07639, FSE 2024) tests whether code completion tools reproduce credentials from their training data. The authors took GitHub code files that contain a credential, removed it and asked the tool to complete the line: 900 prompts for 18 kinds of secret. After filtering out ill-formed strings, 2,702 of the 8,127 suggestions from GitHub Copilot and 129 from Amazon CodeWhisperer had the format of a real credential. Some were identical to the removed one, others came from elsewhere. Two were shown to be working keys (Stripe test secret keys). A separate figure is often quoted with this study and does not come from it: GitGuardian reports that in a sample of about 20,000 repositories where Copilot is active, more than 1,200 (6.4%) leaked at least one secret, a rate it puts 40% above that of all public repositories. That is an observation about repositories, not a result of the prompting method.

Exhibited failure modes

Affected systems

  • GitHub Copilot
  • Amazon CodeWhisperer

Mitigated by

  • Semgrep — Fast, rule-based static analysis with a pattern syntax that mirrors source code.
  • Snyk — Vulnerability scanning for code, dependencies, containers, and IaC.

Sources

Notes

A study, not an incident; incident_date is the date of the first arXiv version (the id says 2024, the year of the FSE publication). Corrected on 2026-11-21 from the paper and the GitGuardian post (published 2025-03-27). Until then this record said that 6.4% of sampled repositories "leaked at least one secret via this technique": it attached GitGuardian's repository statistic to the study's method. It was dated 2024-05-01 as "month of public write-up", which matches neither source. A second source, prompt.security/blog/securing-enterprise-data-in-the-face-of-github-copilot-vulnerabilities, now redirects to the site's home page and was removed. The listed tools catch committed credentials before merge; they do not stop a model from emitting one.

View raw YAML on GitHub →