The Grafana data breach was caused by a single GitHub workflow token that slipped through the rotation process following the TanStack npm supply-chain attack last week.
In the ongoing Shai-Hulud malware campaign attributed to TeamPCP hackers, dozens of TanStack packages infected with credential-stealing code were published on the npm index, compromising developer environments, including Grafana’s.
When the malicious npm package was released, Grafana’s CI/CD workflow consumed it, and the info-stealer module executed in its GitHub environment, exfiltrating GitHub workflow tokens to the attackers.
The company explains that it detected malicious activity resulting from compromised TanStack packages on May 1, and immediately deployed the incident response plan, which included rotating GitHub workflow tokens.
However, one token was missed in the process, and the attacker used it to gain access to the company’s private repositories.
“We performed analysis and quickly rotated a significant number of GitHub workflow tokens, but a missed token led to the attackers gaining access to our GitHub repositories,” reads Grafana’s update.
“A subsequent review confirmed that a specific GitHub workflow we originally deemed not impacted had, in fact, been compromised.”
Previously, the company confirmed that the intruders stole source code, assuring there was no customer impact, and stating that the hackers would not receive a ransom payment.
The continued investigation revealed that the intruder also downloaded operational information and details Grafana uses for its business.
“This includes business contact names and email addresses that would be exchanged in a professional relationship context, not information pulled from or processed through the use of production systems or the Grafana Cloud platform” – Grafana
The company stresses that this was not customer production data, and according to the latest evidence and investigation, no customer production systems or operations have been compromised.
Grafana Labs also noted that its codebase was not modified during the incident, so the code users downloaded throughout the events is considered safe, and users are not required to take any action.
If that evaluation changes based on new evidence from the ongoing investigation, Grafana Labs promised to notify impacted customers directly.
Automated pentesting tools deliver real value, but they were built to answer one question: can an attacker move through the network? They were not built to test whether your controls block threats, your detection rules fire, or your cloud configs hold.
This guide covers the 6 surfaces you actually need to validate.
