Correcting Attribution: TeamPCP’s Supply Chain Attack Predominantly Exploited Trivy, Not LiteLLM
Recent analysis from SOCRadar has clarified the primary vector behind a widespread supply chain attack initially attributed largely to LiteLLM. The compromise, impacting over 2,500 organizations, was predominantly facilitated by the earlier exploitation of Aqua Security’s Trivy scanner. This sophisticated campaign, claimed by the threat actor TeamPCP, leveraged the Shai-Hulud worm to exfiltrate critical secrets and propagate malicious code across numerous open-source software (OSS) packages and repositories. The incident underscores the pervasive risks within the software supply chain and the critical need for vigilant security practices beyond initial reports.
Technical Analysis of the TeamPCP Supply Chain Attack
The attack chain began with the compromise of Aqua Security’s Trivy scanner, specifically a malicious build published on March 19. TeamPCP, known for multiple OSS supply chain attacks, injected malicious code that automatically executed when an infected package was fetched and run. This code’s primary objective was to harvest sensitive data, including credentials, authentication tokens, API keys, and other secrets. The worm-like behavior of the Shai-Hulud malware enabled a ripple effect, propagating downstream to numerous packages and repositories as the malicious libraries were automatically included in subsequent builds. This mechanism explains how the attack surface expanded rapidly, affecting thousands of organizations worldwide.
The misattribution to LiteLLM arose because two poisoned LiteLLM package versions were published on March 24, containing a .pth file that Python automatically executed at interpreter startup. However, SOCRadar’s examination of 2,188 per-organization records, including timestamps and credential types, revealed that for 95% of these entities, data collection activity concluded before the LiteLLM packages were published. The earliest collection occurred merely 18 minutes after the malicious Trivy build became live, with activity surging on March 22 and 23 when malicious Trivy images were available on Docker Hub. The compromise timeframe, largely from March 19 to March 24, aligns directly with the upstream Trivy compromise rather than the brief LiteLLM exposure window. This persistent payload execution on already-infected hosts, even after the source infection was removed, highlights the worm’s insidious nature.
Shai-Hulud Worm Credential Harvesting
The Shai-Hulud worm’s credential harvesting capabilities were extensive, targeting a broad spectrum of secrets. Over 1,000 organizations exposed JWT and authentication tokens, while hundreds more disclosed private keys, AWS access keys, GitLab tokens, OpenAI API keys, Slack webhooks, GitHub Actions tokens, and Google API keys. One unnamed organization alone had roughly 3,477 secrets compromised. Additionally, committer email addresses were exposed across more than 1,100 organizations, granting attackers both developer identities and machine tokens, significantly escalating the potential for further attacks.
The compromised data is actively being brokered. Reports indicate a threat actor is offering a collection of LiteLLM, Trivy, and CanisterWorm data on Telegram, compiled from various stages of this campaign. This confirms the direct financial and strategic motivations behind the TeamPCP supply chain attack analysis.
Affected Systems and Global Reach
The compromise involved six prominent CI/CD platforms: GitHub Actions, GitLab CI, Jenkins, Bitbucket, CircleCI, and Buildkite. Organizations across the globe were impacted, with Germany, Brazil, and France experiencing the highest concentration of affected entities. The widespread nature across diverse CI/CD environments underscores a significant challenge for detecting and mitigating such sophisticated supply chain attacks. High-confidence matches for affected organizations were keyed on CI host identity and legitimate committer domains, indicating exposure rather than confirmed exploitation in all cases, but signaling a severe risk.
Mitigating Trivy Scanner Compromise Risks
Given the scope and nature of this attack, organizations must act decisively to secure their software supply chain and mitigate potential damage from the Trivy scanner compromise.
- Immediate Secret Revocation and Rotation: All developer secrets, API keys, authentication tokens, and private keys that were active between March 19 and March 24, 2024, should be immediately revoked and rotated. This includes credentials for CI/CD platforms (GitHub Actions, GitLab CI, Jenkins, Bitbucket, CircleCI, Buildkite), cloud providers (AWS, Google), and third-party services (OpenAI, Slack).
- Enhanced Supply Chain Security: Implement stringent supply chain security practices, including validating the integrity and authenticity of all upstream dependencies, even for security scanning tools like Trivy. Utilize software composition analysis (SCA) tools and regularly audit all components for suspicious activity or known vulnerabilities.
- CI/CD Environment Hardening: Regularly audit and harden CI/CD pipelines. Ensure least privilege access for all automation accounts and tokens. Implement multi-factor authentication (MFA) everywhere possible. Monitor CI/CD logs for unusual activity, unauthorized package modifications, or unexpected deployments.
- Developer Workstation Security: Educate developers on the risks of open-source software supply chain attacks and secure coding practices. Enforce strong endpoint security measures on developer workstations to prevent initial compromise that could lead to token theft.
- Threat Intelligence Integration: Stay informed about emerging supply chain threats and threat actor TTPs. Integrate relevant threat intelligence feeds to proactively identify and respond to indicators of compromise. Regularly search for “how to detect Shai-Hulud worm exploit” within your environment’s logs and network traffic.
Related: npm Supply Chain Attacks: Shai-Hulud, Miasma, and CI/CD Compromises, Shai-Hulud Campaign: TeamPCP Targets Open-Source Supply Chain