Innovation in the Hands of Threat Actors: Analyzing Supply Chain Attacks

Hot on the heels of highly publicized attacks like those affecting Solarwinds and Codecov, organizations are taking a firm stance on software supply chain security. But in order to be effective at securing the slurry of artifacts that get incorporated into cloud native applications, we must first understand the enemy. In this post, I’ll explore some of the most popular attack types that malicious actors are using to infiltrate the software supply chain and initiate a cascade that can affect countless organizations.

Supply chain attacks in the age of cloud native

Modern applications are increasingly defined by complex, multi-layer structures of containers, custom code, open source components, and third-party artifacts. This new software composition has brought with it new DevOps processes, CI/CD workflows, and automation unlike any we’ve seen in earlier iterations of the software development life cycle. With these advances, attackers have been inspired to innovate, taking advantage of the accelerated development and delivery to infiltrate the software supply chain and carry out sophisticated attacks that slip undetected past traditional application security testing.

While software vulnerabilities still play a role in breaching organizations’ defenses, the software supply chain introduces an inordinate degree of new opportunities to introduce malicious artifacts and to execute unauthorized activities from within. Remember: Malware is not a vulnerability, so it can neither be detected nor resolved using the same methods.

There’s a wildly diverse range of these attacks, and they can be quite complex, yet many share similar basic principles and take advantage of developers’ oversights and security limitations of the CI/CD pipeline. Let’s analyze these further using some examples:

Typo-squatting

This is a tactic of naming malicious files with various misspellings of popular open-source or public libraries. One example of this occurred with the Jellyfish library, which used a misspelled version of the popular library in which attackers replaced the first “L” with a capital “I” [eye]. At the time of discovery, the malicious package had been available for a year, and it had become associated with another popular Python Package Index (PyPI) library, which then became malicious by association, extending the scope of influence of the original malicious package.

These typos are often cleverly disguised and are very easy to overlook when developers are trying to quickly incorporate a component into their project on tight deadlines. Watch out for these — they can be hard to sp0t. For many organizations, the most immediate means to address this is proper developer education and security awareness training to engender caution when selecting libraries and packages for inclusion into projects.

Dependency confusion

This is an interesting approach to infiltrating the software supply chain that came into vogue this year with Alex Birsan’s research published in February. Birsan attempted to find private library names (like in Javascript files or GitHub code, for example) and then create his own public libraries matching those private library names.

Basically, from this research, it’s been proven that automated build processes would often pull the public libraries from places like npm and PyPI instead of private repositories. This is a method of exploiting the automation practices and hands-off approach to CI/CD that many DevOps professionals and developers are baking into their pipelines. Issues like this can go undetected indefinitely, often to be discovered only after a data breach or a surprisingly high bill from your cloud provider for your inadvertent resource overconsumption for someone else’s crypto-mining operation.

Despite the heightened awareness of this tactic, many organizations tend to rely on “proper” configuration of CI/CD tools and a series of “checks and balances” to verify intended outcomes. As you might expect, success varies and depends on the technical capacity and security proficiency of the people managing the tools.

Exploiting trust

I’ve defined this next attack type under the broader concept of exploiting trust, which manifests itself in a lot of different ways. We’ve seen many examples of popular libraries being altered to include malware, perhaps when a malicious developer volunteered to take over the project or when a project is maintained by a community lacking in security awareness. We also have instances in which malicious actors use bots to artificially inflate the ratings and stats of malicious packages to entice or encourage developers to trust the library as they search for components to pull into their projects.

Finally, we’ve seen many highly publicized examples of compromised developer tools. So, while trust was not initially misplaced, the end result of being exploited or infected mimics the result that would have occurred if a malicious software package were adopted deliberately. One good example of this was disclosed in April, when Codecov announced that someone had used an error in Codecov’s Docker image creation process to extract credentials and modify their Bash Uploader script. The attackers then used this modified script to steal credentials from the CI environments of customers using it.

This last category is often the hardest for organizations to overcome, since no amount of developer security education or tool configuration can account for a third party’s deviation from their “trusted” status. Other methods to remove trust from the equation often result in more potential points of failure or delays in the flow from development to deployment. One thing is certain, however: You do not want your organization’s security posture to depend on the security proficiency of a third party.

Managing software supply chain security risks

In most instances, security risks in the software supply chain only manifest themselves in runtime when containers execute actions, network communications occur, and so on. As I mentioned earlier, traditional software security testing – such as static analysis (SAST), dynamic analysis (DAST), and software composition analysis (SCA) – won’t detect these risks. Malware and other supply chain risks are not vulnerabilities, although the artifacts you ingest through your supply chain may very well include them.

So, the most reliable method of detecting such malicious packages and anomalous activity is while the application and containers are in a running state.

Of course, you don’t want the first time you run the application to be in a production environment. I recommend establishing automated methods of spinning up applications in a secure testing sandbox as part of the CI/CD process, prior to promotion into production. When you do this, it’s important to analyze container activity, network communications, and other actions in order to detect any potential attack that may take place. I’ll discuss these best practices for managing software supply chain security risks more in depth in a future blog.

Until then, take a moment to check out this short video overview of Aqua Dynamic Threat Analysis (DTA), which provides a reliable and secure way to fully analyze running software in a secure pre-production environment, document the full attack kill chain, and establish policies to block promotion of artifacts that fail a DTA scan as part of the automated CI/CD process.