Building modern-day software entails using third-party packages and libraries to build fast by reusing existing work. This reuse can and will however come with risks. By using third-party software, that in itself likely depends on other libraries might bury risks i.e. vulnerabilities deep into your tech stack. Incidents, such as log4shell or more recently text4shell have shown that these risks are excruciatingly real. In fact so real (in the case of log4shell) that even until today more than 25% of the total downloads of log4j packages are vulnerable (see Sonatype's log4j monitor, checked 23.11.2022).
The sketch below poignantly summarizes this phenomenon:
While security tools such as synk or sonatype are powerful in spotting these vulnerabilities, we still lack a common language to describe and understand all the dependencies that go into any piece of software we build. Or as Kelsey Hightower puts it: "When cooking for your friends you want to make sure you know exactly what is going into the meal [i.e. ingredients], so you don't cook something awfully bad or intoxicating" (see his talk on this here)
This is exactly what a unified Software Bill of Materials (SBOM) targets. There is also early evidence that generating theses SBOMs, as part of your build documentation helps to improve your software supply chain security assurances (refer to the 2022 State of DevOps report by Google). Trending security frameworks, such as SLSA consider the SBOM as one of the core elements of the provenance pillar within the bigger scheme of improving the integrity of your software artifacts.
Let's now turn to specific SBOM standards and how the open-source tools to create these SBOMs stack up against each other.
There are two main industry standards for SBOMs:
- Software Package Data Exchange (SPDX and SPDX Lite), an ISO standard hosted by the Linux Foundation, which outlines the components, licenses, and copyrights associated with a software package.
- CycloneDX, an open source, lightweight SBOM standard, which is used in application security and supply chain analysis and originated from the Open Web Application Security Project (OWASP).
Our final aim is to generate SBOM data to publish to LeanIX VSM. Since LeanIX VSM accepts the CycloneDX specification, we are looking in the next section at the CycloneDX specification and the most common plugins to generate the SBOM. The below assessment is against the LeanIX VSM product requirements, which are:
- Consistent use of PURLs
- Existence of licenses per library
- Existence of transitive dependencies
CycloneDX specification is a lightweight Software Bill of Materials (SBOM) standard designed for use in application security contexts and supply chain component analysis. It’s an Open source initiative to streamline the way an SBOM (software bill of materials) is represented universally.
- CycloneDX allows for an easy standard to streamline library representation by relying on another standard: package-url or short PURL.
- CycloneDX is extensible to capture other information as well (APIs, services, vulnerabilities …)
- Part of a larger and active open-source community backed by many big and known tech companies
- Allowing interoperability with adjacent tools such as vulnerability databases
Our target environments are java(gradle) and nodejs. There are different ways in which the SBOM generation can be automated in your CI/CD. Also, there are quite a few plugins available to generate the SBOM. In this article, let's compare 3 toolsets:
We've identified the CycloneDX native toolset to be most fitting and accurate for our use cases. See more details in the upcoming sections
|CycloneDX Gradle Plugin (v1.7.2)||CycloneDX NPM Plugin||Syft Container Plugin||Syft File Plugin||Trivy Container Plugin|
|Transitive dependencies||✅ (See comparative study with Gradle dependency graph)||✅||❌||❌||✅ (Not accurate)|
|Pros||- Better data quality|
- Matches expected PURL
|- Better data quality|
- Matches expected PURL
- Easier to setup
|- Easy to setup||- Easy to setup|
|Cons||- Setting up requires code change||- Poor data quality|
- No transitive dependencies
|- Poor data quality|
- No transitive dependencies
|- Poor data quality|
- Inaccurate results with different plugins within the same toolset
- Documentation is tricky and not well maintained
A package URL (purl) is an attempt to standardize existing approaches to reliably identify and locate software packages. A purl is a URL string used to identify and locate a software package in a mostly universal and uniform way across programming languages, package managers, packaging conventions, tools, APIs, and databases. Such a package URL is useful to reliably reference the same software package using a simple and expressive syntax and conventions based on familiar URLs.
A purl is a URL composed of seven components:
Result: Syft does not extract the correct PURL string. Notice the group information. CycloneDX extracts the correct data which is
In the following section, let's deep dive into understanding the transitive dependencies sections of extracted SBOM files. The dependencies are expected in CycloneDX specification. Let’s compare the gradle plugins against the ground truth, gradle dependency tree for specific library.
+--- com.azure.spring:azure-spring-cloud-starter-eventhubs-kafka:2.14.0 | +--- com.azure.spring:azure-spring-cloud-autoconfigure:2.14.0 .... | +--- org.springframework.cloud:spring-cloud-starter-stream-kafka:3.2.1 -> 3.2.3 .... | \--- com.azure.spring:azure-spring-cloud-context:2.14.0 | ....
Result: CycloneDX correctly extracts the transitive dependencies but Syft and Trivy does not have the information for this library ☑️
Similarly, let's check for few other libraries to confirm the accuracy of CycloneDX Gradle plugin.
+--- org.springframework.boot:spring-boot-starter-data-jpa -> 2.6.7 | +--- org.springframework.boot:spring-boot-starter-aop:2.6.7 .... | +--- org.springframework.boot:spring-boot-starter-jdbc:2.6.7 | +--- jakarta.transaction:jakarta.transaction-api:1.3.3 | +--- jakarta.persistence:jakarta.persistence-api:2.2.3 | +--- org.hibernate:hibernate-core:5.6.8.Final | +--- org.springframework.data:spring-data-jpa:2.6.4 | \--- org.springframework:spring-aspects:5.3.19 ...
Result: CycloneDX correctly extracts the transitive dependencies ☑️
+--- org.springframework.boot:spring-boot-starter-actuator -> 2.6.7 | +--- org.springframework.boot:spring-boot-starter:2.6.7 | | +--- org.springframework.boot:spring-boot:2.6.7 | | | +--- org.springframework:spring-core:5.3.19 | | | | \--- org.springframework:spring-jcl:5.3.19 | | | \--- org.springframework:spring-context:5.3.19 | | | +--- org.springframework:spring-aop:5.3.19 | | | | +--- org.springframework:spring-beans:5.3.19 | | | | | \--- org.springframework:spring-core:5.3.19 (*) | | | | \--- org.springframework:spring-core:5.3.19 (*) | | | +--- org.springframework:spring-beans:5.3.19 (*) | | | +--- org.springframework:spring-core:5.3.19 (*) | | | \--- org.springframework:spring-expression:5.3.19 | | | \--- org.springframework:spring-core:5.3.19 (*) | | +--- org.springframework.boot:spring-boot-autoconfigure:2.6.7 | | | \--- org.springframework.boot:spring-boot:2.6.7 (*) | | +--- org.springframework.boot:spring-boot-starter-logging:2.6.7 | | | +--- ch.qos.logback:logback-classic:1.2.11 | | | | +--- ch.qos.logback:logback-core:1.2.11 | | | | \--- org.slf4j:slf4j-api:1.7.32 -> 1.7.36 | | | +--- org.apache.logging.log4j:log4j-to-slf4j:2.17.2 | | | | +--- org.slf4j:slf4j-api:1.7.35 -> 1.7.36 | | | | \--- org.apache.logging.log4j:log4j-api:2.17.2 | | | \--- org.slf4j:jul-to-slf4j:1.7.36 | | | \--- org.slf4j:slf4j-api:1.7.36 | | +--- jakarta.annotation:jakarta.annotation-api:1.3.5 | | +--- org.springframework:spring-core:5.3.19 (*) | | \--- org.yaml:snakeyaml:1.29 | +--- org.springframework.boot:spring-boot-actuator-autoconfigure:2.6.7 | | +--- org.springframework.boot:spring-boot-actuator:2.6.7 | | | \--- org.springframework.boot:spring-boot:2.6.7 (*) | | +--- org.springframework.boot:spring-boot:2.6.7 (*) | | \--- org.springframework.boot:spring-boot-autoconfigure:2.6.7 (*) | \--- io.micrometer:micrometer-core:1.8.5 | \--- org.hdrhistogram:HdrHistogram:2.1.12
Result: CycloneDX correctly extracts the transitive dependencies though the information is not exact it is acceptable and reliable ☑️
Few more libraries are also checked and confirmed to have exact match with Gradle dependency tree. Similarly, transitive dependencies are also checked for NPM libraries and results are the same.
Documenting, understanding and managing the integrity of your builds is increasingly essential for a sound security posture in regards to your software supply chain. This is also imperative to reliably build & provide secure digital products in the 21st century.
The SBOM (Software Bill of Materials) is one key pillar in solving aforementioned challenge. With CycloneDX as an industry-standard driven by the community there is also a clear standard and open-source tooling to easily create those. In this blog we looked at how different open-source tools compare in regards to consistently providing PURLs, transitive dependencies and licenses to power LeanIX VSM to give a holistic and actionable insight into your technology stack.
Our results show that for the LeanIX VSM use case the native CycloneDX plugins provide the most reliable output. Yet, they are also subject to minor inconsistencies, which are mostly already known to the community. The other tools clearly have their merit for other use cases, but fail on the criteria we expect for a reliable visibility into the dependencies of our services here at LeanIX. We are aware that the tools capabilities might change over time. Therefore this analysis can only be considered as snapshot in time.
We recommend the CycloneDX "native" plugins in combined use with LeanIX VSM. See our user documentation here.
Once uploaded into LeanIX VSM, we are getting a nice & clean overview into the dependencies for the services we are developing.
- Cyclone DX specification object
- State of DevOps Google
- How LeanIX dealt with the Log4j incident
- How to use LeanIX VSM to analyze incidents like log4j
- NTIA's Guidance on SBOM Production