A tool overview for generating CycloneDX SBOM

Motivation

Building modern-day software entails using third-party packages and libraries to build fast by reusing existing work. This reuse can and will however come with risks. By using third-party software, that in itself likely depends on other libraries might bury risks i.e. vulnerabilities deep into your tech stack. Incidents, such as log4shell or more recently text4shell have shown that these risks are excruciatingly real. In fact so real (in the case of log4shell) that even until today more than 25% of the total downloads of log4j packages are vulnerable (see Sonatype's log4j monitor, checked 23.11.2022).
The sketch below poignantly summarizes this phenomenon:

While security tools such as synk or sonatype are powerful in spotting these vulnerabilities, we still lack a common language to describe and understand all the dependencies that go into any piece of software we build. Or as Kelsey Hightower puts it: "When cooking for your friends you want to make sure you know exactly what is going into the meal [i.e. ingredients], so you don't cook something awfully bad or intoxicating" (see his talk on this here)

This is exactly what a unified Software Bill of Materials (SBOM) targets. There is also early evidence that generating theses SBOMs, as part of your build documentation helps to improve your software supply chain security assurances (refer to the 2022 State of DevOps report by Google). Trending security frameworks, such as SLSA consider the SBOM as one of the core elements of the provenance pillar within the bigger scheme of improving the integrity of your software artifacts.

Let's now turn to specific SBOM standards and how the open-source tools to create these SBOMs stack up against each other.

There are two main industry standards for SBOMs:

Software Package Data Exchange (SPDX and SPDX Lite), an ISO standard hosted by the Linux Foundation, which outlines the components, licenses, and copyrights associated with a software package.
CycloneDX, an open source, lightweight SBOM standard, which is used in application security and supply chain analysis and originated from the Open Web Application Security Project (OWASP).

Our final aim is to generate SBOM data to publish to LeanIX VSM. Since LeanIX VSM accepts the CycloneDX specification, we are looking in the next section at the CycloneDX specification and the most common plugins to generate the SBOM. The below assessment is against the LeanIX VSM product requirements, which are:

Consistent use of PURLs
Existence of licenses per library
Existence of transitive dependencies

SBOM Representation

CycloneDX specification is a lightweight Software Bill of Materials (SBOM) standard designed for use in application security contexts and supply chain component analysis. It’s an Open source initiative to streamline the way an SBOM (software bill of materials) is represented universally.

Benefits of piggybacking on a CycloneDX rather than re-inventing the wheel:

CycloneDX allows for an easy standard to streamline library representation by relying on another standard: package-url or short PURL.
CycloneDX is extensible to capture other information as well (APIs, services, vulnerabilities …)
Part of a larger and active open-source community backed by many big and known tech companies
Allowing interoperability with adjacent tools such as vulnerability databases

Comparison

Our target environments are java(gradle) and nodejs. There are different ways in which the SBOM generation can be automated in your CI/CD. Also, there are quite a few plugins available to generate the SBOM. In this article, let's compare 3 toolsets:

CycloneDX toolset
Syft toolset
Trivy toolset

TL;DR
We've identified the CycloneDX native toolset to be most fitting and accurate for our use cases. See more details in the upcoming sections

	CycloneDX Gradle Plugin (v1.7.2)	CycloneDX NPM Plugin	Syft Container Plugin	Syft File Plugin	Trivy Container Plugin
Transitive dependencies	✅ (See comparative study with Gradle dependency graph)	✅	❌	❌	✅ (Not accurate)
Licenses information	✅	✅	✅	❌	✅
Pros	- Better data quality - Matches expected PURL	- Better data quality - Matches expected PURL - Easier to setup	- Easy to setup	- Easy to setup
Cons	- Setting up requires code change		- Poor data quality - No transitive dependencies	- Poor data quality - No transitive dependencies	- Poor data quality - Inaccurate results with different plugins within the same toolset - Documentation is tricky and not well maintained

Taking a closer look at the Package URLs (PURL)

A package URL (purl) is an attempt to standardize existing approaches to reliably identify and locate software packages. A purl is a URL string used to identify and locate a software package in a mostly universal and uniform way across programming languages, package managers, packaging conventions, tools, APIs, and databases. Such a package URL is useful to reliably reference the same software package using a simple and expressive syntax and conventions based on familiar URLs.

A purl is a URL composed of seven components:
scheme:type/namespace/name@version?qualifiers#subpath

CycloneDX

pkg:maven/org.springframework/spring-web@5.3.19?type=jar

Syft

pkg:maven/spring-web/spring-web@5.3.19

Result: Syft does not extract the correct PURL string. Notice the group information. CycloneDX extracts the correct data which is org.springframework

In-Depth comparison

In the following section, let's deep dive into understanding the transitive dependencies sections of extracted SBOM files. The dependencies are expected in CycloneDX specification. Let’s compare the gradle plugins against the ground truth, gradle dependency tree for specific library.

Library #1: `azure-spring-cloud-starter-eventhubs-kafka`

CycloneDX Gradle Plugin

{
  "ref": "pkg:maven/com.azure.spring/azure-spring-cloud-starter-eventhubs-kafka@2.14.0?type=jar",
  "dependsOn": [
    "pkg:maven/org.springframework.cloud/spring-cloud-starter-stream-kafka@3.2.3?type=jar",
    "pkg:maven/com.azure.spring/azure-spring-cloud-context@2.14.0?type=jar",
    "pkg:maven/com.azure.spring/azure-spring-cloud-autoconfigure@2.14.0?type=jar"
  ]
}

Syft

No information

Trivy

No information

Gradle dependency tree

+--- com.azure.spring:azure-spring-cloud-starter-eventhubs-kafka:2.14.0
|    +--- com.azure.spring:azure-spring-cloud-autoconfigure:2.14.0
            ....
|    +--- org.springframework.cloud:spring-cloud-starter-stream-kafka:3.2.1 -> 3.2.3
            ....
|    \--- com.azure.spring:azure-spring-cloud-context:2.14.0
|           ....

Result: CycloneDX correctly extracts the transitive dependencies but Syft and Trivy does not have the information for this library ☑️

Similarly, let's check for few other libraries to confirm the accuracy of CycloneDX Gradle plugin.

Library #2: `spring-boot-starter-data-jpa`

CycloneDX

{
  "ref": "pkg:maven/org.springframework.boot/spring-boot-starter-data-jpa@2.6.7?type=jar",
  "dependsOn": [
    "pkg:maven/org.springframework.data/spring-data-jpa@2.6.4?type=jar",
    "pkg:maven/org.springframework.boot/spring-boot-starter-jdbc@2.6.7?type=jar",
    "pkg:maven/org.springframework.boot/spring-boot-starter-aop@2.6.7?type=jar",
    "pkg:maven/jakarta.transaction/jakarta.transaction-api@1.3.3?type=jar",
    "pkg:maven/jakarta.persistence/jakarta.persistence-api@2.2.3?type=jar",
    "pkg:maven/org.hibernate/hibernate-core@5.6.8.Final?type=jar",
    "pkg:maven/org.springframework/spring-aspects@5.3.19?type=jar"
  ]
}

Gradle dependency tree

+--- org.springframework.boot:spring-boot-starter-data-jpa -> 2.6.7
|    +--- org.springframework.boot:spring-boot-starter-aop:2.6.7
    ....
|    +--- org.springframework.boot:spring-boot-starter-jdbc:2.6.7
|    +--- jakarta.transaction:jakarta.transaction-api:1.3.3
|    +--- jakarta.persistence:jakarta.persistence-api:2.2.3
|    +--- org.hibernate:hibernate-core:5.6.8.Final
|    +--- org.springframework.data:spring-data-jpa:2.6.4
|    \--- org.springframework:spring-aspects:5.3.19
...

Result: CycloneDX correctly extracts the transitive dependencies ☑️

Library #3: `spring-boot-actuator`

CycloneDX

{
  "ref": "pkg:maven/org.springframework.boot/spring-boot-actuator@2.6.7?type=jar",
  "dependsOn": [
    "pkg:maven/org.springframework.boot/spring-boot@2.6.7?type=jar"
  ]
}

Gradle dependency tree

+--- org.springframework.boot:spring-boot-starter-actuator -> 2.6.7
|    +--- org.springframework.boot:spring-boot-starter:2.6.7
|    |    +--- org.springframework.boot:spring-boot:2.6.7
|    |    |    +--- org.springframework:spring-core:5.3.19
|    |    |    |    \--- org.springframework:spring-jcl:5.3.19
|    |    |    \--- org.springframework:spring-context:5.3.19
|    |    |         +--- org.springframework:spring-aop:5.3.19
|    |    |         |    +--- org.springframework:spring-beans:5.3.19
|    |    |         |    |    \--- org.springframework:spring-core:5.3.19 (*)
|    |    |         |    \--- org.springframework:spring-core:5.3.19 (*)
|    |    |         +--- org.springframework:spring-beans:5.3.19 (*)
|    |    |         +--- org.springframework:spring-core:5.3.19 (*)
|    |    |         \--- org.springframework:spring-expression:5.3.19
|    |    |              \--- org.springframework:spring-core:5.3.19 (*)
|    |    +--- org.springframework.boot:spring-boot-autoconfigure:2.6.7
|    |    |    \--- org.springframework.boot:spring-boot:2.6.7 (*)
|    |    +--- org.springframework.boot:spring-boot-starter-logging:2.6.7
|    |    |    +--- ch.qos.logback:logback-classic:1.2.11
|    |    |    |    +--- ch.qos.logback:logback-core:1.2.11
|    |    |    |    \--- org.slf4j:slf4j-api:1.7.32 -> 1.7.36
|    |    |    +--- org.apache.logging.log4j:log4j-to-slf4j:2.17.2
|    |    |    |    +--- org.slf4j:slf4j-api:1.7.35 -> 1.7.36
|    |    |    |    \--- org.apache.logging.log4j:log4j-api:2.17.2
|    |    |    \--- org.slf4j:jul-to-slf4j:1.7.36
|    |    |         \--- org.slf4j:slf4j-api:1.7.36
|    |    +--- jakarta.annotation:jakarta.annotation-api:1.3.5
|    |    +--- org.springframework:spring-core:5.3.19 (*)
|    |    \--- org.yaml:snakeyaml:1.29
|    +--- org.springframework.boot:spring-boot-actuator-autoconfigure:2.6.7
|    |    +--- org.springframework.boot:spring-boot-actuator:2.6.7
|    |    |    \--- org.springframework.boot:spring-boot:2.6.7 (*)
|    |    +--- org.springframework.boot:spring-boot:2.6.7 (*)
|    |    \--- org.springframework.boot:spring-boot-autoconfigure:2.6.7 (*)
|    \--- io.micrometer:micrometer-core:1.8.5
|         \--- org.hdrhistogram:HdrHistogram:2.1.12

Result: CycloneDX correctly extracts the transitive dependencies though the information is not exact it is acceptable and reliable ☑️

Few more libraries are also checked and confirmed to have exact match with Gradle dependency tree. Similarly, transitive dependencies are also checked for NPM libraries and results are the same.

Conclusion

Documenting, understanding and managing the integrity of your builds is increasingly essential for a sound security posture in regards to your software supply chain. This is also imperative to reliably build & provide secure digital products in the 21st century.

The SBOM (Software Bill of Materials) is one key pillar in solving aforementioned challenge. With CycloneDX as an industry-standard driven by the community there is also a clear standard and open-source tooling to easily create those. In this blog we looked at how different open-source tools compare in regards to consistently providing PURLs, transitive dependencies and licenses to power LeanIX VSM to give a holistic and actionable insight into your technology stack.

Our results show that for the LeanIX VSM use case the native CycloneDX plugins provide the most reliable output. Yet, they are also subject to minor inconsistencies, which are mostly already known to the community. The other tools clearly have their merit for other use cases, but fail on the criteria we expect for a reliable visibility into the dependencies of our services here at LeanIX. We are aware that the tools capabilities might change over time. Therefore this analysis can only be considered as snapshot in time.

TL;DR
We recommend the CycloneDX "native" plugins in combined use with LeanIX VSM. See our user documentation here.

Once uploaded into LeanIX VSM, we are getting a nice & clean overview into the dependencies for the services we are developing.

References

Published by...

Aravind Metku

Visit author page

Vincent Groves

Visit author page