A Software Bill of Materials (SBOM) is a document that describes the components that make up a specific software artifact. The most common use for SBOMs is as a catalogue of a project’s dependencies from which you can run vulnerability scans and identify vulnerable components that your software depends on.
This is great and there are tons of resources and tools out there that can help you to do this. However, those of us on the Jetstack Consult team have become increasingly interested in how we could expand on this and leverage SBOMs to gain useful insights beyond the typical case of vulnerability analysis.
One particularly interesting project in this space is the OpenSSF Scorecard project, which scores a project’s security profile based on a number of different factors.
This includes checks for things like:
- Recent commits to the project
- Adequate branch protection rules
- The presence of dependency update tooling
These sorts of checks provide useful heuristics for understanding the security posture of an open source project.
When our experts are your experts, you can make the most of Kubernetes
It occurred to us that being able to find the Scorecard score for each of the components listed in an SBOM would be a valuable way to assess the risk that your software’s dependencies may pose. That’s why we built tally, a command line tool that allows you to quickly collate a list of Scorecard scores for the packages described in your SBOMs.
$ tally bom.json
REPOSITORY SCORE
github.com/azure/go-autorest 8.0
github.com/azure/azure-sdk-for-go 7.8
github.com/npm/read-package-json 7.7
github.com/npm/package-json 7.7
github.com/npm/node-semver 7.7
github.com/npm/ini 7.7
...How it Works
Mapping Packages to Repositories
Because scores are associated with GitHub projects, the first step is to find the GitHub repositories for the packages in your SBOM.
Most SBOM specs have a way of encoding the source code repository as a property of a component:
- CycloneDX has the ‘vcs’
External Referencestype - Syft exposes package metadata as properties in the SBOM. In a lot of cases, this includes source control information.
- Also in some cases, like with a lot of Go packages, you can infer the GitHub repository from the name of the package.
We combine these methods in tally to find the repositories behind the packages.
Finding Scores
Once we’ve mapped our packages to their repositories, the next step is to find the scores.
To do this, tally reaches out to the Scorecard API to get results from each repository it discovered in the BOM.
Once it has the scores, tally has everything it needs to output the results.
$ tally bom.json
REPOSITORY SCORE
github.com/azure/go-autorest 8.0
github.com/azure/azure-sdk-for-go 7.8
github.com/npm/read-package-json 7.7
github.com/npm/package-json 7.7
github.com/npm/node-semver 7.7
github.com/npm/ini 7.7We can also print the package information with the wide output.
$ tally bom.json -o wide
TYPE PACKAGE REPOSITORY SCORE
golang github.com/azure/go-autorest/autorest github.com/azure/go-autorest 8.0
golang github.com/azure/go-autorest/autorest/adal github.com/azure/go-autorest 8.0
golang github.com/azure/go-autorest/autorest/date github.com/azure/go-autorest 8.0
golang github.com/azure/go-autorest/autorest/to github.com/azure/go-autorest 8.0
golang github.com/azure/go-autorest/autorest/validation github.com/azure/go-autorest 8.0
golang github.com/azure/go-autorest/logger github.com/azure/go-autorest 8.0
golang github.com/azure/go-autorest/tracing github.com/azure/go-autorest 8.0
golang github.com/azure/azure-sdk-for-go github.com/azure/azure-sdk-for-go 7.8
npm read-package-json github.com/npm/read-package-json 7.7
npm @npmcli/package-json github.com/npm/package-json 7.7Or show the full results, including individual check scores with the json output.
$ tally bom.json -o json | jq -r .
{
"results": [
{
"repository": {
"name": "github.com/azure/go-autorest"
},
"packages": [
{
"type": "golang",
"name": "github.com/azure/go-autorest/autorest"
},
{
"type": "golang",
"name": "github.com/azure/go-autorest/autorest/adal"
}
],
"result": {
"date": "2022-08-15",
"repo": {
"name": "github.com/azure/go-autorest",
"commit": "2fa44cb18b8338d7fa4f749bb798d6cbb3d9ba0c"
},
"scorecard": {
"version": "v4.5.0-17-g7772984",
"commit": "777298477c07c262a4ec7e95ceee839b7b3b75ae"
},
"score": 8,
"checks": [
{
"name": "Maintained",
"score": 10,
"reason": "5 commit(s) out of 30 and 7 issue activity out of 30 found in the last 90 days -- score normalized to 10",
"details": null,
"documentation": {
"short": "Determines if the project is \"actively maintained\".",
"url": "https://github.com/ossf/scorecard/blob/777298477c07c262a4ec7e95ceee839b7b3b75ae/docs/checks.md#maintained"
}
}
]
}
}
]
}Generating Scores
Not all repositories have a score in the public API. To plug this gap, tally optionally supports generating missing scores itself when the -g/--generate flag is set.
This requires a valid token for GitHub in the GITHUB_TOKEN environment variable.
$ export GITHUB_TOKEN=XXXXX
$ tally --generate bom.jsonDepending on the number of packages, this can take quite a long time, so tally caches each score it finds to speed up subsequent runs.
If you want to generate all the scores yourself, you can disable fetching scores from the API by setting --api=false.
$ tally --api=false --generate bom.jsonLimitations
There are some limitations to tally that you should keep in mind.
It Only Supports GitHub Repositories
As Scorecard only supports GitHub repositories, these are the only repositories that tally will currently discover.
It Doesn’t Look for Historic Scorecard Scores
It will only find the most recent scorecard result for each repository. It will not attempt to find the scorecard result for the commit that a particular package version was built from.
This may not be a problem, depending on how you want to interpret the score. You can use the most recent score to provide an indication of the general risk associated with continuing to depend on a particular project. You should be making sure dependencies are kept up to date regardless.
That being said, there is definite value in understanding the scorecard score for the given commit that produced a particular version of a package. We may look at how we could support this in future.
We Can’t Find Repositories for All Packages
Currently, tally will only infer repository information from the contents of the SBOM. This information isn’t always available for every package.
Not All Projects Have a Scorecard Result
It goes without saying that not all packages are version controlled in GitHub and therefore, for some packages, it’s impossible to produce a scorecard result.
Get in Touch
I’d love to hear of anyone using tally or any suggestions for improvements or features.
If your organization is looking to take its first steps toward improving its supply chain security, check out our Software Supply Chain Security Toolkit.
Or, get in touch to see how Jetstack Consult could help.