[New Tool] Add the OSS Review Toolkit (ORT)

[sschuberth]

Tool name and link (must be open source)

OSS Review Toolkit (ORT), https://github.com/oss-review-toolkit/ort

Supported specifications

• CycloneDX
• SPDX

Supported target

• Container images
• Source code repositories
• Packages

Environment setup (if any)

Depends on the package managers to analyze; see the output of ort requirements.

Recommended command to see version

Run ort--version. The version currently is the short Git SHA1 of the source code from which ORT was built. We'll switch to Semver eventually.

Recommended command(s) to build SBOM

Creating an SBOM is a two stage process: First the project source code is analyzed, and then the analyzer result is converted to an SBOM via the reporter tool.

We recommend to run ORT from a built Docker image like:

$ ort analyze -i /project -o /project/ort/analyzer
$ ort report -f cyclonedx,spdx -i /project/ort/analyzer/analyzer-result.yml -o /project/ort/reporter

The resulting CycloneDX and SPDX SBOMs will reside in the /project/ort/reporter directory afterwards.

Additional context

Only supports SPDX 2.x JSON / YAML. CycloneDX is currently limited to 1.4.

I'm the founder and maintainer of ORT. Please let me know if you need more details to get it running.

[surendrapathak]

Hi @sschuberth Thank you for reporting this. ORT SBOM has been added and analyzed for a few repositories and you can find the entries on

1. Homepage: https://sbombenchmark.dev/
2. Tool page: https://sbombenchmark.dev/tools/ort
3. Sample SPDX: https://sbombenchmark.dev/score/ort-14911bfa77_syncthing-v1.13.6.spdx.yml
4. Sample CDX: https://sbombenchmark.dev/score/ort-14911bfa77_lodash-v4.0.0.cdx.xml

SBOMBenchmark uses the following checks of sbomqs to evaluate SBOMs.

The current set of ORT SBOM limitations seems clustered to - license and unique (CPE/PURL) identifiers.
Let me know if you have any questions and we are also open to any suggestions for improving these.

[sschuberth]

Thanks @surendrapathak! I'll have a look as soon as I'm back from vacation.

[sschuberth]

The current set of ORT SBOM limitations seems clustered to - license and unique (CPE/PURL) identifiers.

While I'm not certain what exactly you mean with this, please be aware that some of the SBOM fields, like licenses, are only filled by ORT if either a) the original package metadata does declare any licenses, or b) the scan command was run after the analyze command. As the scan command really looks at the source code / files, this will also provide additional information like checksums etc. So for a more complete comparison, you might want to run the scancommand, too, taking the analyzer result as the input:

ort analyze -i /project -o /results
ort scan -i /results/analyzer-result.yml -o /results
ort report -f CycloneDx,SpdxDocument -i /results/scan-result.yml -o /results

However, running scan also requires a supported scanner to be installed. By default, ScanCode is being used.

[sschuberth]

A remark on the scoring: I realized that this shows 0.0 for SBOM includes creator tool and creator tool's version although the SPDX document contains

creationInfo:
  created: "2023-08-01T07:52:35Z"
  creators:
  - "Tool: OSS Review Toolkit - 14911bfa77"
  licenseListVersion: "3.20"

What else is expected by sbomqs?

[surendrapathak]

Hi @sschuberth I'll check out the scan report before the end of the week and will update you with my findings.

For the Creator Tool name, the implementation relies on the SPDX Creator description from the SPDX value here that implies one word with the name of the tool and version separated by '-'. There is no official schema for SPDX yet; therefore, it needed to be clarified if the identifier name can or cannot have spaces.

Interlynk's implementation for this rule is here: https://github.com/interlynk-io/sbomqs/blob/main/pkg/sbom/spdx.go#L219C10-L219C10

Given, the included SBOM is validated with pyspdxtools as well, we'll take up on loosening the space rule above to make sure it can accommodate strings like 'ORT Review Toolkit'. Here is the issue # for your reference: interlynk-io/sbomqs#180

I can re-run the ORT once I am able to get the results from scan step by the end of the week.

[sschuberth]

I can re-run the ORT once I am able to get the results from scan step by the end of the week.

Great! You may want to ensure that these PRs in ORT are merged before that, and to use the latest ORT version then:

• Add CycloneDX metadata oss-review-toolkit/ort#7405
• SPDX creator tool adjustments oss-review-toolkit/ort#7410

[sschuberth]

I can re-run the ORT once I am able to get the results from scan step by the end of the week.

Also note that scanning might take long time, as ORT scans the full source code of all transitive dependencies over and over again by default unless you set up a shared scan result storage.

So maybe it's not a good idea to set up scanning after all, but then the scoring of license entries in SBOMs needs to be adjusted so that licenses are only expected if they're actually declared in the original package metadata. Otherwise we're running into a "garbage in, garbage out" problem 😉

[surendrapathak]

Hi @sschuberth, I am just checking if you know of any issues building the ORT image on Mac. I am getting "/opt/haskell": not found in the base image but I haven't spotted any key change since the last build.

[sschuberth]

Hi @sschuberth, I am just checking if you know of any issues building the ORT image on Mac.

I'm not on a Mac, and I haven't seen this before. @mbochtler have you maybe?

I am getting "/opt/haskell": not found in the base image but I haven't spotted any key change since the last build.

Indeed we didn't modify the Dockerfile in ORT since this issue was opened initially.

[surendrapathak]

Hi @mbochtler, Any feedback on this? If the image is available elsewhere (DockerHub / GHCR), I am happy to pull and directly re-run it to review the results. Let me know. Thanks.

[sschuberth]

If the image is available elsewhere (DockerHub / GHCR)

It's actually not yet...

[surendrapathak]

I am still blocked for the SBBM upgrade. However, I was able to confirm that Tools parsing is looking good with manually

cli/build/install/ort/bin/ort --version
023c7b06de

I'll give it another try after the long weekend.

[sschuberth]

However, I was able to confirm that Tools parsing is looking good

Thanks for confirming!

I'll give it another try after the long weekend.

You may want to try with Dockerfile-legacy instead for the time being...

[surendrapathak]

Hi @sschuberth , The magic of killing the entire repo and rebuilding from scratch helped the docker image succeed. I was able to find these additional improvement opportunities:

1. Relationships CycloneDX XML includes any relationship data (see attached SBOM). Relationships are one of the requirements for meeting NTIA minimum elements requirements.
2. Unique ID One side-effect of missing relationships is missing out on bom-ref element in the component. A doc-wide unique bom-ref (typically used by CPE or PURL) ensures relationships are uniquely expressed.
3. Supplier NTIA minimum elements calls out for Supplier information (component:supplier) but still not many SBOM generators have a reliable method of providing supplier for open source components.

[sschuberth]

Thank you @surendrapathak for your feedback!

CycloneDX XML includes any relationship data (see attached SBOM). Relationships are one of the requirements for meeting NTIA minimum elements requirements.

Thanks a lot for pointing this out. I believe that maps to this issue in ORT.

NTIA minimum elements calls out for Supplier information (component:supplier) but still not many SBOM generators have a reliable method of providing supplier for open source components.

While it's trivial to write out the supplier field per se, as you're pointing out yourselves the difficulty lies in a common understanding about what the "supplier" of an Open Source package should be. I'll bring this up in the ORT community; might be that we'd require some small model changes to address this appropriately.

[surendrapathak]

@sschuberth Happy to help. Just FYI - In the last couple of months, CISA group working on the community side of SBOM recommendations did consider the Supplier question for the Open Source.
This is still in DRAFT mode but might be worth a look: https://docs.google.com/document/d/13ZLHbFwR83JI6QDvyAyVtLo4hLgXdXGnyG4MT_fV16w/edit#heading=h.us3axf2rbthk

Supplier Name
Was the component modified while in your possession?

• Yes - list the component with your organization in the supplier name field. Additionally, list the upstream supplier in the Pedigree field for CycloneDX format or Originator for SPDX format and include a dependency to upstream component

• No - list the component with your supplier’s name as indicated for commercial or open-source entities

  • Commercial source: list the commercial supplier’s “legal entity” name
  • Open-source: at a minimum, list project name. Consider listing the host foundation before the project name (Ex: Apache Tomcat)

[sschuberth]

Thank again for your input, @surendrapathak! I've captured the supplier-topic on the ORT side at oss-review-toolkit/ort#7449. As ORT has been added as a tool to your service successfully, let's close this! Thanks again.