Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add license reporting and "vcpkg license-report" command. #1514

Draft
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

BillyONeal
Copy link
Member

@BillyONeal BillyONeal commented Oct 17, 2024

This work is primarily at the request of @walbourn , related: microsoft/vcpkg#31770

This PR adds a new command to help the the XBox customers @walbourn was trying to help. vcpkg license-report prints the SPDX licenses from the SBOM of all installed packages.

This PR also prints the SPDX licenses for all packages which are being installed for a given install command. It does not print the licenses for already installed packages or dependencies of already installed packages. The existing dependency planner does not recurse into that, and we don't necessarily have the ports for that available anyway. If there's serious demand for this, we could consider doing it for manifest mode since there we know we have all the ports available at install time, but given that this solves @walbourn 's problem I'm not inclined to add it without substantial user feedback that it's something they need.

  • The warning about licenses not being generated by Microsoft is a direct copy from the text in WinGet. I need to double check with CELA that this wording is OK.
  • Needs docs PR

Resolves microsoft/vcpkg#10812

Some of the 'core infrastructure' work herein was done because I originally intended to record license status information in the installed database. Unfortunately, existing copies of vcpkg didn't record license information in the installed database, and doing this was going to be a breaking change to the on disk format. Moreover, it wouldn't work with ports that had been previously binary cached. This made the change controversial internally and we were not sure we were going to be able to do it. @ras0219-msft pointed out though that we record this information in the SBOMs already, so as long as it's a recent though package build to have an SBOM, we already had the information we needed.

This change is large and has independently reviewable subcomponents, so I'm going to split out sub-reviews.

Also introduce adapt_to_string to help with the several to_string overload pairs.
Drive by fix debug printing claiming that telemetry was not a string and that malformed git commit shas were not strings.
…eady installed' message, --head warning, and merging the 'green success' message into the total elapsed time part.
…nstalled in this invocation are printed, and a separate license-report command that prints all information known in the installed tree.
@dg0yt
Copy link
Contributor

dg0yt commented Oct 21, 2024

Implementation and tests must cover the license field in features. (IDK if it is already covered.)

BillyONeal added a commit to BillyONeal/vcpkg-tool that referenced this pull request Oct 21, 2024
Extracted from microsoft#1514

Rather than each test case inventing their own way of dealing with the Windows/Linux `r`n vs `n difference, this change just always makes the output collection functions to do that transformation.
BillyONeal added a commit to BillyONeal/vcpkg-tool that referenced this pull request Oct 21, 2024
BillyONeal added a commit to BillyONeal/vcpkg-tool that referenced this pull request Oct 21, 2024
BillyONeal added a commit to BillyONeal/vcpkg-tool that referenced this pull request Oct 21, 2024
BillyONeal added a commit to BillyONeal/vcpkg-tool that referenced this pull request Oct 21, 2024
BillyONeal added a commit that referenced this pull request Oct 22, 2024
* Commonize `r`n / `n handling in test cases.

Extracted from #1514

Rather than each test case inventing their own way of dealing with the Windows/Linux `r`n vs `n difference, this change just always makes the output collection functions to do that transformation.

* Fix damaged -contains pointed out by @ras0219-msft
# Conflicts:
#	azure-pipelines/end-to-end-tests-dir/build-test-ports.ps1
#	azure-pipelines/end-to-end-tests-dir/ci-verify-versions.ps1
BillyONeal added a commit that referenced this pull request Oct 23, 2024
# Conflicts:
#	include/vcpkg/base/contractual-constants.h
#	include/vcpkg/statusparagraph.h
#	src/vcpkg/spdx.cpp
#	src/vcpkg/statusparagraph.cpp
BillyONeal added a commit to BillyONeal/vcpkg-tool that referenced this pull request Oct 24, 2024
Extracted from microsoft#1514

build_package never prints usage so the setting should not be there.
BillyONeal added a commit to BillyONeal/vcpkg-tool that referenced this pull request Oct 24, 2024
BillyONeal added a commit to BillyONeal/vcpkg-tool that referenced this pull request Oct 24, 2024
Extracted from microsoft#1514

We were paying to package the object up into a value, then unpackage it.
BillyONeal added a commit to BillyONeal/vcpkg-tool that referenced this pull request Oct 24, 2024
Resolves microsoft/vcpkg#10812

Depends on microsoft#1529
Extracted from  microsoft#1514

Splits database_load_check into database_load, which merely loads the current database, and database_load_collapse, which additionally smashes any outstanding update files.
Splits get_installed_files into get_installed_files and get_installed_files_and_upgrade. The former avoids the format conversion and thus does not need to write.

Reading commands database_load / get_installed_files:
* export
* list
* owns
* package_info
* update

Writing commands which use database_load_collapse / get_installed_files_and_upgrade. Also intend to call database_load_collapse at the end after successful completion so that the status file has all update records merged in where possible.
* build
* ci
* install
* remove
* set-installed
* upgrade
BillyONeal added a commit that referenced this pull request Nov 1, 2024
Extracted from #1514

We were paying to package the object up into a value, then unpackage it.
BillyONeal added a commit that referenced this pull request Nov 1, 2024
BillyONeal added a commit that referenced this pull request Nov 1, 2024
* Remove PrintUsage from BuildPackageOptions.

Extracted from #1514

build_package never prints usage so the setting should not be there.

* Fix flipped print usage order in commands.install.cpp
# Conflicts:
#	include/vcpkg/base/contractual-constants.h
#	include/vcpkg/base/message-data.inc.h
#	include/vcpkg/spdx.h
#	include/vcpkg/statusparagraph.h
#	src/vcpkg-test/statusparagraphs.cpp
#	src/vcpkg/commands.install.cpp
#	src/vcpkg/commands.set-installed.cpp
#	src/vcpkg/spdx.cpp
#	src/vcpkg/statusparagraph.cpp
@BillyONeal
Copy link
Member Author

Implementation and tests must cover the license field in features. (IDK if it is already covered.)

This indeed does not work. Also, we don't record feature information in the SPDX file at all, so a more structural change is necessary for that. Drafting this for now.

I'm going to try to land #1377 , then add recording of features, then come back to this.

@BillyONeal BillyONeal marked this pull request as draft November 16, 2024 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot load status db when filesystem is readonly
2 participants