Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cargo: fix cross #196333

Draft
wants to merge 3 commits into
base: staging
Choose a base branch
from
Draft

cargo: fix cross #196333

wants to merge 3 commits into from

Conversation

alyssais
Copy link
Member

Description of changes
Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

# but it does support checking these idiosyncratic PKG_CONFIG_${TRIPLE}
# environment variables.
# [1]: https://github.com/rust-lang/pkg-config-rs/issues/53
export PKG_CONFIG_@rustTarget@=@targetPrefix@@baseBinName@
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't really approve of this rust-specific thing being added in all derivations that use pkg-config, even non-rust ones. I also don't disapprove, but for a quick merge you could move this change out of this PR and let me review only the cargo stuff (with that PKG_CONFIG_ variable being set just like in rustc).

Copy link
Member

@Mic92 Mic92 Oct 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if it was in the cargo setuphook? Than as soon as you add rustc to a build, you would get it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the pkg-config wrapper is the right place for this. It's the only place that makes sense that is shared between the rustc build, and all the builds that use Cargo.

Technically, it doesn't have anything to do with pkg-config (since it comes from the pkg-config crate), but it doesn't have anything to do with Cargo either. It's also extremely similar to PKG_CONFIG_FOR_BUILD, defined on the line above, so it makes sense to set both very similar environment variables in one place.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it doesn't go here.

We can have a separate setup hook that Rust. buildRustCrate, buildRustPackage, etc. all use.

Copy link
Member

@Ericson2314 Ericson2314 Oct 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rust-lang/pkg-config-rs#139 Also, I opened an issue in the relevant package build.rs file use, so we can nip this in the bud.

Copy link
Member

@Ericson2314 Ericson2314 Nov 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a small amount of code, but changing it is mass rebuild. So I think about the "mass rebuild" costs of these sorts of things too.

The rebuild could be indirect from changing the toRustTarget stuff too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also think how if we do two setup hooks it's easy to combine later, but if we do one setup hook it's hard to split later.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be an extremely rare change to make, especially for the limited set of architectures we actually build for. We do full rebuilds every couple of weeks anyway, so I don't think it's at all worth optimising for avoidance of mass-rebuilds in infrastructural stuff like this. Indeed, avoiding mass rebuilds is how we end up with unnecessary conditionals making maintenance more complicated forever, because it avoided mass rebuilds once. And Rust is so widely used that any change to it is going to cause huge rebuilds anyway, regardless of whether it affects pkg-config.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And having a separate setup hook just for this (cargoPkgConfigSetupHook?) would mean broken cross packages forever, because since it would only be required for cross, nobody would ever remember to apply it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we did this, would we want to put this same logic sort of in bintools-wrapper and cc-wrapper too?

@alyssais alyssais mentioned this pull request Oct 27, 2022
13 tasks
Copy link
Contributor

@yu-re-ka yu-re-ka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first time I looked at this, it was not obvious that PKG_CONFIG_FOR_BUILD is set directly above the newly added code, and that it is autoconf/meson specific.
I now think this might be fine, but would give @Ericson2314 the last word (to change their mind or explain why this is still a bad idea)

@Mindavi
Copy link
Contributor

Mindavi commented Jan 12, 2023

@Ericson2314 and @alyssais, what would be a good way to move this forward? I do kinda agree that it feels wrong to modify the pkg-config wrapper, if a hook were to be added, could that be added to buildRustPackage or injected when cargo or rust is added? Then there might be less of a need to remember to add it.

@alyssais
Copy link
Member Author

@Ericson2314 and @alyssais, what would be a good way to move this forward? I do kinda agree that it feels wrong to modify the pkg-config wrapper, if a hook were to be added, could that be added to buildRustPackage or injected when cargo or rust is added? Then there might be less of a need to remember to add it.

I asked John to expand on his most recent comment on Matrix, and AIUI we're still waiting for him to find the time to talk it through with me.

By moving this from the rustc derivation, the necessary environment
variables will also be set for other Rust packages that need them,
like cargo.
It's not possible to run installCheckPhase when cross-compiling, so we
have to tack this on to postInstall instead.
This fixes cross-compilation.
@ofborg ofborg bot requested a review from AndersonTorres January 13, 2023 20:42
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/unable-to-build-pkgsstatic-gcc/27795/6

@alyssais alyssais added the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 13, 2023
@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 19, 2024
@wegank wegank marked this pull request as draft March 20, 2024 15:53
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 20, 2024
@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: merge conflict This PR has merge conflicts with the target branch 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: cross-compilation Building packages on a different platform than they will be used on 6.topic: rust 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ 10.rebuild-linux-stdenv This PR causes stdenv to rebuild
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants