Build matrix generation

[shlevy]

It would be great if we could automatically generate build matrices from cabal files. E.g. given:

cabal-version:      3.0
name:               foo
version:            0.1.0.0
tested-with:        GHC == { 8.10.7, 9.2.4 }

library
  exposed-modules:
    Foo

  build-depends:
    , base             ^>= { 4.14, 4.16 }
    , primitive        ^>= 0.7
    , resourcet        ^>= { 1.2, 1.3 }

It would be awesome if I could get derivations to build foo with various the cartesian product of GHC 8.10.7 or 9.2.4 and resourcet ^>=1.2 and ^>=1.3, with the appropriate base being chosen on each path, and also great if I could get some variant of "build with latest GHC with allow-newer=foo:^all"

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[angerman]

@shlevy is the build-depends legal cabal syntax?

[shlevy]

Yes

[michaelpj]

I think this needs a clearer specification of what's desired. Using tested-with to imply the set of GHC versions to test with seems... reasonable? Picking a set of constraints to exercise the bounds of the package seems much harder.

And both of these are much harder to specify in a project context where we might have multiple local packages. For the GHC versions it might be okay, you could say something like:

• Collect all the tested-with GHC versions in the project.
• Make a matrix with those as a dimension
• For each GHC version, build only the packages that have it in tested-with

[andreabedini]

A week later but here are my thoughts.

The problem is the usual project/package distinction. Haskell.nix does not build packages but projects. There's not a single tested-with or a single (set of) build-depends to look at, but many.

Haskell.nix could automatically spit out projectVariants for all combination of GHCs and package versions; but in that case we would be testing is the fasibility of the whole project.

That a package says resourcet ^>= { 1.2, 1.3 } does not imply there's a feasible project with resourcet == 1.2.5 since this depends on all packages in the project allowing that version. We could say the statemet resourcet ^>= { 1.2, 1.3 } is a promise that the package would work with those versions of resourcet if it had the chance :-).

Now, I do like if something like this existed and what we could do is to add an "entrypoint" separate from project and friends, that is specifically made to test building a package in isolation. This is basically how the CHaP is built: we have a function that given a package name and version, makes a tailored project and builds that. Note that haskell.nix already has hackage-package and snapshots."lts-13.28" which do something conceptually similar. Something like that could indeed offer the automated facilities you are thinking about.

Australians are known to say "yeah but nah", but I am going with "nah but yeah" in this case 😆

Also notice that it's already quite immediate to do the last thing you mention:

flake = project.flake {
  variants = { allow-newer-caret = { cabalProjectLocal = "allow-newer: foo:^all"; }; };
}

produces hydraJobs.allow-newer-caret etc

[shlevy]

Let's separate out two functionalities:

1. Generating a matrix from an explicit specification (GHC versions, dependency versions, "latest" for each). If you give a specification which doesn't actually work from your cabal project, then solving will just fail for that element of the matrix
2. Generating a specification from cabal bounds and tested versions. This could start with just multiple GHC versions, then some kind of first-pass "if you use caret grouping syntax you want to try the latest from each spec", then perhaps later some kind of "OK but also we need to meet the constraints of the rest of the project, so drop out variants that cabal can't solve"

[andreabedini]

I think this is a good plan. @angerman I'll move this to the board if you are happy with it.

[michaelpj]

Related: https://github.com/input-output-hk/cardano-haskell-packages/issues/174

An important quirk which we need to think about: often if a package doesn't build with a particular version of GHC, it will not only not build, but the solver will fail. That's harder for us to deal with, because it means that the cabal project IFD will fail, and we get at best nothing and at worst an eval error. So if we're going to use the tested-with metadata, we need a way to get it that's independent.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[andreabedini]

It's still on the board 😂