Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move building of python arrow modules from pyarrow to apache-arrow. #30285

Merged
merged 1 commit into from
Oct 8, 2024

Conversation

smoser
Copy link
Member

@smoser smoser commented Oct 8, 2024

A couple things here:

  1. fix FTBFS state that apache-arrow was currently in.
  • need zstd-static because:

    CMake Error at /usr/lib64/cmake/zstd/zstdTargets.cmake:107 (message):
    The imported target "zstd::libzstd_static" references the file
    "/usr/lib/libzstd.a" uses=cmake/configure
    but this file does not exist. Possible reasons include:

  1. cherry-pick a patch (from 2016) to rapidjson that allows a user of it
    (apache-arrow) to build with gcc-14.

  2. Add environment variables to build so the python build modules
    have access. I had hoped to localize these or use substitution
    in the environment block, but that is not supported.

    A nested pipeline's environment does not pass through to a uses step. chainguard-dev/melange#1548
    RFE: support substitution in environment.environment in nested pipelines or actions chainguard-dev/melange#1402

  3. drop unnecessary runtime dependency from libarrow to libarrow.

  4. remove pyarrow.

  5. bump the epoch on apache-arrow to 3 to be newer than pyarrow was.

  6. provide multiple versions of the python modules, and make them provide 'pyarrow' for backwards compat.

@smoser
Copy link
Member Author

smoser commented Oct 8, 2024

it looks to me right now that apache-arrow is just FTBFS. 😢

@smoser smoser force-pushed the fix/pyarrow-into-apache-arrow branch from 5f84d6e to d906537 Compare October 8, 2024 19:11
@smoser smoser changed the title try pyarrow -> apache-arrow Move building of python arrow modules from pyarrow to apache-arrow. Oct 8, 2024
@smoser smoser force-pushed the fix/pyarrow-into-apache-arrow branch from d906537 to 6854be2 Compare October 8, 2024 19:15
@smoser smoser marked this pull request as ready for review October 8, 2024 19:16
A couple things here:
1. fix FTBFS state that apache-arrow was currently in.

  * need zstd-static because:

       CMake Error at /usr/lib64/cmake/zstd/zstdTargets.cmake:107 (message):
         The imported target "zstd::libzstd_static" references the file
            "/usr/lib/libzstd.a" uses=cmake/configure
         but this file does not exist.  Possible reasons include:

2. cherry-pick a patch (from 2016) to rapidjson that allows a user of it
   (apache-arrow) to build with gcc-14.

3. Add environment variables to build so the python build modules
   have access.  I had hoped to localize these or use substitution
   in the environment block, but that is not supported.

   chainguard-dev/melange#1548
   chainguard-dev/melange#1402

4. drop unnecessary runtime dependency from libarrow to libarrow.

5. remove pyarrow.

6. bump the epoch on apache-arrow to 3 to be newer than pyarrow was.
@smoser smoser force-pushed the fix/pyarrow-into-apache-arrow branch from 6854be2 to a916c7c Compare October 8, 2024 19:23
@smoser smoser merged commit f54907c into wolfi-dev:main Oct 8, 2024
10 checks passed
gdonval pushed a commit to gdonval/wolfi-os that referenced this pull request Oct 9, 2024
…olfi-dev#30285)

A couple things here:
1. fix FTBFS state that apache-arrow was currently in.

  * need zstd-static because:

CMake Error at /usr/lib64/cmake/zstd/zstdTargets.cmake:107 (message):
         The imported target "zstd::libzstd_static" references the file
            "/usr/lib/libzstd.a" uses=cmake/configure
         but this file does not exist.  Possible reasons include:

2. cherry-pick a patch (from 2016) to rapidjson that allows a user of it
   (apache-arrow) to build with gcc-14.

3. Add environment variables to build so the python build modules
   have access.  I had hoped to localize these or use substitution
   in the environment block, but that is not supported.

   chainguard-dev/melange#1548
   chainguard-dev/melange#1402

4. drop unnecessary runtime dependency from libarrow to libarrow.

5. remove pyarrow.

6. bump the epoch on apache-arrow to 3 to be newer than pyarrow was.

7. provide multiple versions of the python modules, and make them
provide 'pyarrow' for backwards compat.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants