Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework the compression library search #5085

Open
wants to merge 21 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/aocc-cmake.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ jobs:
CC=mpicc cmake -C $GITHUB_WORKSPACE/config/cmake/cacheinit.cmake -G Ninja \
-DCMAKE_BUILD_TYPE=${{ inputs.build_mode }} \
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_PARALLEL:BOOL=ON \
-DHDF5_ENABLE_SUBFILING_VFD:BOOL=ON \
-DHDF5_BUILD_CPP_LIB:BOOL=OFF \
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/i386-cmake.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ jobs:
cmake -C ../config/cmake/cacheinit.cmake -G "Unix Makefiles" \
--log-level=VERBOSE \
-DCMAKE_BUILD_TYPE=${{ inputs.build_mode }} \
-DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF \
-DHDF5_BUILD_CPP_LIB:BOOL=OFF \
-DLIBAEC_USE_LOCALCONTENT:BOOL=OFF \
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/julia-cmake.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ jobs:
cd "${{ runner.workspace }}/build"
cmake -C $GITHUB_WORKSPACE/config/cmake/cacheinit.cmake -G Ninja \
-DCMAKE_BUILD_TYPE=${{ inputs.build_mode }} \
-DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_PARALLEL:BOOL=OFF \
-DHDF5_BUILD_CPP_LIB:BOOL=OFF \
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/netcdf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ jobs:
-DHDF5_BUILD_DOC:BOOL=OFF \
-DLIBAEC_USE_LOCALCONTENT:BOOL=OFF \
-DZLIB_USE_LOCALCONTENT:BOOL=OFF \
-DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=ON \
-DH5_NO_DEPRECATED_SYMBOLS:BOOL=OFF \
-DBUILD_TESTING:BOOL=OFF \
-DCMAKE_INSTALL_PREFIX:PATH=/usr/local \
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/nvhpc-cmake.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ jobs:
cd "${{ runner.workspace }}/build"
cmake -C $GITHUB_WORKSPACE/config/cmake/cacheinit.cmake -G Ninja \
-DCMAKE_BUILD_TYPE=${{ inputs.build_mode }} \
-DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_PARALLEL:BOOL=ON \
-DMPIEXEC_NUMPROC_FLAG:STRING=-np \
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/vol_adios2.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ jobs:
-DHDF5_ENABLE_PARALLEL:BOOL=ON \
-DHDF5_ENABLE_THREADSAFE:BOOL=ON \
-DHDF5_ALLOW_UNSUPPORTED:BOOL=ON \
-DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF \
${{ github.workspace }}/hdf5
cat src/libhdf5.settings
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/vol_async.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ jobs:
-DHDF5_ENABLE_PARALLEL:BOOL=ON \
-DHDF5_ENABLE_THREADSAFE:BOOL=ON \
-DHDF5_ALLOW_UNSUPPORTED:BOOL=ON \
-DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF \
-DHDF5_VOL_ALLOW_EXTERNAL:STRING="GIT" \
-DHDF5_VOL_URL01:STRING="https://github.com/HDFGroup/vol-async.git" \
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/vol_cache.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ jobs:
-DHDF5_ENABLE_PARALLEL:BOOL=ON \
-DHDF5_ENABLE_THREADSAFE:BOOL=ON \
-DHDF5_ALLOW_UNSUPPORTED:BOOL=ON \
-DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF \
-DHDF5_VOL_ALLOW_EXTERNAL:STRING="GIT" \
-DHDF5_VOL_URL01:STRING="https://github.com/HDFGroup/vol-async.git" \
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/vol_ext_passthru.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ jobs:
-DBUILD_STATIC_LIBS=OFF \
-DHDF5_TEST_API:BOOL=ON \
-DHDF5_ENABLE_PARALLEL:BOOL=ON \
-DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF \
-DHDF5_VOL_ALLOW_EXTERNAL:STRING="GIT" \
-DHDF5_VOL_URL01:STRING="https://github.com/hpc-io/vol-external-passthrough.git" \
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/vol_log.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ jobs:
-DHDF5_ENABLE_PARALLEL:BOOL=ON \
-DHDF5_ENABLE_THREADSAFE:BOOL=ON \
-DHDF5_ALLOW_UNSUPPORTED:BOOL=ON \
-DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF \
${{ github.workspace }}/hdf5
cat src/libhdf5.settings
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/vol_rest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ jobs:
-DHDF5_BUILD_HL_LIB:BOOL=ON \
-DHDF5_TEST_API:BOOL=ON \
-DHDF5_ALLOW_UNSUPPORTED:BOOL=ON \
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=OFF \
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF \
-DHDF5_VOL_ALLOW_EXTERNAL:STRING="GIT" \
-DHDF5_VOL_URL01:STRING="https://github.com/HDFGroup/vol-rest.git" \
-DHDF5_VOL_VOL-REST_BRANCH:STRING="master" \
Expand Down
31 changes: 19 additions & 12 deletions CMakeFilters.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,13 @@ endif ()
# Option for ZLib support
#-----------------------------------------------------------------------------
set(H5_ZLIB_FOUND FALSE)
option (HDF5_ENABLE_ZLIB_SUPPORT "Enable Zlib Filters" ON)
if(NOT DEFINED ZLIB_PACKAGE_NAME)
set(ZLIB_PACKAGE_NAME "zlib")
endif ()
if(NOT DEFINED ZLIBNG_PACKAGE_NAME)
set(ZLIBNG_PACKAGE_NAME "zlib-ng")
endif ()
option (HDF5_ENABLE_ZLIB_SUPPORT "Enable Zlib Filters" OFF)
if (HDF5_ENABLE_ZLIB_SUPPORT)
if (NOT H5_ZLIB_HEADER)
if (NOT ZLIB_USE_EXTERNAL)
Expand All @@ -88,11 +94,12 @@ if (HDF5_ENABLE_ZLIB_SUPPORT)
endif ()
set(ZLIB_FOUND FALSE)
if (HDF5_USE_ZLIB_STATIC)
set(ZLIB_SEACH_TYPE static)
set(ZLIB_SEARCH_TYPE static)
else ()
set(ZLIB_SEACH_TYPE shared)
set(ZLIB_SEARCH_TYPE shared)
endif ()
find_package (ZLIB NAMES ${PACKAGE_NAME} COMPONENTS ${ZLIB_SEACH_TYPE})
# Search pure Config mode first
find_package (ZLIB NAMES ${PACKAGE_NAME} OPTIONAL_COMPONENTS ${ZLIB_SEARCH_TYPE})
if (NOT ZLIB_FOUND)
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.24.0")
set(ZLIB_USE_STATIC_LIBS ${HDF5_USE_ZLIB_STATIC})
Expand Down Expand Up @@ -140,7 +147,7 @@ if (HDF5_ENABLE_ZLIB_SUPPORT)
message (VERBOSE "Filter HDF5_ZLIB is ON")
else ()
set (HDF5_ENABLE_ZLIB_SUPPORT OFF CACHE BOOL "" FORCE)
message (WARNING " ZLib support in HDF5 was enabled but not found")
message (FATAL_ERROR " ZLib support in HDF5 was enabled but not found")
endif ()
message(STATUS "H5_ZLIB_HEADER=${H5_ZLIB_HEADER}")
endif ()
Expand All @@ -149,6 +156,9 @@ endif ()
# Option for SzLib support
#-----------------------------------------------------------------------------
set(H5_SZIP_FOUND FALSE)
if(NOT DEFINED LIBAEC_PACKAGE_NAME)
set(LIBAEC_PACKAGE_NAME "libaec")
endif ()
option (HDF5_ENABLE_SZIP_SUPPORT "Use SZip Filter" ON)
lrknox marked this conversation as resolved.
Show resolved Hide resolved
if (HDF5_ENABLE_SZIP_SUPPORT)
option (HDF5_ENABLE_SZIP_ENCODING "Use SZip Encoding" ON)
Expand All @@ -160,10 +170,8 @@ if (HDF5_ENABLE_SZIP_SUPPORT)
endif ()
set(libaec_USE_STATIC_LIBS ${HDF5_USE_LIBAEC_STATIC})
set(SZIP_FOUND FALSE)
find_package (SZIP NAMES ${LIBAEC_PACKAGE_NAME}${HDF_PACKAGE_EXT} COMPONENTS ${LIBAEC_SEARCH_TYPE})
if (NOT SZIP_FOUND)
find_package (SZIP) # Legacy find
endif ()
# Search pure Config mode, there is not a FindSZIP module available
find_package (SZIP NAMES ${LIBAEC_PACKAGE_NAME}${HDF_PACKAGE_EXT} OPTIONAL_COMPONENTS ${LIBAEC_SEARCH_TYPE})
set(H5_SZIP_FOUND ${SZIP_FOUND})
if (H5_SZIP_FOUND)
set (H5_SZIP_INCLUDE_DIR_GEN ${SZIP_INCLUDE_DIR})
Expand All @@ -173,8 +181,7 @@ if (HDF5_ENABLE_SZIP_SUPPORT)
else ()
if (HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "GIT" OR HDF5_ALLOW_EXTERNAL_SUPPORT MATCHES "TGZ")
EXTERNAL_SZIP_LIBRARY (${HDF5_ALLOW_EXTERNAL_SUPPORT} ${HDF5_ENABLE_SZIP_ENCODING})
message (VERBOSE "Filter SZIP is built")
message (VERBOSE "... with library AEC")
message (VERBOSE "Filter SZIP is built using library AEC")
set (LINK_COMP_LIBS ${LINK_COMP_LIBS} ${H5_SZIP_STATIC_LIBRARY})
endif ()
endif ()
Expand All @@ -193,6 +200,6 @@ if (HDF5_ENABLE_SZIP_SUPPORT)
endif ()
else ()
set (HDF5_ENABLE_SZIP_SUPPORT OFF CACHE BOOL "" FORCE)
message (WARNING "SZIP support in HDF5 was enabled but not found")
message (FATAL_ERROR "SZIP support in HDF5 was enabled but not found")
endif ()
endif ()
4 changes: 2 additions & 2 deletions config/cmake/scripts/HDF5options.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,8 @@ set (ADD_BUILD_OPTIONS "${ADD_BUILD_OPTIONS} -DHDF5_ALLOW_EXTERNAL_SUPPORT:STRIN
#set(ENV{SZIP_ROOT} "some_location")
#set (ADD_BUILD_OPTIONS "${ADD_BUILD_OPTIONS} -DH5_SZIP_LIBRARY:FILEPATH=some_location/lib/szlib.lib -DH5_SZIP_INCLUDE_DIR:PATH=some_location/include")

### disable using ext zlib
#set (ADD_BUILD_OPTIONS "${ADD_BUILD_OPTIONS} -DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=OFF")
### enable using ext zlib
#set (ADD_BUILD_OPTIONS "${ADD_BUILD_OPTIONS} -DHDF5_ENABLE_ZLIB_SUPPORT:BOOL=ON")
### disable using ext szip
#set (ADD_BUILD_OPTIONS "${ADD_BUILD_OPTIONS} -DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF")
#set (ADD_BUILD_OPTIONS "${ADD_BUILD_OPTIONS} -DHDF5_ENABLE_SZIP_ENCODING:BOOL=OFF")
Expand Down
4 changes: 2 additions & 2 deletions release_docs/INSTALL_CMake.txt
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ Download from https://github.com/HDFGroup/hdf5/blob/develop/config/cmake/scripts
External compression plugin libraries from https://github.com/HDFGroup/hdf5_plugins:
hdf5_plugins.tar.gz

External compression szip and zlib libraries:
External compression szip and zlib libraries:0
ZLIB "https://github.com/madler/zlib/releases/download/v1.3.1/zlib-1.3.1.tar.gz"
ZLIBNG "https://github.com/zlib-ng/zlib-ng/archive/refs/tags/2.2.2.tar.gz"
LIBAEC "https://github.com/MathisRosenhauer/libaec/releases/download/v1.1.3/libaec-1.1.3.tar.gz"
Expand Down Expand Up @@ -939,7 +939,7 @@ if (BUILD_TESTING)
HDF5_ALLOW_EXTERNAL_SUPPORT "Allow External Library Building (NO GIT TGZ)" "NO"
HDF5_ENABLE_PLUGIN_SUPPORT "Enable PLUGIN Filters" OFF
HDF5_ENABLE_SZIP_SUPPORT "Use SZip Filter" OFF
HDF5_ENABLE_ZLIB_SUPPORT "Enable Zlib Filters" ON
HDF5_ENABLE_ZLIB_SUPPORT "Enable Zlib Filters" OFF

if (HDF5_USE_ZLIB_NG)
ZLIBNG_USE_EXTERNAL "Use External Library Building for ZLIBNG" OFF
Expand Down
14 changes: 13 additions & 1 deletion release_docs/RELEASE.txt
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ New Features
- Renamed the option: HDF5_ENABLE_Z_LIB_SUPPORT

The option has been renamed to HDF5_ENABLE_ZLIB_SUPPORT to be consistent
with the naming of other options.
with the naming of other options. Also, the option defaults to OFF.

- Added support for MinGW + MSYS2 when building with CMake

Expand Down Expand Up @@ -510,6 +510,18 @@ Bug Fixes since HDF5-2.0.0 release

Configuration
-------------
- Changed the zlib/szip compression find message to FATAL ERROR

The message was changed to indicate that zlib/szip compression was requested and
that it was not found. If an option is requested, not finding it should always
be an error.

- Removed the module search find_package for szip library

There is not a szip module file to use, so the find_package only uses
find_package in config mode. The choice then is to either build szip, with libaec,
inline or find a system installed szip library, built with CMake.

- Changed name of libhdf5hl_fortran installed by autotools to libhdf5_hl_fortran. The
new name is consistent with the name of the lib when installed by CMake and with the
other hl libs.
Expand Down
Loading