Add support for Exodus IOSS properties related to field interpretation to the Catalyst IOSS Database #456
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gsjaardema
approved these changes
May 20, 2024
packages/seacas/libraries/ioss/src/catalyst/Iocatalyst_DatabaseIO.C
Outdated
Show resolved
Hide resolved
packages/seacas/libraries/ioss/src/catalyst/Iocatalyst_DatabaseIO.C
Outdated
Show resolved
Hide resolved
tokusanya
pushed a commit
to tokusanya/seacas
that referenced
this pull request
Jul 16, 2024
…n to the Catalyst IOSS Database (sandialabs#456) * Refactored testing infrastructure to add transient unstructured fields * First test failing, for wrong reason * Test Fixture completed. First failing test done. * Added failing tests for all 5 Props. Need to update Frwrk for Sur_Split_type * Refactored get*block() to be public * Added failing list for SURFACE_SPLIT_TYPE. Compromises made * Fixed conduit error related to scoping. Refactored tests a bit * Implemented Exodus IOSS properties. Removed asserts. Finished verifying fields * Refactored readField() to not have DatabaseIO as parameter
gsjaardema
added a commit
that referenced
this pull request
Jul 18, 2024
* Ioss: Initial workflow design to support dynamic topology Basic idea is to create a topology modification observer that can be registered on the region. This provides a callback mechanism for re-defining the Region mesh when a topology change occurs. The API is still currently fluid and will change until something is settled upon. A pre-requsite to this was adding the capability for the Region to handle file creation such that it can either create multiple output files when the topology changes or uses the NETCDF capability to create multiple file groups within a single file. Unit tests are included to show how use cases drive the development * APREPRO: Remove unused variable [ci skip] * EXODUS: Import a specific exodus version from exomerge * IOSS: Add option to omit_blocks to io_shell * SIMPI: Install headers in incldue subdir so do not interfere with client use of mpi * Add support for Exodus IOSS properties related to field interpretation to the Catalyst IOSS Database (#456) * Refactored testing infrastructure to add transient unstructured fields * First test failing, for wrong reason * Test Fixture completed. First failing test done. * Added failing tests for all 5 Props. Need to update Frwrk for Sur_Split_type * Refactored get*block() to be public * Added failing list for SURFACE_SPLIT_TYPE. Compromises made * Fixed conduit error related to scoping. Refactored tests a bit * Implemented Exodus IOSS properties. Removed asserts. Finished verifying fields * Refactored readField() to not have DatabaseIO as parameter * APREPRO: EQ and NE comparisons exp and string are valid * IOSS: Clean up fmt includes/use * GJOIN: Fix nset sset name handling * GJOIN: Fix mcdel call * Discontinuous galerkin (#403) * EXODUS: refactor field metadata attribute output * EXODUS: Beginnings of basis read * EXODUS: basis query * EXODUS: these got lost with some branch mishaps * EXODUS: Spelling fix * EXODUS: See if can fix intel errors * EXODUS: Fix intel build * EXODUS: Move internal function to correct header * Windows fixes: make functions non-inline Making certain functions in Ioss::Utils non-inline to avoid export errors on Windows. Accessing the non-exported static variables was causing errors. This avoids that. Also added API to access current values for various streams. * eliminate warnings * IOSS: cgns - do not create assemblies from Unspecified VC * Getting kokkos build working again [ci skip] * TEST: Enable kokkos build * IOSS: Change byte-size output to debug * shellcheck recommendations * TESTING: Remove adios2 until figure out issues * EPU: Minor improvement in message (plural vs singular) * IOSS: Rename qsort to sort * IO_MODIFY: Better help output * IO_MODIFY: clean-up comment [ci skip] * Revert "EPU: Minor improvement in message (plural vs singular)" This reverts commit 6ca490c. * IOSS: Switch to using doctest from Catch2 * IOSS: cgns - remove lambdas to see if fixes stria issue * IOSS: Remove shadowed variable * Revert "Remove shadowed variable." This reverts commit 4d54381a3264dea155de65d0588cb192b37c0e1f. * Revert "IOSS: cgns - remove lambdas to see if fixes stria issue" This reverts commit 8dde87a74344b8b57f0f948e58f73e82c950e074. * IOSS: cgns - get CG_BUILD_PARALLEL define for all versions * Update readme -- replace Catch2 with doctest [ci skip] * @wortiz has signed the CLA from Pull Request #303 * @mwestphal has signed the CLA from Pull Request #304 * BLOT: Fix new argument mismatch warning with gcc-12 * Fix a recursive warning on assignement copy operator * IOSS: DEBUG->DbgOut, WARNING->WarnOut to avoid windows issues * IOSS: new version of clang-format * New version of clang-format * New version of clang-format * TPL: Add INSTALL_PATH to metis/parmetis installs * MODULE: Remove old unused unsupported lines [ci skip] * @gjtempl has signed the CLA from Pull Request #305 * Updates for serial build Add SEACAS_HAVE_MPI include guards for ioss unit_tests that use MPI. These are still run for serial builds and this change allows them to compile and run. Add -DMATIO_SHARED:BOOL=${SHARED} to TPS/matio/runcmake.sh. Matio's CMake isn't respecting the CMAKE variable BUILD_SHARED_LIBS and still builds shared version when SHARED=NO. * IOSS: Handle uppercase field suffices * IOSS: Clean up uppercase suffix fix * ZELLIJ: Add offset option * KOKKOS: Use current version * TPL: Fix kokkos runcmake.sh for current version * EXODUS: Clean up * Copy top-level README to packages/seacas so available in Trilinos snapshot * CONFIG: Update name of kokkos library * EXODUS.PY: Add the get_block_id_map method This adds functionality to add being able to easily find the elements contained within EX_ELEM_BLOCK or other type without keeping track of global and internal indicies. * ZELLIJ: Better cache utilization for offset and scale * IOSS: Handle specified field ordering (via index) if present * IOSS: use reserve instead of resize since pushing back * IO_MODIFY: Add rename capability * IO_MODIFY: Clean up rename capability * CONFIG: Add gnubrew compiler option * CONFIG: Update hdf5 versions * IO_MODIFY: update help options * EXODUS: Fix problem with ex_get_block_id_map function * SUPES: Make a variable be a correct spelling * EXODUS: clang-analyzer fixes * Bring up-to-date with master * Clean up bad merge * EXODUS: Clean up bad merge * EXODUS: Clean up anothr bad merge * EPU: Bring in fix from master * EXODUS: Clean up python and varid lookup * EXODUS: Support cardinality and user-define suffices; refactor * EXODUS: Further refinement/refactoring of DG field interface * EXODUS: Refactor component separator to allow for empty value * EXODUS: Minor cleanup * EXODUS: Refactor function naems; add test of ex_field utils * EXODUS: Add missing file from last commit * EXODUS: Fix bad handling of user-defined suffices * EXODUS: Fix suffix handling in name building * CI: Fix cmake version requirement * EXODUS: More work on the basis metadata read * EXODUS: Fix null test -- use double pipe * EXODUS: refactor field metadata attribute output * EXODUS: Beginnings of basis read * EXODUS: basis query * EXODUS: these got lost with some branch mishaps * EXODUS: Spelling fix * EXODUS: See if can fix intel errors * EXODUS: Fix intel build * EXODUS: Move internal function to correct header * TEST: Enable kokkos build * EPU: Minor improvement in message (plural vs singular) * Revert "EPU: Minor improvement in message (plural vs singular)" This reverts commit 6ca490c. * IOSS: Remove shadowed variable * Revert "Remove shadowed variable." This reverts commit 4d54381a3264dea155de65d0588cb192b37c0e1f. * CONFIG: Add gnubrew compiler option * Bring up-to-date with master * EXODUS: Support cardinality and user-define suffices; refactor * EXODUS: Further refinement/refactoring of DG field interface * EXODUS: Refactor component separator to allow for empty value * EXODUS: Minor cleanup * EXODUS: Refactor function naems; add test of ex_field utils * EXODUS: Add missing file from last commit * EXODUS: Fix bad handling of user-defined suffices * EXODUS: Fix suffix handling in name building * CI: Fix cmake version requirement * EXODUS: More work on the basis metadata read * EXODUS: Fix null test -- use double pipe * EXODUS: Test field metadata read/write on nodes * CI: Fix cxx standard setting * EXODUS: Fix bad merge * EXODUS: Fix bad merge * EXODUS: Fix varid query for assembly and blob * EXODUS.PY: See if this fixes ci build issues * EXODUS: Fix ex__get_varid function * Revert "EXODUS.PY: See if this fixes ci build issues" This reverts commit 86f202d. * EXODUS: Clean up ex_attribute struct * EXODUS: realpath is gnu extension in c11/c99 * EXODUS: Provide realpath prototype * EXODUS: Add missing functions for mingw * EXODUS: Fix ex_attribute initiialization * CI: Disable nczarr in netcdf * EXODUS: Fix codacy null / 0 issue * EXODUS: Removed unused struct member * IOSS: Clean up some includes; remove the ... for ??? * EXODUS: Fix bad master merge * EXODUS: Fix intel compiler build * BLOT: Increase resolution of node/element id output * IOSS: Change meta_data to metadata * EXODUS: Use internal naming for static function * Remove ioss as required library * EXODUS: Add function name to output so know where it came from * IOSS: Add function mapping Ioss field names to exodus ex_field_type * IOSS: Output field metadata basic implemetation * IOSS: Rename function for consistency * IOSS: Add missing serialize io call * APREPRO: Make strings test more robust * IOSS: Build io_modify only if exodus enabled * IOSS: Detect if exodus enabled for exodus include * GREPOS: Intel overoptimizes unless inimap separate file * GREPOS: Fix bad merge * CONFIG: Silence CMake warning about python interpreter finding * IOSS: Fix use of c++14/17 type-traits * EXODUS: Filter out attributes Field@ * IOSS: Add mapping from exodus type to ioss field type * IOSS: Handle composite fields with different component separators * IOSS: Field - Add print method; support 2 separators for composite fields * IOSS: get_fields works after enhanced field found some fields * IOSS: Read/interpret exodus files with field metadata * IOSS: Eliminate some char **names uses -- convert to NameList * IOSS: Use more NameList and IntVector * CI: Remove cmake policy; not recognized in docker build * IOSS: CGNS - earlier versions have the CG_BUILD_HDF5 define in cgnsconfig.h * CI: Update to latest version [ci skip] * IOSS: Minor change to trigger workflows * IOSS: Propogate name_array changes to parallel * IOSS: Add serialize io calls wehre needed * IOSS: Use correct export macro * IOSS: Scalar fields do not need metadata; clutters file also * Io modify handle cgns (#453) * IOSS: io_modify - try to get cgns structured mesh coordinate mods working * IOSS: io_modify - fix coordinate modification routines * IOSS: io_modify - try to handle block geometry modify in cgns * IOSS: io_info -- bbox works for structured mesh * IOSS: io_modify - redo geometry transformation code --------- Co-authored-by: Greg Sjaardema <gdsjaar@sandia.gov> * EXODUS: Fix element count in test writer * IOSS: Handle user-defined field metadata output * IOSS: Handle composite field multiple separators * CI: See if can fix appveyor build [ci skip] * IOSS: Put debug output in ifdef * CI: Actually trigger build... * CI: See if this affects appveyor... * CI: Change install name/location * EXODUS: Fix formatting of comment; turn off clang-format * IOSS: Initial steps in adding Basis capability * IOSS: Initial steps in adding Basis capability to field metadata query * CI: mkdir lib before plugin path in case doesnot exist [ci skip] * APREPRO: Fix some issues found in sierra input files * Testing an undefined variable against a string in a boolean (EQ, NE, LT, ...) was always returning true. . Changed such that an undefined variable in a boolean test with a string expression treats the undefined variable as the empty string. `{if (undefined_var == "Greg")}' Will parse as `{if ("" == "Greg")}' and output an undefined_variable warning. * The `exp()` function will output an ERANGE error for both underflow and overflow. We are ok with underflow not giving an error. Changed so only get math error in case of overflow. * EXPLORE: Add/Refactor qa, qainfo and info list commands * CI: Better error handling in script * EPU: Delete input files if requested * APREPRO: Fix to_string when fmt:: full precision output being used * APREPRO: Refactor last change * IOSS: heartbeat - add HEARTBEAT_FLUSH_INTERVAL property, document 0 will flush every step * EXODUS: Fix output of Curl gradient field * IOSS: Get field-metadata basis somewhat working * APREPRO: Bring branch up-to-date and fix bad merge * IOSS: Refactor function names for ComposedVariable * EXODUS: Move basis to global attribute * IOSS: Add quadrature storage type; beginnings of allowing multiple basis types * EXODUS: Redo so basis, quad stored at global level * IOSS: Modify tokenize to optionally return empty tokens * IOSS: Start of refactor for allowing multiple basis; basis at root of exodus * EXODUS: Allow more than one basis; field can specify which basis it is using * IOSS: Refactor basis struct/type * Iniitial support for quadrature storage type * EXODUS: Treat Quad@ attribute as internal * EXODUS: Add quadrature type and multiple basis/quad types to test read/write * IOSS: cleanup quadrature implementation * IOSS: Better const correctness * IOSS: VariableType -- function to return quad/basis types * IOSS: Support quadrature type in ioss<->exodus type mapping * IOSS: exodus - output basis/quadrature types * IOSS: Better suffix upper/lower detection * EXODUS: Support nesting of user-defiend fields * IOSS: Better compare output for fields * IOSS: Remove debug print * IOSS: Allow nesting on user-defined fields; refactor basis/quad output * EXODUS: Remove unused variable * EXODUS: Output from test program to stdout, not stderr * EXODUS: Add test for enhanced field metadata * EXODUS: Fix pipestatus values * EXODUS: Fix windows build * EXODUS: Remove shadowed variable * EXODUS: Fix enhanced fields for random attribute ordering * EXODUS: Test - more robust when nc4 changes attribute ordering * EXODUS: get_X_metadata functions have count argument added so call-to-call order works * EXODUS: Check for a null pointer * EXODUS: Provide internal strsep and strlcat * EXODUS: Improve portability * EXODUS: A better way to get strdup * EXODUS: Fix some compiler warnings; see if that makes test run... * EXODUS: Fix memory leak in test code * EXODUS: Deallocat struct in correct place * EXODUS: Reorder unit test and update output * IOSS: Add type and type_string to variable types * IOSS: Refactor detailed field output * IOSS: Allow detailed field output * IOSS: Minor cleanup/refactor * NEM_SLICE: Minor fix * IOSS: Clean up fmt includes/use * IOSS: Rewrite how catylyst deals with field names * IOSS: Fix up some changes for catalyst for this branch * IOSS: Fix default separator setting; do not recognize fields if property set * EXODUS: Refactor get/put basis and quad metadata * IOSS: Minor tweak to field printing * IOSS: Refactor variable type determination * EXODUS: Update nesting define; fix test dump file * IOSS: Simplify datapool - remove seldom used capabilty * IOSS: Refactor some names; add print * EXODUS: Eliminate memory leak * EXODUS: Fix spelling error * EXODUS: Fix compiler warning * EXODUS: Clean up some function name/doc * Spelling fixes [ci skip] * IOSS: Add backward compatible interfaces back --------- Co-authored-by: Utkarsh Ayachit <utkarsh.ayachit@kitware.com> Co-authored-by: Greg Sjaardema <gdsjaar@sandia.gov> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Mathieu Westphal <mathieu.westphal@kitware.com> Co-authored-by: Gary Templet <gjtempl@sandia.gov> Co-authored-by: Mario LoPrinzi <mvlopri@sandia.gov> * Catalyst: Add support for Exodus IOSS Properties in Conduit Representation (#458) * Made conduit mimic exodus prop changes in Cat Database on read * Added field_data equality checks btwn Exodus and Catalyst DBs in exo prop tests * EXODUS: Increment version for latest enhanced field changes * Address a few coverity warnings * IOSS: Add omit_sets option to io_shell * IOSS: io_shell help formatting cleanup * EXODUS_FOR: Fix potential integer overflow * CI: Try to clean up hdf5 install; make some git clones faster * IOSS: Print filename on the open/create/close timing output * IOSS: Use c++14 deprecated syntax * IOSS: Better output of open/close/create times * SLICE: Use default data_stroage_type * EXPLORE: Fix behavior after bad parse warning * EXPLORE: Better warning/info message on SELECT * EXPLORE: Fix extra double quote in output [ci skip] * SLICE: Fix order of file close and mpi_finalize * Line decomp parallel (#459) * IOSS: refactor to reduce strings; give flexibility and efficiency * IOSS: Start of support for parallel line decomp * IOSS: Explicit template instantiation fix * IOSS: Allow passing filename down into line_decomp * IOSS: initial line_decompose(); compile/link does not run * IOSS: Refactor to try to eliminate duplicate code * IOSS: Add some missing includes * SLICE: Fix data_storage type * Share decomp code between slice and ioss line decomp * IOSS: Fix serial build * Pull element centroid into common utils class * Templative line_decompose; pass correct vector * Handle specified in guided_decompose * Remove unused function * elementToProc does not need 64-bit range * EXPLORE: Fix behavior after bad parse warning * EXPLORE: Better warning/info message on SELECT * Minor rearrange include files * IOSS: See if this fixes/affects msys2 build * IOSS: Add some logging/hwm code to line decomp * IOSS: Another try to see how affects msys2 build * IOSS: Better hwm logging output * Unify Slice and DecompositionUtils zoltan_decompose * IOSS: Fix msys2 build * clang-format run * SLICE: Version should be updated for latest chagnes * IOSS: Reduce storage potentially; fix zoltan free call * SLICE: Fix order of file close and mpi_finalize * IOSS: Enable decomposition statistics for line decomp * IOSS: io_shell - add_processor_id_field works for exodus also * IOSS: compose output will add a proc_id map to output * IOSS: thread-safe output_processor_id_map * CI: safer variable naming --------- Co-authored-by: Greg Sjaardema <gdsjaar@sandia.gov> * Correct Jamfile to match Seacas's CMake Library Names (#460) * Correct Jamfile to match Seacas's CMake Library Names Jamfile change for Sierra's build system to support a spack built version of Seacas. This change correctly matches Seacas library names to CMake's version of Seacas's libraries and changes the path name to 'seacas' rather than 'sierra_seacas' Signed-off-by: akimler <160887023+akimler@users.noreply.github.com> * Update Jamfile Fixed accidently removed Jamfile changes Signed-off-by: akimler <160887023+akimler@users.noreply.github.com> * Update Jamfile Remove whitespace Signed-off-by: akimler <160887023+akimler@users.noreply.github.com> --------- Signed-off-by: akimler <160887023+akimler@users.noreply.github.com> * IOSS: Reduce memory usage for line decomp * IOSS: Add line_decomp io_shell option; clean up output * IOSS: Reduce memory usage of line decomp * Catalyst API 2 (#461) Fixed crash bug when IOSS field suffix separator is empty. Added tests for case when reading from Conduit files written by Catalyst IOSS database. Changed time step selection when reading from Conduit files to use the IOSS integer property CATALYST_READER_TIME_STEP instead of IOSS database filename. * IOSS: Catalyst API 2 (#462) Allow CATALYST_READER_TIME_STEP to be defined as an environment variable. Changed default timestep to 0 for reading from Conduit files. * Fix errors detected in Trilinos integration * EXOMERGE: Handle non-3D models more correctly * GREPOS: Remove debug output * EXODUS: Fix output of map with zero entries * Address coverity issues * IOSS: Catalyst API 2 (#463) * IOSS: Catalyst API 2 Added support to database for using my_processor and processor_count IOSS properties when operating in serial parallel mode and reading Conduit files. * IOSS: Catalyst API 2 Changed Conduit serial parallel test to use files from the one MPI rank can.ex2 test. --------- Co-authored-by: Greg Sjaardema <gsjaardema@gmail.com> * Fix export symbols (#465) * IOSS: Catalyst API 2 (#466) Create the cell_ids and cell_node_ids fields for a StructuredBlock when asked for by a reading application, if these fields are not already stored in the Conduit data. Uses the get_cell_ids() and get_cell_node_ids() methods on StructuredBlock. * CI: See if can figure out intel build failure Signed-off-by: Greg Sjaardema <gdsjaar@sandia.gov> * CI: Another try with syntax fixed * CI: Use newer release of intel compiler 2024.2 * IOSS: Fix so latest fmt version works * NEM_SPREAD: Add missing include file * CI: Remove HDF5-1.8 support * IOSS: Fix reading db with inconsistent timestep count * IOSS: Clean up timestep count fix * CI: Spack build should use develop * CI: See if this uses commit sha * CI: Remove typo * CI: Spack build on master only [ci skip] * IOSS: Add progress output to chain/face generation * IOSS: Reduce Face memory * EXODUS: Better control over exodus verbosity * CI: Fix spack workflow syntax [ci skip] * IOSS: Add property to disable field metadata output * Modifications based on review --------- Signed-off-by: akimler <160887023+akimler@users.noreply.github.com> Signed-off-by: Greg Sjaardema <gdsjaar@sandia.gov> Co-authored-by: Greg Sjaardema <gdsjaar@sandia.gov> Co-authored-by: Alex Pelletier <108826411+ajpelle@users.noreply.github.com> Co-authored-by: Greg Sjaardema <gsjaardema@gmail.com> Co-authored-by: Utkarsh Ayachit <utkarsh.ayachit@kitware.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Mathieu Westphal <mathieu.westphal@kitware.com> Co-authored-by: Gary Templet <gjtempl@sandia.gov> Co-authored-by: Mario LoPrinzi <mvlopri@sandia.gov> Co-authored-by: akimler <160887023+akimler@users.noreply.github.com> Co-authored-by: tjotaha <tjotaha@sandia.gov> Co-authored-by: Spiros Tsalikis <spiros.tsalikis@kitware.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added support (and appropriate tests) for Exodus properties relating to field interpretation to the Catalyst IOSS database. The properties added are:
The first four were implemented leveraging the get_fields() method in Ioss_Utils, while for SURFACE_SPLIT_TYPE we return empty sidesets and sideblocks (and -1 for related fields) if the SURFACE_SPLIT_TYPE changed between the write and read Catalyst database.