Improve reinitialization efficiency for contact

[dewenyushu]

Motivation

When dealing with problems involving intermittent contact or sliding between material bodies, the elements in contact are continually changing. This leads to frequent alterations in the sparsity patterns of the Jacobian matrix, necessitating frequent reinitialization. Preallocating memory in such cases can be extremely expensive.

To address this, hash table assembly has been developed and implemented in the following PR/MR:

• [WIP] Use hash table matrices when requested #28979
• Allow creating matrices based on hash tables libMesh/libmesh#3983
• https://gitlab.com/petsc/petsc/-/merge_requests/7939
• https://gitlab.com/petsc/petsc/-/merge_requests/7988)

However, hash table assembly may be slower than preallocation when preallocation is accurate, which is often the case for scenarios with minimal relative motion between contact pairs. Therefore, it's preferable to first determine if contact pairs have changed before attempting to reset the hash state.

Design

Initial design:

• Check for contact pair changes: Determine if any contact pairs have changed to help identify when a reset of the hash state is necessary.

• Leverage PETSc postcheck callbacks: To efficiently check whether contact pairs have changed, consider utilizing PETSc's postcheck callbacks rather than previous residual evaluation.

Impact

Significantly improve efficiency for contact simulations.

[dewenyushu]

Here is an example input that is converted from an existing MOOSE contact test. It does not have frequent change of contact pairs during the simulation, which shows computational advantage of preallocation vs the hash table initialization. Specifically, with uniform_refine = 5, the run times are 72s for hash table assembly vs 55s using preallocation on 1 processor. For coarser cases or multiple processors, the preallocation is constantly faster than the hash table initialization, a few seconds difference in runtime.

Inputs.zip

[maxnezdyur]

Few questions.

1. Does the preallocation problem occur with mortar contact too?
2. How bad of a memory vs computational speed problem would it be to have the ability for users to ask for "full" preallocation between two contact pairs so that even with large relative motion.

[lindsayad]

Yes the preallocation issue occurs with mortar contact as well. PETSc automatically shrinks unused sparsity pattern entries. We would have to explicitly insert zeroes for contact pairs uncoupled at any given Jacobian evaluation. I do not know what this means for memory/cpu performance penalty. The performance losses that @dewenyushu shared for the hash table based assembly are much better than what she reported to me on slack. I guess I am not all that sad about 55 -> 72 seconds when I've seen improvements from multiple hours to tens of seconds for Jacobian assembly for an assessment case that @vanwdani pointed me to

[dewenyushu]

Yes the preallocation issue occurs with mortar contact as well. PETSc automatically shrinks unused sparsity pattern entries. We would have to explicitly insert zeroes for contact pairs uncoupled at any given Jacobian evaluation. I do not know what this means for memory/cpu performance penalty. The performance losses that @dewenyushu shared for the hash table based assembly are much better than what she reported to me on slack. I guess I am not all that sad about 55 -> 72 seconds when I've seen improvements from multiple hours to tens of seconds for Jacobian assembly for an assessment case that @vanwdani pointed me to

Yeah that's the runtime difference I can get using a existing test case. I can probably share with you privately the actual problem we are looking at, which shows more than 5 times of slowdown (15515s -> 85680s), if you are interested.

[lindsayad]

Yes I am interested as long as it's checked into git somewhere