Inverse Dirichlet Adaptive Loss #504
Conversation
…nd add computation to discretize_inner_functions
| """ | ||
|
|
||
| mutable struct InverseDirichletAdaptiveLoss{T <: Real} <: AbstractAdaptiveLoss | ||
| reweight_every::Int64 |
There was a problem hiding this comment.
Rename this reweights_steps? That's a bit more consistent with the rest of the ecosystem
There was a problem hiding this comment.
I chose reweight_every to be consistent with the variable names chosen for MiniMaxAdaptiveLoss and GradientScaleAdaptiveLoss. Is the naming convention is reweight_steps across other files?
There was a problem hiding this comment.
progress_steps, timeseries_steps, p_steps, etc. It's not the biggest deal, but it's slightly more consistent with what else is out there in SciML.
src/pinns_pde_solve.jl
Outdated
| pde_grads_std_max = maximum(pde_grads_std_all) | ||
| bc_grads_std = [std(Zygote.gradient(bc_loss_function, 0)[1]) for bc_loss_function in bc_loss_funcitons] | ||
|
|
||
| nonzero_divisor_eps = adaloss_T isa Float64 ? Float64(1e-11) : convert(adaloss_T, 1e-7) |
There was a problem hiding this comment.
What's the justification on the 1e-11 and 1e-7?
There was a problem hiding this comment.
I looked over the implementation by the authors of the paper and they don't include an additional epsilon in the denominator, so I think I should remove this.
|
The biggest thing is that this needs tests. |
|
This seems to work ok for the 2D Poisson equation but it does not outperform the nonadaptive loss across several seeds. |
|
Try the incompressible Navier-Stokes from the paper. I think this may only make a difference when it's "sufficiently hard", and Poisson is simple enough that non-adaptive is fine. |
|
Some of the review comments haven't been addressed, but other than that I think this is pretty close to merging. Using the Poisson 2D as the test is fine for CI, but we should test it separately on something harder. That separating testing can then turn into a tutorial. |
|
I've started working on a tutorial/test for the incompressible Navier-Stokes and I'll add the tutorial to docs/src/pinn. Should I add the test to adaptive_loss_tests.jl too? |
If it's not too long. That might get constrained by compute time when used in the test infrastructure. |
Co-authored-by: Christopher Rackauckas <accounts@chrisrackauckas.com>
|
This looks really good but I don't have time until my project deadline on Tuesday to review further. |
|
This should get rebased due to changes in #553. That would hopefully make it much cleaner too. |
Adds inverse dirichlet adaptive loss function to address #500