STOPPING CRITERION

[ghost]

Good time of a day! I'm wondering if there's a way to end training procedure when I have reached desired accuracy in terms of energy and forces? I mean, the one implemented in nequip.

[simonbatzner]

Hi Tim, thanks for your interest in our code. Yes, the stopping criterion can be anything you want to be, so you'd just adapt this line in the yaml to be validation_f_mae e.g.:

early_stopping_patiences:                                                          # stop early if a metric value stopped decreasing for n epochs
  validation_loss: 50

[ghost]

Thanks a lot!

[tienmng]

Is 50 a good number for this stopping criterion? Thanks!

[Linux-cpp-lisp]

Depends entirely on your dataset and application; loss in particular is hard to reason about across systems. You could, for example, early stop on validation_f_mae being 50 (whatever force units you are using) if that is reasonable

[tienmng]

Then I'm guessing I have to set lr_scheduler_patience to be less than 50 to drop the learning rate if the f_mae doesn't improve after <50 epochs before the training stops. Please correct me if I'm wrong. Thanks!

[Linux-cpp-lisp]

ah whoops I was confusing this with my post below about lower bounds, this is on patience--- yes, this is a number of epochs; how reasonable 50 is is still dataset dependent but will probably never be crazy. If you are using a lot of data, fewer epochs of patience may be reasonable. Setting this higher will have basically no effect, since you'll just run longer without improving before early stopping. Deployment and evaluation by default always take the best model so far, not the last model (defined by validation loss), so you can run as many epochs as you want where nothing is improving without changing later results.

[Linux-cpp-lisp]

You can also early stop on absolute thresholds:

early_stopping_lower_bounds:                                                       # stop early if a metric value is lower than the bound
  validation_f_mae: 1.0e-5

See full.yaml.