CHBMIT datastet- Bad AUC result for patient chb3

[Afef00]

When I was trying to run the code for the patient 3 in the CHBMIT dataset the training gives me an AUC of 0.307 . At the last epoch the training accuracy was 98.36% however the validation accuracy was 29.31%. I was wondering why the training stopped at this bad results and does not keep training?

[truongduynhan]

Hi Afel00,
There was an early stopping that monitors both training loss and validation loss. In your case, it is likely that the monitor saw the model overfitting so it stopped the training.

[truongduynhan]

Hi Afel00,

Can you show the whole history of training? Just in case, I re-run for CHBMIT patient 3 and this is what I got.
Train on 3574 samples, validate on 1192 samples
Epoch 1/50
2021-10-13 13:54:40.103201: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-10-13 13:54:40.637237: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7

• 23s - loss: 0.3755 - accuracy: 0.8276 - val_loss: 0.7650 - val_accuracy: 0.5512
  Epoch 2/50
• 9s - loss: 0.1902 - accuracy: 0.9256 - val_loss: 1.0192 - val_accuracy: 0.5612
  Epoch 3/50
• 8s - loss: 0.1221 - accuracy: 0.9550 - val_loss: 0.9407 - val_accuracy: 0.6820
  Epoch 4/50
• 8s - loss: 0.0647 - accuracy: 0.9757 - val_loss: 0.9588 - val_accuracy: 0.7156
  Epoch 5/50
• 8s - loss: 0.0366 - accuracy: 0.9863 - val_loss: 1.0670 - val_accuracy: 0.7181
  Epoch 6/50
• 8s - loss: 0.0266 - accuracy: 0.9916 - val_loss: 0.7829 - val_accuracy: 0.7836
  Epoch 7/50
• 9s - loss: 0.0324 - accuracy: 0.9868 - val_loss: 1.8023 - val_accuracy: 0.6594
  Epoch 8/50
• 9s - loss: 0.0196 - accuracy: 0.9938 - val_loss: 0.7990 - val_accuracy: 0.7844
  Epoch 9/50
• 9s - loss: 0.0153 - accuracy: 0.9958 - val_loss: 1.3199 - val_accuracy: 0.7257
  Epoch 10/50
• 9s - loss: 0.0145 - accuracy: 0.9964 - val_loss: 1.1505 - val_accuracy: 0.7131
  Epoch 11/50
• 9s - loss: 0.0164 - accuracy: 0.9950 - val_loss: 2.0753 - val_accuracy: 0.6703
  Epoch 12/50
• 9s - loss: 0.0088 - accuracy: 0.9975 - val_loss: 2.0546 - val_accuracy: 0.6829
  Epoch 13/50
• 9s - loss: 0.0092 - accuracy: 0.9969 - val_loss: 1.4302 - val_accuracy: 0.7324
  Epoch 14/50
• 9s - loss: 0.0091 - accuracy: 0.9955 - val_loss: 1.1846 - val_accuracy: 0.7341
  Epoch 15/50
• 9s - loss: 0.0041 - accuracy: 0.9978 - val_loss: 1.6080 - val_accuracy: 0.6988
  Epoch 16/50
• 9s - loss: 0.0078 - accuracy: 0.9969 - val_loss: 2.0751 - val_accuracy: 0.6711
  Epoch 17/50
• 9s - loss: 0.0050 - accuracy: 0.9986 - val_loss: 1.7395 - val_accuracy: 0.6913
  Test AUC is: 0.9930037313432836