In dist training, update loss running avg every step, sync on log #2320
| Job | Run time |
|---|---|
| 1m 22s | |
| 1h 0m 26s | |
| 1h 24m 44s | |
| 31m 34s | |
| 34m 2s | |
| 41m 53s | |
| 34m 40s | |
| 1m 32s | |
| 42m 48s | |
| 1h 4m 52s | |
| 31m 47s | |
| 30m 54s | |
| 39m 26s | |
| 32m 34s | |
| 1m 40s | |
| 41m 50s | |
| 1h 4m 47s | |
| 27m 57s | |
| 29m 29s | |
| 38m 37s | |
| 30m 49s | |
| 12h 47m 43s |