Skip to content

Commit

Permalink
update example command run training.sh
Browse files Browse the repository at this point in the history
  • Loading branch information
samsja committed Sep 5, 2024
1 parent b920cc1 commit d999451
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions open_diloco/run_training.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@


# you can either pass a fixed initial peer or set it to auto and the script will start a dht server for you
## # ./run_training.sh 2 1 auto --per-device-train-batch-size 16 --batch-size 512 --local-steps 10 --total-steps 88000 --c4-tiny

## ./run_training.sh 4 1 auto --per-device-train-batch-size 8 --total-batch-size 128 --lr 1e-2 --path-model ../tests/models/llama-2m-fresh --project debug --no-torch-compile --hv.local-steps 100 --fake-data --hv.matchmaking_time 2

# Function to get CUDA devices based on the number of GPUs and index
function get_cuda_devices() {
Expand Down

0 comments on commit d999451

Please sign in to comment.