Skip to content

Commit

Permalink
fix it
Browse files Browse the repository at this point in the history
  • Loading branch information
samsja committed Sep 24, 2024
1 parent 71a9f89 commit 88b5250
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion open_diloco/simulate_multi_node.sh
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ mkdir -p logs
for i in $(seq 0 $(($N - 1 )))
do
> logs/log$i
CUDA_VISIBLE_DEVICES=$(get_cuda_devices $NUM_GPU $i) uv run torchrun --nproc_per_node=$NUM_GPU --node-rank $i --rdzv-endpoint localhost:9999 --nnodes=$N $@ > logs/log$i 2>&1 &
CUDA_VISIBLE_DEVICES=$(get_cuda_devices $NUM_GPU $i) torchrun --nproc_per_node=$NUM_GPU --node-rank $i --rdzv-endpoint localhost:9999 --nnodes=$N $@ > logs/log$i 2>&1 &
child_pids+=($!)
done

Expand Down

0 comments on commit 88b5250

Please sign in to comment.