Does this program support training based on muti-GPUS ?

[ChelsyHu]

i run it on a server with 3GPUs, it is working now, but it seems this program only used one GPU.

Fri May 3 12:09:24 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 520.61.05 Driver Version: 520.61.05 CUDA Version: 11.8 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla V100S-PCI... Off | 00000000:17:00.0 Off | 0 |
| N/A 81C P0 60W / 250W | 22334MiB / 32768MiB | 100% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 Tesla V100S-PCI... Off | 00000000:4B:00.0 Off | 0 |
| N/A 57C P0 44W / 250W | 32194MiB / 32768MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 2 Tesla V100S-PCI... Off | 00000000:CA:00.0 Off | 0 |
| N/A 42C P0 26W / 250W | 8MiB / 32768MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1482 G /usr/lib/xorg/Xorg 4MiB |
| 0 N/A N/A 2075466 C python3 22326MiB |
| 1 N/A N/A 1482 G /usr/lib/xorg/Xorg 4MiB |
| 1 N/A N/A 2062468 C ...onda3/envs/SGA/bin/python 32186MiB |
| 2 N/A N/A 1482 G /usr/lib/xorg/Xorg 4MiB |
+-----------------------------------------------------------------------------+