💡[feat] NVIDIA Multi-Instance GPU (MIG) Support

[TrevorM15]

Describe the problem

Currently the only option to inform determined of the number of GPUs a node has is with the maxSlotsPerPod field in the values.yaml file. However, Some GPUs that NVIDIA offers have the ability to partition the memory (MIG) for smaller workloads that don't saturate the GPU memory. In addition Nebuly offers a way to dynamically partition GPUs. Therefore, in order to better utilize resources, a nice feature would be the ability to specify MIG resources in something like maxSlotsPerPod.

Describe the solution you'd like

Possible yaml to describe 2 40GB A100s and requesting 8 MIGs of 10GB each spread across the two GPUs.

maxSlotsPerPod:
  value: 2
  mig:
    - 1g.5gb: 7
    - 2g.10gb: 4
    - 3g.20gb: 2

resources:
  slots_per_trial:
    2g.10gb: 8

Describe alternatives you've considered

No response

Additional context

No response

[ioga]

thank you for your suggestion.

we are planning to address this by allowing to specify arbitrary gpu resource requests per slot at resource pool level (e.g. multiple mig slices nvidia.com/mig-1g.5gb: 3 instead of fixed nvidia.com/gpu: 1).

[Wildshire]

Hello!

Do we have any news on this topic?

Best

[ioga]

No progress on this yet, sorry.