Nvidia GPU Support for Windows

[juliusfrost]

Feature request description

Hello, I was looking into Podman as an alternative to Docker for machine learning applications. As far as I can tell there is support for Linux systems through nvidia-container-toolkit but I couldn't find instructions for this on Windows. It would be great if there was support for this as this + Podman Desktop would be a nice drop in replacement for Docker Desktop which is currently the only viable solution for GPU containers on Windows.

Suggest potential solution

It seems that the Podman virtual machine doesn't have access to the Nvidia drivers, but other machines on WSL2 do (tested by running nvidia-smi). I don't understand why, but fixing this may be a start.

Have you considered any alternatives?

Not sure what the best approach is here.

Additional context

No response

[rhatdan]

@n1hility @baude WDYT?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[n1hility]

this one is still relevant

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rootfs]

I tested podman desktop on my windows 11 pro, with nvidia cuda for wsl2 installed. I can verify podman virtual machine has the same access to nvidia-smi as other wsl2 machines.

[rootfs]

In order to use gpu, I also did the following:

ssh to podman machine

podman machine ssh

inside podman machine, install nvidia container toolkit

following instructions from https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installing-with-yum-or-dnf

#curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
# sudo yum install -y nvidia-container-toolkit
# sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
# nvidia-ctk cdi list

If successful, nvidia-ctk cdi list will show all the devices.

test it out

[aa956]

Does not seem to work anymore.

PS C:\> podman machine ssh
Connecting to vm podman-machine-default. To close connection, use `~.` or `exit`
Warning: Permanently added '[localhost]:51973' (ED25519) to the list of known hosts.
Last login: Fri Jan 26 21:01:31 2024 from ::1
[root@desktop02 ~]# nvidia-ctk cdi list
INFO[0000] Found 1 CDI devices
nvidia.com/gpu=all
[root@desktop02 ~]# nvidia-container-cli info
NVRM version:   546.33
CUDA version:   12.3

Device Index:   0
Device Minor:   0
Model:          NVIDIA GeForce RTX 4060 Ti
Brand:          GeForce
GPU UUID:       GPU-47bcd798-877b-083b-5b3c-4ceae75bb8a5
Bus Location:   00000000:01:00.0
Architecture:   8.9
[root@desktop02 ~]#
logout
Connection to localhost closed.
PS C:\> podman run --device nividia.com/gpu=all --security-opt=label=disable nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -benchmark -gpu
Error: preparing container 9f627b7fe28f8765aad55445b172ffc08f22f6fc24fa6a6b24b9b883d49a3aec for attach: setting up CDI devices: unresolvable CDI devices nividia.com/gpu=all
PS C:\> podman --version
podman.exe version 4.9.0
PS C:\>

[rootfs]

Does not seem to work anymore.

PS C:\> podman machine ssh
Connecting to vm podman-machine-default. To close connection, use `~.` or `exit`
Warning: Permanently added '[localhost]:51973' (ED25519) to the list of known hosts.
Last login: Fri Jan 26 21:01:31 2024 from ::1
[root@desktop02 ~]# nvidia-ctk cdi list
INFO[0000] Found 1 CDI devices
nvidia.com/gpu=all
[root@desktop02 ~]# nvidia-container-cli info
NVRM version:   546.33
CUDA version:   12.3

Device Index:   0
Device Minor:   0
Model:          NVIDIA GeForce RTX 4060 Ti
Brand:          GeForce
GPU UUID:       GPU-47bcd798-877b-083b-5b3c-4ceae75bb8a5
Bus Location:   00000000:01:00.0
Architecture:   8.9
[root@desktop02 ~]#
logout
Connection to localhost closed.
PS C:\> podman run --device nividia.com/gpu=all --security-opt=label=disable nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -benchmark -gpu
Error: preparing container 9f627b7fe28f8765aad55445b172ffc08f22f6fc24fa6a6b24b9b883d49a3aec for attach: setting up CDI devices: unresolvable CDI devices nividia.com/gpu=all
PS C:\> podman --version
podman.exe version 4.9.0
PS C:\>

podman run --device nividia.com/gpu=all 

is a typo, you need to set nvidia.com/gpu=all