Using CUDA with pytorch?

Question

Is there a way to reliably enable CUDA on the whole model?

I want to run the training on my GPU. I found on some forums that I need to apply .cuda() on anything I want to use CUDA with (I've applied it to everything I could without making the program crash). Surprisingly, this makes the training even slower.

Then, I found that you could use this torch.set_default_tensor_type('torch.cuda.FloatTensor') to use CUDA. With both enabled, nothing changes. What is happening?

Possible duplicate of If I'm not specifying to use CPU/GPU, which one is my script using? — Omegastick
– Omegastick, Commented Jun 21, 2018 at 8:22
MyModel() is presumably just an example variable name for the model being used in the code. — End genocide - save Gaza
– End genocide - save Gaza, Commented Mar 13, 2021 at 15:15
Does this answer your question? How to run PyTorch on GPU by default? — End genocide - save Gaza
– End genocide - save Gaza, Commented Mar 13, 2021 at 19:20

ted · Accepted Answer · 2020-02-12 06:21:20Z

94

You can use the tensor.to(device) command to move a tensor to a device.

The .to() command is also used to move a whole model to a device, like in the post you linked to.

Another possibility is to set the device of a tensor during creation using the device= keyword argument, like in t = torch.tensor(some_list, device=device)

To set the device dynamically in your code, you can use

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

to set cuda as your device if possible.

There are various code examples on PyTorch Tutorials and in the documentation linked above that could help you.

edited Feb 12, 2020 at 6:21

ted

14.9k10 gold badges70 silver badges117 bronze badges

answered Jun 24, 2018 at 21:30

M. Deckers

1,34311 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Josiah Yoder Over a year ago

When calling tensor.to(device), for the device argument you can use 'cpu', 'cuda', 'cuda:0', 'cuda:1', etc. 'cuda' and 'cuda:0' mean the same thing in most circumstances. Click on the PyTorch tab within Section 5.6.1 of d2l.ai for more details.

Josiah Yoder Over a year ago

You can check if a tensor is located on the GPU by printing tensor.device.

End genocide - save Gaza · Accepted Answer · 2021-03-13 19:08:58Z

4

With both enabled, nothing changes.

That is because you have already set every tensor to GPU.

Is there a way to reliably enable CUDA on the whole model?

model.to('cuda')

I've applied it to everything I could

You only need to apply it to tensors the model will be interacting with, generally:

the model's pramaters model.to('cuda')
the features data features = features.to('cuda')
the target data targets = targets.to('cuda')

answered Mar 13, 2021 at 19:08

End genocide - save Gaza

25k10 gold badges113 silver badges133 bronze badges

Collectives™ on Stack Overflow

Using CUDA with pytorch?

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related