I've checked the code that produces this error message and it looks correct. Can you show me a snippet that throws it?

Sorry, I can't. I don't have a snapshot of my code that gave the error. But, I solved it by using .cuda() on my tensor.

@apaszke -- Does this help?

import torch
from torch.autograd import Variable
from torchvision.models import vgg16

x = Variable(torch.zeros((1, 3, 224, 224)))
model = vgg16(pretrained=False)
model.cuda()
model(x)

throws

...
RuntimeError: expected CPU tensor (got CUDA tensor)

I'm very new to pytorch but it does sound backwards -- x works on the CPU model, and I don't know how it would've become a CUDA tensor. Perhaps the error message is not referring to x, but to something that touches x inside the model.

@bkj @apaszke This error seems to appear when you call .cuda on the module and the input tensor, but forget to use the value returned by the call on the tensor (rather than the original tensor, which stays on the cpu).

It seems that the error message is getting something wrong: should it say "expected CUDA tensor (got CPU tensor)"?

I find it somewhat confusing that calling .cuda on a module moves it to gpu wheareas calling .cuda on a tensor returns a new tensor but the original one stays on the cpu.

Will fix. Yes, cuda() has different semantics for modules and tensors, but it's because modules are much more heavyweight objects that maintain a lot of state. It could have been emphasised more it the docs though.

I also came across the same issue.
If I run the code:

model = SimpleNet_2cls_2(img_shape, cls_num)
  
model = model.cuda()

It will have the error:
RuntimeError: expected CPU tensor (got CUDA tensor)
But If I run the code:

  model = SimpleNet_2cls_2(img_shape, cls_num)
  
#  model = model.cuda()
  
  model = torch.nn.DataParallel(model).cuda()

There is no error.
What is the difference here?

It seems that calling .cuda() on the input Variable fixes this, but is a non-issue on VGG and Alexnet. The inconsistency is a tad disturbing.

Yes, even I'am having the same issue. Can we expect some update on this?

I have also met the same situation.

state_action_values = self.model.forward(state_batch.cpu()).gather(1, action_batch)

this code snippet is the same as in the Pytorch DQN tutorial. When I run this I got:

RuntimeError: expected CPU tensor (got CUDA tensor)

If I run the snippet below, also got the same error message.

state_action_values = self.model.forward(state_batch).gather(1, action_batch)

@bhattad2 & @bywbilly make sure you explicitly call .cuda() on your Variables. That should fix the issue and make it compatible with all models.

As @varunagrawal said, adding .cuda() to all the Variables worked.

@karlTUM and @varunagrawal it worked, thanks a lot!

I have also met this issue, input.cuda() does not put Variable input on GPU, we have to explicitly assign the return value to input. But model.cuda() will put the model on GPU, which is a different behaviour.
I think it is better to make the two behaviour consistent.

I had the same issue. While the error was definitely justified, it did raise:
RuntimeError: expected CPU tensor (got CUDA tensor)
while the tensor I actually gave the model was a CPU tensor, and the model was on cuda. The error should have been the other way around.

Note: for me it was on a Conv1D, but I don't think this changes much since they all use ConvnD in the end.

I had the same issue, and solved thanks to @karlTUM.
It seems there is a bug when calling .cuda() on modules, so please try
model = torch.nn.DataParallel(model).cuda() instead.

Get the same issue. When the model get a CPU tensor but it's in GPU, the error said it need a CPU tensor but get a GPU tensor.

I am seeing this error for models when run with the JIT tracer and the default tensor type being a CUDA tensor. JITing on the CPU or running without JIT on CUDA, seems fine though. I am still trying to isolate a minimal failing example. Any debugging tips are appreciated.

@neerajprad Cause JIT load always go to CPU. You need always an explicit move #12710

@neerajprad Cause JIT load always go to CPU. You need always an explicit move #12710

Is that true for jit.trace too? I am not able to construct a minimal example where the tracer fails in these cases, but I have many larger models failing with: expected type CUDADoubleType but got CPUDoubleType (compute_types at /home/npradhan/workspace/pyro_dev/pytorch/aten/src/ATen/native/TensorIterator.cpp:134).

EDIT: Original issue - pyro-ppl/pyro#1419.

@bkj I tried this on v1.3.1

import torch
from torchvision.models import vgg16

x = torch.zeros((1, 3, 224, 224))
model = vgg16(pretrained=False)
model.cuda()
model(x)

and it throws

...
RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

so I think the error message has been improved to match what we expect.

@neerajprad Curious were you able to construct a minimal example for jit.trace?

@neerajprad I am closing this issue now because the error message has been improved and it was originally not related to JIT. Please feel free to open a new issue and link to this issue if you are able to construct a minimal example for jit.trace. Thanks!