PyTorch DataLoader returning list instead of tensor on custom Dataset

Question

I do a trial about my dataset, this is my complete code:

data_root='D:/AuxiliaryDocuments/NYU/'
raw_data_transforms=transforms.Compose([#transforms.ToPILImage(),
                           transforms.CenterCrop((224,101)),
                           transforms.ToTensor()])
depth_data_transforms=transforms.Compose([transforms.CenterCrop((74,55)),
                                      transforms.ToTensor()])

filename_txt={'image_train':'image_train.txt','image_test':'image_test.txt',
          'depth_train':'depth_train.txt','depth_test':'depth_test.txt'}


class Mydataset(Dataset):
  def __init__(self,data_root,transformation,data_type):
     self.transform=transformation
     self.image_path_txt=filename_txt[data_type]
     self.sample_list=list()
     f=open(data_root+'/'+data_type+'/'+self.image_path_txt)
     lines=f.readlines()
     for line in lines:
        line=line.strip()
        line=line.replace(';','')
        self.sample_list.append(line)
     f.close()

  def __getitem__(self, index):
     item=self.sample_list[index]
     img=Image.open(item)
     if self.transform is not None:
        img=self.transform(img)
     idx=index
     print(type(img))
    return idx,img

 def __len__(self):
    return len(self.sample_list)

I print the type of img that is <class 'torch.Tensor'>, then I used the coding below：

test=Mydataset(data_root,raw_data_transforms,'image_train')
test_1=Mydataset(data_root,depth_data_transforms,'depth_train')
test2=DataLoader(test,batch_size=4,num_workers=0,shuffle=False)
test_2=DataLoader(test_1,batch_size=4,num_workers=0,shuffle=False)

print the information:

for idx,data in enumerate(test_2):
   print(idx,data)
   print(type(data))

but the type of data is '<class 'list'>', which I need is tensor.

Harshit Kumar · Accepted Answer · 2021-09-20 07:03:07Z

4

This is the expected output. DataLoader in your case is supposed to return a list. The output of DataLoader is (inputs batch, labels batch).

e.g.

for idx, data in enumerate(test_dataloader):
  if idx == 0:
    print(type(data))
    print(len(data), data[0].shape, data[1].shape)


<class 'list'>
2 torch.Size([64, 1, 28, 28]) torch.Size([64])

Here, the 64 labels corresponds to 64 images in the batch.

In order to pass it to the model, you can do

#If you return img first in your Dataset
    return img, idx

# Either
for idx, data in enumerate(test_dataloader):
    # pass inputs to model
    out = model(data[0])
    # your labels are data[1]

# Or
for idx, (inputs, labels) in enumerate(test_dataloader):
    # pass inputs to model
    out = model(inputs)
    # your labels are in "labels" variable

edited Sep 20, 2021 at 7:03

answered Sep 20, 2021 at 4:43

Harshit Kumar

13.1k10 gold badges62 silver badges78 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Feona Over a year ago

but I want to import the data to my network which requires the type of data that must be tensor

Harshit Kumar Over a year ago

Please check my edit.

Collectives™ on Stack Overflow

PyTorch DataLoader returning list instead of tensor on custom Dataset

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related