4

The following code is a batch data provider for .mat files, but has the following problem when running it:

    TypeError: expected str, bytes or os.PathLike object, not FIFOQueue

The code is:

    import numpy as np
    import tensorflow as tf
    from tensorflow.python.framework import ops
    from tensorflow.python.framework import dtypes
    import h5py

    def Reader(filename):
        with h5py.File(filename, 'r') as f:
            image = np.transpose(np.array(f.get('patch_x'), dtype=np.float32))
            label = np.transpose(np.array(f.get('patch_y'), dtype=np.float32))
        image = ops.convert_to_tensor(image, dtype=dtypes.float32)
        label = ops.convert_to_tensor(label, dtype=dtypes.float32)

        return image, label

    def Inputs(filenames, batch_size, shuffle=True):
        filenames = ops.convert_to_tensor(filenames, dtype=dtypes.string)
        filename_queue = tf.train.string_input_producer(filenames, shuffle=shuffle)
        image, label = Reader(filename_queue)
        image = tf.cast(image, tf.float32)
        label = tf.cast(label, tf.float32)

        num_preprocess_threads = 4
        if shuffle:
            image_batch, label_batch = tf.train.shuffle_batch([image, label], batch_size=batch_size, num_threads=num_preprocess_threads, capacity=5*batch_size, min_after_dequeue=2*batch_size)
        else:
            image_batch, label_batch = tf.train.batch([image, label], batch_size=batch_size, num_threads=num_preprocess_threads, capacity=5*batch_size, min_after_dequeue=2*batch_size)
        return image_batch, label_batch

Does anyone know how to convert a string tensor to a python string easily? Thanks.

UPDATED 1: when using the filename.dequeue(), the error information is:

    TypeError: expected str, bytes or os.PathLike object, not Tensor
2
  • 1
    what line is the error on? Commented Jul 21, 2017 at 21:11
  • @Aaron this line with h5py.File(filename, 'r') as f: Commented Jul 21, 2017 at 21:38

2 Answers 2

3

To convert a string TensorFlow tensor to Python string, run bytes.decode(string_tensor.numpy()).

Sign up to request clarification or add additional context in comments.

Comments

0

tf.train.string_input_producer() return a queue, not a string. You have to use Graph operation, which get string tensor from this queue and read file from disk. For example you can use chain of operation:

image = tf.image.decode_jpeg(tf.read_file(filename_queue.dequeue()))

if you have jpeg files.

In TensorFlow 1.2 there is new structure Dataset for creating input pipeline. I think it is good idea to use Dataset instead queues.

8 Comments

Thanks for your answer. I think the data loader in pytorch is more simple. [pytorch.org/tutorials/beginner/data_loading_tutorial.html]
Using Dataset or Queue is not necessary. You can use tf.placeholder for graph input data, read data from the files by existing python code, and feed this data to the graph.
Many talent programmers provide a lot of interfaces based on the tensorflow, those are confusing to me. In fact, we just need some simplest interfaces like those classes in caffe, give an input, then output a new tensor. I don't know why they developed so many children using many many different tf packages.
There are too many tfxxxx. Why don't they use the original classes and functions in official tensorflow package?
I can't ask to this question, I am not developer of TF:) But I think there are not a problem using only "core" part of TF and don't use additional modules.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.