2,081 questions
6
votes
0
answers
110
views
partially decode, stream and filter big data with tensorflow_datasets (tfds)
I have two issues (Note that this code is generated in google colab):
Issue 1 I want to stream the droid dataset, which is almost 2TB big. I want to only use data which matches my filter conditions. ...
0
votes
1
answer
130
views
SSL:CERTIFICATE_VERIFY_FAILED for TensorFlow dataset loading
Issue
I have previously loaded this dataset in Google Colab without error, but I now encounter an error.
Code
!pip install --upgrade 'tensorflow_data_validation[visualization]<2'
import tensorflow ...
- 23
0
votes
1
answer
73
views
Dataset from generator in tensorflow error GetNextFromShard
I have a train step for a Vertex AI pipeline that uses a template code for setting up distributed training. When fitting any model I get a message in the first epoch indicating the following message:
...
0
votes
1
answer
40
views
tensorflow dataset loop endless will model.predict
I use this code to setup my dataset for trainning and predict:
train_dataset = train_dataset.batch(train_batch_sz)
train_dataset = train_dataset.repeat().prefetch(5)
test_dataset_sim = test_dataset
...
- 453
1
vote
2
answers
122
views
How to load small sample of tensorflow_dataset data
I'm trying to play around with a neural network using the "lm1b" dataset in my computer before training a larger model in a cloud machine.
What I'm struggling with is that every time I try ...
- 481
1
vote
1
answer
111
views
Tensorflow dataset splitted sizing parameter problem: Local rendezvous is aborting with status: OUT_OF_RANGE: End of sequence
Pretty new with data generator and dataset from tensorflow. I struggle with sizing batch, epochs and step... I can't figure the good set up to remove error "Local rendezvous is aborting with ...
- 453
0
votes
0
answers
44
views
Using 'from_tensor_slices' causes the model to stop learning
I am trying to train a simple CNN on custom image data. While training with raw numpy data, the model learns as expected. However when I convert the same raw numpy data to dataset using tf.data....
- 13.9k
1
vote
1
answer
33
views
Give a list of image path pairs to load using tf.data.Dataset.map but it seems to only read 2
I am learning how to use Tensorflow with a piece of old code provided at a workshop a couple of years back (i.e. it should've been tested and works). However it didn't work, and my investigations led ...
- 13
0
votes
1
answer
74
views
VAalue_Error: as_list() is not defined on an unknown TensorShape when using dataset generator
There are a number of previous answers to this error but none relate to my issue.
I am creating a tensorflow dataset using a generator and for the examples sake am trying fit a very simple API model.
...
- 11
-1
votes
1
answer
129
views
Input image is not compatible with tensorflow model input shape [closed]
I was building a model and I wanted to test its performance, thus I imported a local file and load it and try to predict its label with the following code:
from tensorflow.preprocessing import image
# ...
- 197
0
votes
1
answer
68
views
Batch items in Tensorflow Dataset that contain varying first dimension, but constant second dimension
I have a Tensorflow dataset that contains tensors of rank 2. Each tensor was read from a file and has varying first dimension and constant second dimension. For example, the shapes of each item can be ...
- 11
1
vote
0
answers
43
views
Multi-Input network for image classification using TensorflowDataset gets none values not supported error
I have a dataset of tfrecords and mapped them to a Tensorflow dataset. The dataset structure has a dict structure as follows:
{ "image" : ImageDataTensor, "additional_features" : ...
- 23
0
votes
1
answer
82
views
tensorflow from_tensor_slices gives "args_0:0"
I'm trying to create a tensorflow dataset using from_tensor_slices but the string I'm passing is getting messed up within the map function:
def test(x):
print(x)
return x
filenames=['a','b','...
- 1,009
0
votes
0
answers
62
views
How can I create sequences (windows) from data in a TFRecord?
I have around 1800 short videos. Each video is around 30 seconds. I trained a VAE to encode each frame into a latent vector (of size 200).
Using this VAE, then I created a TFrecord with an entry for ...
- 1,638
1
vote
0
answers
74
views
Tensorflow Performance Drop When Training Sequential Model Using `tf.data.Dataset.from_generator`
I'm training a Sequential model in a binary classification problem. My dataset is in HDF format and consists of many files, which are often too large to fit in memory. To handle this, I tried using a ...
- 11
2
votes
0
answers
44
views
Time Series Data: Sequence classification with tensorflow - ValueError:
I am currently in the process of using an existing network architecture (TSMixer: An All-MLP Architecture for Time Series Forecasting) for a sequence classification.
The existing structure of my data: ...
- 21
1
vote
0
answers
47
views
How to convert image list and mask list to TensorFlow dataset?
I'm having trouble converting image and mask data into Dataset that can be used in keras.Model.
I have a list of 100 images with a shape of (100, 128, 128, 3)) and I have a list of 100 ...
- 11
0
votes
0
answers
238
views
model.fit is not working "RuntimeError: `tf.data.Dataset` only supports Python-style iteration in eager mode or within tf.function."
I want to code a simple machine learning on JupyterLab but got error as Title
I have collect the data as x are lamda, eps, c (the shape of data is 102010 rows × 3 columns) and y is sen (the shape of ...
- 1
0
votes
0
answers
283
views
Tensorflow returning TypeError for no reason
I am trying to work with tensorflow_datasets library, the dataset I am working on is STL-10, but it doesn't matter much, because no matter what dataset I use, the same error is returned. I am using ...
- 71
0
votes
0
answers
141
views
How can I tackle Attribute Error when I import tensorflow_datasets?
I have been trying to import the tensorflow_datasets module but it repeatedly shows an attribute error.
I have tried clearing the cache, reinstalling the package again and also restarting the device ...
0
votes
0
answers
163
views
Why is KerasCV augmentation layer destructively editing image labels?
I'm currently approximately following Gitesh Chawda's guide for YOLOV8 from here
Everything is going as expected until the image augmentation stage, at which point the images are correctly augmented, ...
-1
votes
1
answer
113
views
Food101 Tensorflow Dataset Requested split "validation" does not exist
I'm trying to use food101 dataset but while it has meta folder of train and test it can only see train data.
(train_data, test_data), ds_info = tfds.load(name="food101",
...
1
vote
0
answers
91
views
"Correct" way of preparing and feeding multiple pandas dataframes containing multivariate time series into a LSTM in tensorflow? (data pipeline)
I am pretty new with working with Tensorflow and in coding in general. So I'm sorry if this question seems trivial or was answered somewhere else in a way that I maybe didn't understand as a solution ...
- 21
0
votes
1
answer
98
views
create an efficient dataset to feed a keras siamese model
I need help to build a generator for a Siamese model in Keras and Tensorflow. I succeeded in creating a python generator but it resulted too slow for my purposes. My idea was to use Tensorflow ...
- 11
0
votes
1
answer
124
views
tf.data.dataset next(iter()) yields the same value
I am trying to build a datapipeline for inference on some video files. The videos are alot in number hence, I am using the the tf.data.dataset pipeline with a from_generator method to create ...
- 51