How to give a sequece of numpy array as input to CNN

Question

How can I reshape a sequence of arrays of shape (90,30,1662)? Meaning 90 arrays with 30 frames each and 1662 keypoints for each frames.And 90 array meaning 30 videos of numpy arrays for a word with 30 frames per video.

x_train, x_test, y_train, y_test=train_test_split(x, y, test_size=0.05)
x_train.shape ---->(85, 30, 1662)


from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense
from tensorflow.keras.callbacks import TensorBoard


model = Sequential()
model.add(LSTM(64, return_sequences=True, activation='relu', input_shape=(30,1662)))
model.add(LSTM(128, return_sequences=True, activation='relu'))
model.add(LSTM(64, return_sequences=False, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(actions.shape[0], activation='softmax'))

How can I add CNN before the LSTM?

medium-dimensional · Accepted Answer · 2022-05-23 15:50:32Z

1

Reference: https://machinelearningmastery.com/cnn-long-short-term-memory-networks/

You can add CNN model first(with input shape=(30,90,1622)), and use LTSM model to encapsulate CNN model.

It will look like this:

cnn = Sequential()
cnn.add(Conv2D(your output size, (your filter size,your filter size), 
    activation='relu', padding='same', input_shape=(30,90,1622)))
cnn.add(MaxPooling2D(pool_size=(2, 2)))
cnn.add(Flatten())
model = Sequential()
model.add(TimeDistributed(cnn, ...)) # convert to LTSM type
model.add(LSTM(..))
model.add(Dense(...))

edited May 23, 2022 at 15:50

medium-dimensional

2,30114 silver badges25 bronze badges

answered May 22, 2022 at 8:04

蔡東霖

113 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Israel mehari Over a year ago

I tried what u requested

Israel mehari Over a year ago

model = Sequential() model.add(TimeDistributed(Conv2D(3, (2,2), activation='relu', padding='same', input_shape= (30,90,1622)))) model.add(TimeDistributed(MaxPooling2D(pool_size=(2,2)))) model.add(TimeDistributed(Flatten())) model.add(LSTM(64, return_sequences=True, activation='relu')) model.add(LSTM(128, return_sequences=True, activation='relu')) model.add(LSTM(64, return_sequences=False, activation='relu')) model.add(Dense(64, activation='relu')) model.add(Dense(32, activation='relu')) model.add(Dense(actions.shape[0], activation='softmax'))

Israel mehari Over a year ago

am getting an error ValueError: Input 0 of layer conv2d_3 is incompatible with the layer: : expected min_ndim=4, found ndim=2. Full shape received: (None, 1662)

蔡東霖 Over a year ago

check this: keras.io/api/layers/recurrent_layers/time_distributed You may build conv2d layer before going time distributed function.

Collectives™ on Stack Overflow

How to give a sequece of numpy array as input to CNN

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related