concatenate (merge) layer keras with tensorflow

Question

I'd like to make a model as following.

input data    input data
    |             | 
 convnet1      convet2
    |             |
maxpooling    maxpooling
    |             |
    - Dense layer -
           |
      Dense layer

So, I've wrote following code.

model1 = Sequential()
model1.add(Conv2D(32, (3, 3), activation='relu', input_shape=(bands, frames, 1)))
print(model1.output_shape)
model1.add(MaxPooling2D(pool_size=(2, 2)))
model1.add(Flatten())

model2 = Sequential()
model2.add(Conv2D(32, (9, 9), activation='relu', input_shape=(bands, frames, 1)))
print(model2.output_shape)
model2.add(MaxPooling2D(pool_size=(4, 4)))
model2.add(Flatten())

modelall = Sequential()
modelall.add(concatenate([model1, model2], axis=1))
modelall.add(Dense(100, activation='relu'))

modelall.add(Dropout(0.5))
modelall.add(Dense(10, activation='softmax')) #number of class = 10
print(modelall.output_shape)

modelall.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

modelall.fit([X_train, X_train], y_train, batch_size=batch_size, nb_epoch=training_epochs)
score = modelall.evaluate(X_test, y_test, batch_size=batch_size)

However, I got an error.

AttributeError: 'Sequential' object has no attribute 'get_shape'

The whole error traceback as follows.

  Traceback (most recent call last):
  File "D:/keras/cnn-keras.py", line 54, in <module>
    model.add(concatenate([modelf, modelt], axis=1))
  File "C:\Users\Anaconda3\lib\site-packages\keras\layers\merge.py", line 508, in concatenate
    return Concatenate(axis=axis, **kwargs)(inputs)
  File "C:\Users\Anaconda3\lib\site-packages\keras\engine\topology.py", line 542, in __call__
    input_shapes.append(K.int_shape(x_elem))
  File "C:\Users\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py", line 411, in int_shape
    shape = x.get_shape()
AttributeError: 'Sequential' object has no attribute 'get_shape'

Is the error caused by tensorflow? Any idea on how to fix it?

shouldn't it be modelall.add(concatenate([model1, model2], axis=-1))? Channels come last in TensorFlow. — Autonomous
– Autonomous, Commented May 18, 2017 at 8:54
It does not seem to be the cause of the error. I tried what you said, but I got a same error. — HS Cho
– HS Cho, Commented May 18, 2017 at 9:14
Also, since you are concatenating sequential models and not tensors, I think you should use Concatenate (with a capital C). — Autonomous
– Autonomous, Commented May 18, 2017 at 9:18
I imported 'concatenate' from keras layer like this. 'from keras.layers import concatenate'. But still get the same error... — HS Cho
– HS Cho, Commented May 18, 2017 at 9:37
You imported concatenate and not Concatenate. Try from keras.layers import Concatenate, or simply use the Model class. However, I think multi-input single output should be possible in Sequential(). — Autonomous
– Autonomous, Commented May 18, 2017 at 20:34

Daniel Möller · Accepted Answer · 2017-05-18 13:06:41Z

6

You cannot use a Sequential model for creating branches, that doesn't work.

You must use the functional API for that:

from keras.models import Model    
from keras.layers import *

It's ok to have each branch as a sequential model, but the fork must be in a Model.

#in the functional API you create layers and call them passing tensors to get their output:

conc = Concatenate()([model1.output, model2.output])

    #notice you concatenate outputs, which are tensors.     
    #you cannot concatenate models


out = Dense(100, activation='relu')(conc)
out = Dropout(0.5)(out)
out = Dense(10, activation='softmax')(out)

modelall = Model([model1.input, model2.input], out)

It wasn't necessary here, but usually you create Input layers in the functional API:

inp = Input((shape of the input))
out = SomeLayer(blbalbalba)(inp)
....
model = Model(inp,out)

answered May 18, 2017 at 13:06

Daniel Möller

86.8k24 gold badges202 silver badges222 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Autonomous Over a year ago

I think multiple input one output is still okay with Sequential().

Daniel Möller Over a year ago

No, because you won't have a model from start to end to train it.

HS Cho Over a year ago

Thank you Daniel. I fixed the problem with your advise. And also thank you Chandakkar.

Daniel Möller Over a year ago

:) -- Would you check it as a valid answer then?

Collectives™ on Stack Overflow

concatenate (merge) layer keras with tensorflow

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related