Merge multiple CNNs

Question

I am trying to perform Conv1D on multiple inputs in my model. So I have 15 inputs of size 1x1500 each, where each one is an input to a series of layers. So I have 15 convolutional models which I want to merge before Fully Connected Layer. I have defined the convolutional model in a function, but I cannot understand how to call the function and then merge them.

def defineModel(nkernels, nstrides, dropout, input_shape):
    model = Sequential()
    model.add(Conv1D(nkernels, nstrides, activation='relu', input_shape=input_shape))
    model.add(Conv1D(nkernels*2, nstrides, activation='relu'))
    model.add(BatchNormalization())
    model.add(MaxPooling1D(nstrides))
    model.add(Dropout(dropout))
    return model


models = {}
for i in range(15):
    models[i] = defineModel(64,2,0.75,(64,1))

I have successfully concatenated 4 models as follows:

merged = Concatenate()([ model1.output, model2.output, model3.output, model4.output])

merged = Dense(512, activation='relu')(merged)
merged = Dropout(0.75)(merged)
merged = Dense(1024, activation='relu')(merged)
merged = Dropout(0.75)(merged)
merged = Dense(40, activation='softmax')(merged)
model = Model(inputs=[model1.input, model2.input, model3.input, model4.input], outputs=merged)

How do I do it for 15 layers in the for loop as writing 15 layers separately isn't efficient?

Are the multi inputs you are using related to each other? Are they supposed to be fed to their respective models in parallel ? Also, are you planning to train the model separately or simultaneously ? — Wazaki
– Wazaki, Commented Oct 17, 2018 at 6:34

today · Accepted Answer · 2018-10-17 09:56:41Z

2

Of course, as @GabrielM suggested, using functional API is the best way to do this, however if you don't want to modify the define_model function you can also do it like this:

models = []
inputs = []
outputs = []
for i in range(15):
    model = defineModel(64,2,0.75,(64,1))
    models.append(model)
    inputs.append(model.input)
    outputs.append(model.output)


merged = Concatenate()(outputs) # this should be output tensors and not models

# the rest is the same ...

model = Model(inputs=inputs, outputs=merged)

edited Oct 17, 2018 at 9:56

answered Oct 17, 2018 at 9:49

today

33.6k8 gold badges100 silver badges122 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Gabriel M · Accepted Answer · 2018-10-17 07:13:08Z

1

I think the best you can do is use the functional API everywhere:

def defineModel(nkernels, nstrides, dropout, input_shape):
    l_input = Input( shape=input_shape )
    model = Conv1D(nkernels, nstrides, activation='relu')(l_input)
    model = Conv1D(nkernels*2, nstrides, activation='relu')(model)
    model = BatchNormalization()(model)
    model = MaxPooling1D(nstrides)(model)
    model = Dropout(dropout)(model)
    return model, l_input


models = []
inputs = []
for i in range(15):
    model, input = defineModel(64,2,0.75,(64,1))
    models.append( model )
    inputs.append( input )

Then it is easy to recover the lists of inputs and the outputs of the submodels and merge them

merged = Concatenate()(models)

merged = Dense(512, activation='relu')(merged)
merged = Dropout(0.75)(merged)
merged = Dense(1024, activation='relu')(merged)
merged = Dropout(0.75)(merged)
merged = Dense(40, activation='softmax')(merged)
model = Model(inputs=inputs, outputs=merged)

Normally, these operations are not a bottle neck. None of this should have a major impact during training or inference

answered Oct 17, 2018 at 7:13

Gabriel M

1,5144 gold badges17 silver badges25 bronze badges

4 Comments

Gala Over a year ago

Works really fine. Thank you!!

Gala Over a year ago

I have another error which goes like this Error when checking target: expected dense_3 to have 3 dimensions, but got array with shape (240, 40) , I have written the code as you suggested and then my next line is as follows model.fit(xx[0:15], label, validation_split=0.2, batch_size=25, epochs=30). Here label is of size 240x40 and input I have is of size 240x15x1500.

Gabriel M Over a year ago

xx is of shape [240,15,1500]?

Gala Over a year ago

No my bad. xx is a list. I am actually getting some input as 240x15x1500 which i want to split as 1x1500 so that I can give the input to model. I have written a code like this xx = np.split(data,15,axis=1)

Collectives™ on Stack Overflow

Merge multiple CNNs

2 Answers 2

Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related