Keras Masking Output Layer

Question

I have the following Model in Keras:

main_input = Input(shape=(None, 2, 100, 100), dtype='float32', name='input')

hidden = ConvLSTM2D(filters=16, 
                    kernel_size=(5, 5),  
                    padding='same', 
                    return_sequences=False, 
                    data_format='channels_first')(main_input)

output = Conv2D(filters=1, 
                kernel_size=(1, 1), 
                padding='same',
                activation='sigmoid',
                kernel_initializer='glorot_uniform',
                data_format='channels_first',
                name='output')(hidden)

sgd = SGD(lr=0.002, momentum=0.0, decay=0.0, nesterov=False)

I want to multiply the output, which is a 2d array, by a mask (there is a separate mask for each example). How can I do this in Keras?

hhz · Accepted Answer · 2019-04-15 01:23:55Z

I think you should input the mask of each sample to the model at the same time.

Here is the suggested code:

from keras.layers import Multiply

main_input = Input(shape=(None, 2, 100, 100), dtype='float32', name='input')

mask=Input(shape=(1, 100, 100), dtype='float32', name='mask')

hidden = ConvLSTM2D(filters=16, 
                    kernel_size=(5, 5),  
                    padding='same', 
                    return_sequences=False, 
                    data_format='channels_first')(main_input)

output = Conv2D(filters=1, 
                kernel_size=(1, 1), 
                padding='same',
                activation='sigmoid',
                kernel_initializer='glorot_uniform',
                data_format='channels_first',
                name='output')(hidden)
output_with_mask=Multiply()([output, mask])

model=Model([main_input, mask], output_with_mask)

The summary is as follow:

    __________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input (InputLayer)              (None, None, 2, 100, 0                                            
__________________________________________________________________________________________________
conv_lst_m2d_7 (ConvLSTM2D)     (None, 16, 100, 100) 28864       input[0][0]                      
__________________________________________________________________________________________________
output (Conv2D)                 (None, 1, 100, 100)  17          conv_lst_m2d_7[0][0]             
__________________________________________________________________________________________________
mask (InputLayer)               (None, 1, 100, 100)  0                                            
__________________________________________________________________________________________________
multiply_7 (Multiply)           (None, 1, 100, 100)  0           output[0][0]                     
                                                                 mask[0][0]                       
==================================================================================================
Total params: 28,881
Trainable params: 28,881
Non-trainable params: 0
__________________________________________________________________________________________________

Hi, one quick question. When I ran the code I got "ValueError: Error when checking input: expected mask to have 5 dimensions, but got array with shape (649, 1, 100, 100)"

Richard X · Accepted Answer · 2020-01-11 05:37:46Z

Making this work with tensorflow 2.0 and tf.keras.

import tensorflow as tf
from tensorflow.keras.layers import Multiply, Conv2D, ConvLSTM2D, Input

main_input = Input(shape=(None, 2, 100, 100), dtype='float32', name='input')

mask=Input(shape=(1, 100, 100), dtype='float32', name='mask')

hidden = ConvLSTM2D(filters=16, 
                    kernel_size=(5, 5),  
                    padding='same', 
                    return_sequences=False, 
                    data_format='channels_first')(main_input)

output = Conv2D(filters=1, 
                kernel_size=(1, 1), 
                padding='same',
                activation='sigmoid',
                kernel_initializer='glorot_uniform',
                data_format='channels_first',
                name='output')(hidden)
output_with_mask=Multiply()([output, mask])

lupaulus · Accepted Answer · 2019-04-13 20:51:24Z

0

Creating an new output and use your old output as second hiden layer.

You want to make an second convolution (with an spécial mask) on your "old output" to get your new output

Hope it will help you

answered Apr 13, 2019 at 20:51

lupaulus

4071 gold badge9 silver badges29 bronze badges

3 Comments

Rehaan Ahmad Over a year ago

How do I specify what filter I want to use with the convolution?

Rehaan Ahmad Over a year ago

Also, I don't want to convolve - rather I'm simply multiplying two arrays elementwise (not summing together like a convolution)

lupaulus Over a year ago

One solution I see is to use the numpy library to multiply your 2D array like an matrix

Collectives™ on Stack Overflow

Keras Masking Output Layer

3 Answers 3

2 Comments

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related