How to concatenate two layers in keras?

Question

I have an example of a neural network with two layers. The first layer takes two arguments and has one output. The second should take one argument as result of the first layer and one additional argument. It should looks like this:

x1  x2 x3

 \  / /

  y1   /

   \  /

    y2

So, I'd created a model with two layers and tried to merge them but it returns an error: The first layer in a Sequential model must get an "input_shape" or "batch_input_shape" argument. on the line result.add(merged).

Model:

first = Sequential()

first.add(Dense(1, input_shape=(2,), activation='sigmoid'))

second = Sequential()

second.add(Dense(1, input_shape=(1,), activation='sigmoid'))

result = Sequential()

merged = Concatenate([first, second])

ada_grad = Adagrad(lr=0.1, epsilon=1e-08, decay=0.0)

result.add(merged)

result.compile(optimizer=ada_grad, loss=_loss_tensor, metrics=['accuracy'])

Comments

how to run model.fit when we have two models

Answer 1

As I can understand your problem, the main reason for this error is the result defined as Sequential() is just a container for the model and you have not defined input for it. 

It looks like you’re trying to build a set result to take the third input x3.

Improvised code for your problem:

first = Sequential()

first.add(Dense(1, input_shape=(2,), activation='sigmoid'))

second = Sequential()

second.add(Dense(1, input_shape=(1,), activation='sigmoid'))

third = Sequential()

# provide the input to result with will be your x3

third.add(Dense(1, input_shape=(1,), activation='sigmoid'))

#then add a few more layers to first and second.

# concatenate them

merged = Concatenate([first, second])

# then concatenate the two outputs

result = Concatenate([merged,  third])

ada_grad = Adagrad(lr=0.1, epsilon=1e-08, decay=0.0)

result.compile(optimizer=ada_grad, loss='binary_crossentropy',

               metrics=['accuracy'])

You can try another way of building a model that this type of input structure would be to use the functional API.

For example:

from keras.models import Model

from keras.layers import Concatenate, Dense, LSTM, Input, concatenate

from keras.optimizers import Adagrad

first_input = Input(shape=(2, ))

first_dense = Dense(1, )(first_input)

second_input = Input(shape=(2, ))

second_dense = Dense(1, )(second_input)

merge_one = concatenate([first_dense, second_dense])

third_input = Input(shape=(1, ))

merge_two = concatenate([merge_one, third_input])

model = Model(inputs=[first_input, second_input, third_input], outputs=merge_two)

ada_grad = Adagrad(lr=0.1, epsilon=1e-08, decay=0.0)

model.compile(optimizer=ada_grad, loss='binary_crossentropy',

               metrics=['accuracy'])

Concatenation works like this:

  a        b         c

a b c   g h i a b c g h i

d e f   j k l d e f j k l

i.e rows are just joined.

2) You can say that x1 is input to first, x2 is input into second and x3 input into third.

I hope this solution solved your problem. 

If you wish to learn more about Machine Learning, visit Machine Learning tutorial and Machine Learning course by Intellipaat.