How to prepare a dataset for Keras?

Question

Motivation

To run a set of labeled vectors through Keras neural network.

Example

Looking at Keras dataset example mnist:

keras.datasets import mnist

(x_tr, y_tr), (x_te, y_te) = mnist.load_data()

print x_tr.shape

It seem to be a 3 dimensional numpy array:

(60000, 28, 28)

1st dimension is for the samples

2nd and 3rd for each sample features

Attempt

Building the labeled vectors:

X_train = numpy.array([[1] * 128] * (10 ** 4) + [[0] * 128] * (10 ** 4))

X_test = numpy.array([[1] * 128] * (10 ** 2) + [[0] * 128] * (10 ** 2))

Y_train = numpy.array([True] * (10 ** 4) + [False] * (10 ** 4))

Y_test = numpy.array([True] * (10 ** 2) + [False] * (10 ** 2))

X_train = X_train.astype("float32")

X_test = X_test.astype("float32")

Y_train = Y_train.astype("bool")

Y_test = Y_test.astype("bool")

The training code

model = Sequential()

model.add(Dense(128, 50))

model.add(Activation('relu'))

model.add(Dropout(0.2))

model.add(Dense(50, 50))

model.add(Activation('relu'))

model.add(Dropout(0.2))

model.add(Dense(50, 1))

model.add(Activation('softmax'))

rms = RMSprop()

model.compile(loss='binary_crossentropy', optimizer=rms)

model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,

          show_accuracy=True, verbose=2, validation_data=(X_test, Y_test))

score = model.evaluate(X_test, Y_test, show_accuracy=True, verbose=0)

print('Test score:', score[0])

print('Test accuracy:', score[1])

Result

Test score: 13.9705320154

Test accuracy: 1.0

Why do I get such a bad result for such a simple dataset? Is my dataset malformed?

Thanks!

Answer 1

A softmax on just one output node doesn't make much sense. If you change model.add(Activation('softmax')) to model.add(Activation('sigmoid')).

You can also use two output nodes, where 1, 0 represents the case of True and 0, 1 represents the case of False. Then you can use a softmax layer. You just have to change your Y_train and Y_test accordingly.

Hope this answer helps you!