Explore Courses Blog Tutorials Interview Questions
0 votes
in Machine Learning by (19k points)

I'm learning how to create convolutional neural networks using Keras. I'm trying to get a high accuracy for the MNIST dataset.

Apparently categorical_crossentropy is for more than 2 classes and binary_crossentropy is for 2 classes. Since there are 10 digits, I should be using categorical_crossentropy. However, after training and testing dozens of models, binary_crossentropy consistently outperforms categorical_crossentropy significantly.

On Kaggle, I got 99+% accuracy using binary_crossentropy and 10 epochs. Meanwhile, I can't get above 97% using categorical_crossentropy, even using 30 epochs (which isn't much, but I don't have a GPU, so training takes forever).

Here's what my model looks like now:

model = Sequential()

model.add(Convolution2D(100, 5, 5, border_mode='valid', input_shape=(28, 28, 1), init='glorot_uniform', activation='relu'))

model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Convolution2D(100, 3, 3, init='glorot_uniform', activation='relu'))

model.add(MaxPooling2D(pool_size=(2, 2)))



model.add(Dense(100, init='glorot_uniform', activation='relu'))


model.add(Dense(100, init='glorot_uniform', activation='relu'))


model.add(Dense(10, init='glorot_uniform', activation='softmax'))

model.compile(loss='binary_crossentropy', optimizer='adamax', metrics=['accuracy'])

1 Answer

0 votes
by (33.1k points)

You should simply try to calculate the accuracy manually, and you will see that it is different from the one reported by Keras with the model.evaluate method.

For example:

score = model.evaluate(x_test, y_test, verbose=0) 


# 0.99794011611938471

# Actual accuracy calculated manually:

import numpy as np

y_pred = model.predict(x_test)

acc = sum([np.argmax(y_test[i])==np.argmax(y_pred[i]) for i in range(10000)])/10000


# 0.98999999999999999

Hope this answer helps you! For more details, go through Machine Learning Online Course. Also, study the Tensorflow Tutorial for more knowledge.

Browse Categories