Keras model.predict always predicts 1

Question

I'm working on some Artificial Intelligence project and I want to predict the bitcoin trend but while using the model. predict function from Keras with my test_set, the prediction is always equal to 1 and the line in my diagram is therefore always straight.

import csv

import matplotlib.pyplot as plt

import numpy as np

import pandas as pd

from cryptory import Cryptory

from keras.models import Sequential, Model, InputLayer

from keras.layers import LSTM, Dropout, Dense

from sklearn.preprocessing import MinMaxScaler

def format_to_3d(df_to_reshape):

    reshaped_df = np.array(df_to_reshape)

    return np.reshape(reshaped_df, (reshaped_df.shape[0], 1, reshaped_df.shape[1]))

crypto_data = Cryptory(from_date = "2014-01-01")

bitcoin_data = crypto_data.extract_coinmarketcap("bitcoin")

sc = MinMaxScaler()

for col in bitcoin_data.columns:

    if col != "open":

        del bitcoin_data[col]

training_set = bitcoin_data;

training_set = sc.fit_transform(training_set)

# Split the data into train, validate and test

train_data = training_set[365:]

# Split the data into x and y

x_train, y_train = train_data[:len(train_data)-1], train_data[1:]

model = Sequential()

model.add(LSTM(units=4, input_shape=(None, 1))) # 128 -- neurons**?

# model.add(Dropout(0.2))

model.add(Dense(units=1, activation="softmax"))  # activation function could be different

model.compile(optimizer="adam", loss="mean_squared_error")  # mse could be used for loss, look into optimiser

model.fit(format_to_3d(x_train), y_train, batch_size=32, epochs=15)

test_set = bitcoin_data

test_set = sc.transform(test_set)

test_data = test_set[:364]

input = test_data

input = sc.inverse_transform(input)

input = np.reshape(input, (364, 1, 1))

predicted_result = model.predict(input)

print(predicted_result)

real_value = sc.inverse_transform(input)

plt.plot(real_value, color='pink', label='Real Price')

plt.plot(predicted_result, color='blue', label='Predicted Price')

plt.title('Bitcoin Prediction')

plt.xlabel('Time')

plt.ylabel('Prices')

plt.legend()

plt.show()

The training set performance looks like this:

1566/1566 [==============================] - 3s 2ms/step - loss: 0.8572

Epoch 2/15

1566/1566 [==============================] - 1s 406us/step - loss: 0.8572

Epoch 3/15

1566/1566 [==============================] - 1s 388us/step - loss: 0.8572

Epoch 4/15

1566/1566 [==============================] - 1s 388us/step - loss: 0.8572

Epoch 5/15

1566/1566 [==============================] - 1s 389us/step - loss: 0.8572

Epoch 6/15

1566/1566 [==============================] - 1s 392us/step - loss: 0.8572

Epoch 7/15

1566/1566 [==============================] - 1s 408us/step - loss: 0.8572

Epoch 8/15

1566/1566 [==============================] - 1s 459us/step - loss: 0.8572

Epoch 9/15

1566/1566 [==============================] - 1s 400us/step - loss: 0.8572

Epoch 10/15

1566/1566 [==============================] - 1s 410us/step - loss: 0.8572

Epoch 11/15

1566/1566 [==============================] - 1s 395us/step - loss: 0.8572

Epoch 12/15

1566/1566 [==============================] - 1s 386us/step - loss: 0.8572

Epoch 13/15

1566/1566 [==============================] - 1s 385us/step - loss: 0.8572

Epoch 14/15

1566/1566 [==============================] - 1s 393us/step - loss: 0.8572

Epoch 15/15

1566/1566 [==============================] - 1s 397us/step - loss: 0.8572

I'm supposed to print a plot with the Real Price and the Predicted Price, the Real Price is displayed properly but the Predicted price is only a straight line because of that model. predict that only contains the value 1.

Thanks in advance!

Answer 1

Actually, you are trying to predict a price value means that you're aiming at solving a regression problem and not a classification problem.

However, in the previous layer of your network 

(model.add(Dense(units=1, activation=" softmax")))

you have a single neuron (which would be adequate for a regression problem), but you've chosen to use a softmax activation function. The softmax function is used in multi-class classification problems, to normalize the outputs into a probability distribution. If you have a single output neuron and you apply softmax, the final result will always 1.0, as it is the only parameter of the probability distribution.

In short, for regression problems, you do not use an activation function, as the network is intended to already output the predicted value.

If you wish to know more about Keras Model then visit this Artificial Intelligence Course.

Comments

I created an account just to respond to this. Your explanation was crystal clear and solved my problem, thank you so much.