Artificial neural network with Ruby

Question

I'm trying to use some artificial neural network with Ruby, I tried few gems (ruby-fann, Ai4r) and both of them when I run trained network on some test inputs always returns me same thing [1.0, 1.0], but in training data all outputs are around 100. I have 10 input neurons, and 2 output. I tried different network configurations, desire error, with different number of layers and different number of neurons. While training ruby-fann's NN I get this:

1. Current error: 10034.23571597 Bit fail: 1938

100. Current error: 9937.62571597 Bit fail: 1938

After that all errors and fails are the same. Here is my code:

inputs_to_learn = []

outputs_to_learn = []

some cycle:

    inputs_to_learn.push([i1,i2,i3,i4,i5,i6,i7,i8,i9,i10,])

    outputs_to_learn.push([o1, o2])

train = RubyFann::TrainData.new(:inputs=>inputs_to_learn, :desired_outputs=>outputs_to_learn)

fann = RubyFann::Standard.new(:num_inputs=>10, :hidden_neurons=>[30, 30, 30, 30], :num_outputs=>2) 

fann.train_on_data(train, 1000, 100, 0.5)

result = fann.run([100, 100, 95, 105, 95, 105, 95, 105, 95, 105])

So the question is, why do I get 1.0 if train data is 100 times bigger numbers.

Answer 1

There are many choices available when creating a neural network. One important alternative is the transfer function used to the total of all inputs times weights on every neuron. This may likewise be called the activation function.

This assumes you mainly on the output layer. For the networks, values of 100 are not possible in the output, so the network cannot correctly learn them.

It is very common to see networks using the logistic function (or sigmoid), that always outputs between 0.0 and 1.0. With some NN libraries, that might be the only option you have.

You have two basic options to advance the convergence of your network here:

1) See the documentation for the library you are working with, and see if it is likely to have a linear output layer. This could be a good option for other purposes if you are trying to create a regression model on a consecutive variable.

2) Compare your output variable to fit the available output range of 0.0..1.0. You will need to map your training data to this extent, and then reverse that mapping when evaluating the output, using the network for new input data.

For best production, ideally, you should also scale the inputs, based on your training data. Typically you normalize them to have mean 0.0, standard deviation 1.0 on the training set, although limiting range to 0.0..1.0 or -1.0..1.0 is rather common too.