Training neural network in Ruby

Question

I'm a total beginner when it comes to neural networks. I've been wrestling with ruby-fann and ai4r all day and unfortunately, I don't have anything to show for it, so I figured I would come onto Stack Overflow and ask the knowledgeable people here.

I have a set of samples -- each day has one data point, but they don't fit any clear pattern that I've been able to figure out (I tried a couple of regressions). Still, I think it would be neat to see if there was any way to predict the data going into the future just from the date, and I thought a neural network would be a good way to generate a function that could hope to express that relationship.

The dates are DateTime objects and the data points are decimal numbers, like 7.68. I've been converting the DateTime objects to floats and then dividing by 10,000,000,000 to get a number between 0 and 1, and I've been dividing the decimal numbers by 1,000 to also get a number between 0 and 1. I have over a thousand samples... here's what a short excerpt looks like:

[

  ["2012-03-15", "7.68"],

  ["2012-03-14", "4.221"],

  ["2012-03-13", "12.212"],

  ["2012-03-12", "42.1"]

]

Which when transformed looks like this:

[

  [0.13317696, 0.000768],

  [0.13316832, 0.0004221],

  [0.13315968, 0.0012212],

  [0.13315104, 0.00421]

]

I kind of wish this transformation wasn't necessary, but I digress. The problem is that both ai4r and ruby-fann return one constant number, generally something in the middle of the range of the samples, when I run them. Here's the code for ruby-fann:

@fann = RubyFann::Standard.new(:num_inputs=>1, :hidden_neurons=>[3, 3], :num_outputs=>1)

training_data = RubyFann::TrainData.new(:inputs => formatted_data.collect{|d| [d.first]}, :desired_outputs => formatted_data.collect{|d| [d.last]})

@fann.train_on_data(training_data, 1000, 1, 0.0001)

@fann.run([DateTime.now.to_f / 10000000000.0]) # Always something random, and always the same number no matter what date I request it for

And for ai4r:

@ai4r = Ai4r::NeuralNetwork::Backpropagation.new([1, 3, 3, 1])

1000.times do

  formatted_data.each do |data|

    @ai4r.train(data.first, data.last)

  end

end

@ai4r.eval([DateTime.now.to_f / 10000000000.0]) # A different result from above, but always seemingly random and the same for any requested date

I feel like I'm missing something really basic here. I know this is a rather open-ended question but if anyone could help me figure out how I'm improperly teaching my neural networks, I'd really appreciate it!

Answer 1

One of your obstacles is that you are dealing with way too small numbers. When I set the desired error to 0.0001 I can't even predict the training data you specified above correctly. You should think of any other way to encode your input and output. 

It depends on the problem, but if the day's value is even somewhat a function of the previous days' values, treating this as a time series might generate better results.

You would then instead teach the NN to produce the day's value as a function of a window of, say, the previous ten days values; you could also keep the date parameter as a real input scale between [0, 1] as you consider it has a significant effect on the day's value.

If you wish to know about Neural Network then visit this Neural Network Tutorial.