Unable to approximate the sine function using a neural network

Question

I am trying to approximate the sine() function using a neural network I wrote myself. I have tested my neural network on a simple OCR problem already and it worked, but I am having trouble applying it to approximate sine(). My problem is that during training my error converges on exactly 50%, so I'm guessing it's completely random.

I am using one input neuron for the input (0 to PI), and one output neuron for the result. I have a single hidden layer in which I can vary the number of neurons but I'm currently trying around 6-10.

I have a feeling the problem is because I am using the sigmoid transfer function (which is a requirement in my application) which only outputs between 0 and 1, while the output for sine() is between -1 and 1. To try to correct this I tried multiplying the output by 2 and then subtracting 1, but this didn't fix the problem. I'm thinking I have to do some kind of conversion somewhere to make this work.

Any ideas?

Answer 1

Use a linear output unit:

R code:

set.seed(1405)

x <- sort(10*runif(50))

y <- sin(x) + 0.2*rnorm(x)

library(nnet)

nn <- nnet(x, y, size=6, maxit=40, linout=TRUE)

plot(x, y)

plot(sin, 0, 10, add=TRUE)

x1 <- seq(0, 10, by=0.1)

lines(x1, predict(nn, data.frame(x=x1)), col="green")

Hope this answer helps.