For several days now, I am trying to build a simple sine-wave sequence generation using LSTM, without any glimpse of success so far.
I started from the time sequence prediction example
All what I wanted to do differently is:
- Use different optimizers (e.g RMSprob) than LBFGS
- Try different signals (more sine-wave components)
This is the link to my code. "experiment.py" is the main file
What I do is:
- I generate artificial time-series data (sine waves)
- I cut those time-series data into small sequences
- The input to my model is a sequence of time 0...T, and the output is a sequence of time 1...T+1
What happens is:
- The training and the validation losses goes down smoothly
- The test loss is very low
- However, when I try to generate arbitrary-length sequences, starting from a seed (a random sequence from the test data), everything goes wrong. The output always flats out
I simply don't see what the problem is. I am playing with this for a week now, with no progress in sight. I would be very grateful for any help.