neuralnet prediction returns the same values for all predictions

Question

I'm trying to build a neural net with the neuralnet package and I'm having some trouble with it. I've been successful with the nnet package but no luck with the neuralnet one. I have read the whole documentation package and can't find the solution, or maybe I'm not able to spot it.

The training command I'm using is

nn<-neuralnet(V15 ~ V1 + V2 + V3 + V4 + V5 + V6 + V7 + V8 + V9 + V10 + V11 + V12 + V13 + V14,data=test.matrix,lifesign="full",lifesign.step=100,hidden=8)

and for prediction

result<- compute(nn,data.matrix)$net.result

The training takes a whole lot longer than the nnet training. I have tried using the same algorithm as nnet (backpropagation instead of resilient backpropagation) and nothing, changed the activation function too (and the linear.output=F) and pretty much everything else, and the result didn't improve. Predicted values are all the same. I don't understand why the nnet works for me, while the neuralnet one doesn't.

I could really use some help, my (lack of) understanding of both things (neural nets and R) it's probably the cause, but can't find why.

My dataset is from UCI. I want to use the neural network for binary classification. A sample of the data would be:

25,Private,226802,11th,7,Never-married,Machine-op-inspct,Own-child,Black,Male,0,0,40,United-States,<=50K. 38,Private,89814,HS-grad,9,Married-civ-spouse,Farming-fishing,Husband,White,Male,0,0,50,United-States,<=50K. 28,Local-gov,336951,Assoc-acdm,12,Married-civ-spouse,Protective-serv,Husband,White,Male,0,0,40,United-States,>50K. 44,Private,160323,Some-college,10,Married-civ-spouse,Machine-op-inspct,Husband,Black,Male,7688,0,40,United-States,>50K. 18,?,103497,Some-college,10,Never-married,NA,Own-child,White,Female,0,0,30,United-States,<=50K. 34,Private,198693,10th,6,Never-married,Other-service,Not-in-family,White,Male,0,0,30,United-States,<=50K. 29,?,227026,HS-grad,9,Never-married,?,Unmarried,Black,Male,0,0,40,United-States,<=50K. 63,Self-emp-not-inc,104626,Prof-school,15,Married-civ-spouse,Prof-specialty,Husband,White,Male,3103,0,32,United-States,>50K. 24,Private,369667,Some-college,10,Never-married,Other-service,Unmarried,White,Female,0,0,40,United-States,<=50K. 55,Private,104996,7th-8th,4,Married-civ-spouse,Craft-repair,Husband,White,Male,0,0,10,United-States,<=50K. 65,Private,184454,HS-grad,9,Married-civ-spouse,Machine-op-inspct,Husband,White,Male,6418,0,40,United-States,>50K. 36,Federal-gov,212465,Bachelors,13,Married-civ-spouse,Adm-clerical,Husband,White,Male,0,0,40,United-States,<=50K. 26,Private,82091,HS-grad,9,Never-married,Adm-clerical,Not-in-family,White,Female,0,0,39,United-States,<=50K.

Converted into a matrix, with the factors as numerical values:

Summary of the predicted values:

The value of the Wilcoxon-Mann-Whitney test (area under the curve) shows that the prediction performance is virtually the same as a random.

performance(predneural,"auc") @y.values [1] 0.5013319126

Answer 1

The first reason is the normalization when you get weird results with neural networks. Your data must be normalized, otherwise, the training will result in skewed NN(neural network) which will produce the same outcome all the time, it is the most common symptom of the error.

Seeing your dataset, there are values that are >>1 which means they are all treated by NN essentially the same. The reason behind that is the traditionally used response functions are (almost) constant outside some range around 0.

Note that always normalize your data before feeding it into a neural network.