I found several questions related to this, but no one solved my doubts. In particular, the two answers to this question confused me even more.
I'm training a linear SVM on top of a set of features - Convolutional Neural Net features resulting from images. I have, for example, a 3500x4096 X matrix with examples on rows and features on columns, as usual.
I'm wondering how to properly standardize/normalize this matrix before feeding the SVM. I see two ways (using sklearn):
Standardizing features. It results in features with 0 mean and unitary std.
X = sklearn.preprocessing.scale(X)
Normalizing features. It results in features with unitary norm.
X = sklearn.preprocessing.normalize(X, axis=0)
My results are sensibly better with normalization (76% accuracy) than with standarding (68% accuracy).
Is it a completely dataset-dependent choice? Or how can one choose between the two techniques?