How to choose the right kernel functions

Question

I have a very general question: how do I choose the right kernel function for SVM? I know the ultimate answer is to try all the kernels, do out-of-sample validation, and pick the one with the best classification result. But other than that, is there any guideline for trying the different kernel functions?

Answer 1

The Radial Basis Function kernel makes a good default kernel if there is no expert knowledge present(once you have established it is a problem requiring a non-linear model).

The choice of the kernel and kernel/regularisation parameters can be automated by optimizing cross-validation based model selection (or use the radius-margin or span bounds). The simplest thing to do is to minimize a continuous model selection criterion using the Nelder-Mead simplex method, which doesn't require gradient calculation and works well for sensible numbers of hyper-parameters. If you have more hyper-parameters to tune, automated model selection is likely to result in severe over-fitting, due to the variance of the model selection criterion. you can also use gradient-based optimization, but the performance gain is not usually worth the effort of coding it up.

Automated choice of kernels and kernel/regularization parameters is a tricky issue, as it is very easy to overfit the model selection criterion (typically cross-validation based), and you can end up with a worse model than you started with. The automated model selection also can bias performance evaluation, so make sure your performance evaluation evaluates the whole process of fitting the model (training and model selection).

For knowing how support vector machines use kernel functions to classify data, refer the following link:http://www.simafore.com/blog/bid/113227/How-support-vector-machines-use-kernel-functions-to-classify-data

If you wish to learn about Kernel Function then visit this (Support Vector Machine) SVM Tutorial.