What is Support Vector Machine? SVM Algorithm in Machine Learning

Support Vector Machine or SVM algorithm is a simple yet powerful Supervised Machine Learning algorithm that can be used for building both regression and classification models. SVM algorithm can perform really well with both linearly separable and non-linearly separable datasets. Even with a limited amount of data, the support vector machine algorithm does not fail to show its magic.
SVM2

SVM Figure 1: Linearly Separable and Non-linearly Separable Datasets

Before diving right into understanding the support vector machine algorithm in Machine Learning, let us take a look at the important concepts this blog has to offer.

Interested in learning Machine Learning? Click here to learn more in this Machine Learning Training in Bangalore!

Support Vector Machine Algorithm Example

Support vector machine or SVM algorithm is based on the concept of ‘decision planes’, where hyperplanes are used to classify a set of given objects.
Let us start off with a few pictorial examples of support vector machine algorithm. As we can see in Figure 2, we have two sets of data. These datasets can be separated easily with the help of a line, called a decision boundary.
SVM3

SVM Figure 2: Decision Boundary

But there can be several decision boundaries that can divide the data points without any errors. For example, in Figure 3, all decision boundaries classify the datasets correctly. But how do we pick the best decision boundary?
SVM44

SVM Figure 3: Other Possible Decision Boundaries

Well, here’s the tip: the best decision boundary is the one which has maximum distance from the nearest points of these two classes, as shown in Figure 4.
SVM6

SVM Figure 4: Maximum Distance from the Nearest Points

Also remember that the nearest points from the optimal decision boundary that maximize the distance are called support vectors.

SVM Figure 5: Margin and Maximum Margin Classifier

The region that the closest points define around the decision boundary is known as the margin.
That is why the decision boundary of a support vector machine model is known as the maximum margin classifier or the maximum margin hyperplane.

In other words, here’s how a support vector machine algorithm model works:

  • First, it finds lines or boundaries that correctly classify the training dataset.
  • Then, from those lines or boundaries, it picks the one that has the maximum distance from the closest data points.

Alright, in the above support vector machine example, the dataset was linearly separable. Now, the question, how do we classify non-linearly separable datasets as shown in Figure 6?
SVM8

SVM Figure 6: Non-linearly Separable Dataset

Clearly, straight lines can’t be used to classify the above dataset. That is where Kernel SVM comes into the picture.
SVM9

SVM Figure 7: After Using Kernel Support Vector Classifier

What does Kernel SVM do? How does it find the classifier? Well, the Kernel SVM projects the non-linearly separable datasets of lower dimensions to linearly separable data of higher dimensions. Kernel SVM performs the same in such a way that datasets belonging to different classes are allocated to different dimensions. Interesting, isn’t it?
Well, before exploring how to implement SVM in Python programming language, let us take a look at the pros and cons of support vector machine algorithm.

Advantages of Support Vector Machine Algorithm

  • Accuracy
  • Works very well with limited datasets
  • Kernel SVM contains a non-linear transformation function to convert the complicated non-linearly separable data into linearly separable data.

Disadvantages of Support Vector Machine Algorithm

  • Does not work well with larger datasets
  • Sometimes, training time with SVMs can be high

Become Master of Machine Learning by going through this online Machine Learning course in Singapore.

How Does the Support Vector Machine Algorithm Work?

SVM libraries are packed with some popular kernels such as Polynomial, Radial Basis Function or rbf, and Sigmoid. The classification function used in SVM in Machine Learning is SVC. The SVC function looks like this:
sklearn.svm.SVC (C=1.0, kernel= ‘rbf’, degree=3)
Important parameters are:

  • C: Keeping large values of C will indicate the SVM model to choose a smaller margin hyperplane. Small value of C will indicate the SVM model to choose a larger margin hyperplane.
  • kernel: It is the kernel type to be used in SVM model building. It can be ‘linear’, ‘rbf’, ‘poly’, or ‘sigmoid’. The default value of kernel is ‘rbf’.
  • degree: It’s only considered in the case of polynomial kernel. It is the degree of the polynomial kernel function. The default value of degree is 3.

Alright, let us dive right into the hands-on of SVM in Python programming language.

If you have any doubts or queries related to Data Science, do post on Machine Learning Community.

Building a Support Vector Machine Classification Model in Machine Learning Using Python

Problem Statement: Use Machine Learning to predict cases of breast cancer using patient treatment history and health data
Dataset: Breast Cancer Wisconsin (Diagnostic) Dataset
Let us have a quick look at the dataset:
SVM10
Classification Model Building: Support Vector Machine in Python
Let us build the classification model with the help of a Support Vector Machine algorithm.
Step 1: Load Pandas library and the dataset using Pandas
SVM11
Let us have a look at the shape of the dataset:
SVM12
SVM13
Step 2: Define the features and the target
SVM14
Have a look at the features:
SVM15


Have a look at the target:
SVM17

Step 3: Split the dataset into train and test using sklearn before building the SVM algorithm model
SVM19
Step 4: Import the support vector classifier function or SVC function from Sklearn SVM module. Build the Support Vector Machine model with the help of the SVC function
SVM20
Step 5: Predict values using the SVM algorithm model
SVM22
Step 6: Evaluate the Support Vector Machine model

Implementing Kernel SVM with Sklearn SVM module

SVM24

Go through this Artificial Intelligence Interview Questions And Answers to excel in your Artificial Intelligence Interview.

Polynomial Kernel

Importing the libraries:
Importing the SVC function and setting kernel as ‘poly’:


Making predictions:

Evaluating the model:
SVM28

Gaussian Kernel

Importing the SVC function and setting kernel as ‘rbf’:
SVM30

Making predictions:


SVM37

For the best of career growth, check out Intellipaat’s Machine Learning Course and get certified.

Sigmoid Kernel

Importing the SVC function and setting SVM kernel as ‘sigmoid’:

SVM40
Making predictions:
SVM39
Evaluating the model:
SVM41
SVM42

What did we learn so far?

In this SVM tutorial blog, we answered the question, ‘what is SVM?’ Some other important concepts such as SVM full form, pros and cons of SVM algorithm, and SVM examples, are also highlighted in this blog . We also learned how to build support vector machine models with the help of the support vector classifier function. Additionally, we talked about the implementation of Kernel SVM in Python and Sklearn, which is a very useful method while dealing with non-linearly separable datasets.

Compare SVM Machine Learning model with other Supervised Machine Learning classification models like Random Forest and Decision Tree!

Leave a Reply

Your email address will not be published. Required fields are marked *

Solve : *
25 + 1 =