Creating and Visualizing Decision Tree Algorithm in Machine Learning Using Sklearn

In this decision tree tutorial blog, we will talk about what a decision tree algorithm is, and we will also mention some interesting decision tree examples. The blog will also highlight how to create a decision tree classification model and a decision tree for regression using the decision tree classifier function and the decision tree regressor function, respectively. Also, it will discuss about decision tree analysis, how to visualize a decision tree algorithm in Machine Learning using Python, Scikit-Learn, and the Graphviz tool.

Creating and Visualizing Decision Tree Algorithm in Machine Learning Using Sklearn
21st Sep, 2019
2345 Views
1 comment(s)

What is Decision Tree? Decision Tree in Python and Scikit-Learn

Decision Tree algorithm is one of the simplest yet powerful Supervised Machine Learning algorithms. Decision Tree algorithm can be used to solve both regression and classification problems in Machine Learning. That is why it is also known as CART or Classification and Regression Trees. As the name suggests, in Decision Tree, we form a tree-like model of decisions and their possible consequences.

Check this Intellipaat Decision Tree in Data Science video :

Before we dive right into understanding this interesting algorithm, let us take a look at the concepts this blog has to offer.

Without much delay, let’s get started!

Decision Tree Algorithm Example

Monica’s cousin Marry is visiting Central Park this weekend. Now, Monica needs to make some plans for the weekend, whether to go out for shopping, go for a movie, spend time in the Central Park coffee shop, or just stay in and play a board game. Well, she decides to create a Decision Tree to make things easy. Let us see what she has created.

Here, the interior nodes represent different tests on an attribute (for example, whether to go out or stay in), branches hold the outcomes of those tests, and leaf nodes represent a class label or some decision taken after measuring all attributes. Each path from the root node to the leaf nodes represents a decision tree classification rule.

Rule 1: If it’s not raining and not too sunny, then go out for shopping.

Rule 2: If it’s not raining but too sunny outside, then go for a movie.

Rule 3: If it’s raining outside and the cable has signal, then watch a TV show.

Rule 4: If it’s raining and the cable signal fails, then spend time in the coffee shop downstairs

That’s how a decision tree helps Monica to make the perfect weekend plan with her cousin.

Types of Decision Tree Algorithms

There are two types of decision trees. They are categorized based on the type of the target variable they have. If the decision tree has a categorical target variable, then it is called a ‘categorical variable decision tree’. Similarly, if it has a continuous target variable, it is called a ‘continuous variable decision tree’.

  • Root Node: This node gets divided into different homogeneous nodes. It represents entire sample.
  • Splitting: It is the process of splitting or dividing a node into two or more sub-nodes.
  • Interior Nodes: They represent different tests on an attribute.
  • Branches: They hold the outcomes of those tests.
  • Leaf Nodes: When the nodes can’t be split further, they are called leaf nodes.
  • Parent and Child Nodes: The node from where sub-nodes are created is called a parent node. And, the sub-nodes are called the child nodes.

Advantages of Decision Tree Algorithms

  • Easy to understand
  • Requires minimum data cleaning
  • No constraint on the data type

Disadvantage of Decision Tree Algorithms

  • Possibility of over fitting

Check this Intellipaat Machine Learning video :

How Does a Decision Tree in Machine Learning Work?

The process of training and predicting the target features using a decision tree in Machine Learning is given below:

  • Feed a dataset, containing a number of training instances, with a set of features and a target
  • Train the decision tree classification or regression models with the help of DecisionTreeClassifier () or DecisionTreeRegressor () methods, and add the required criterion while building the decision tree model
  • Use Graphviz to visualize the decision tree model

That’s it! Your decision tree model is ready.

Decision Tree in Machine Learning – DecisionTreeClassifier () and DecisionTreeRegressor ()

DecisionTreeClassifier (): It is nothing but the decision tree classifier function to build a decision tree model in Machine Learning using Python. The DecisionTreeClassifier() function looks like this:

DecisionTreeClassifier (criterion = ‘gini’, random_state = None, max_depth = None, min_samples_leaf =1)

Here are a few important parameters:

  • criterion: It is used to measure the quality of a split in the decision tree classification. By default, it is ‘gini’; it also supports ‘entropy’.
  • max_depth: This is used to add maximum depth to the decision tree after the tree is expanded.
  • min_samples_leaf: This parameter is used to add the minimum number of samples required to be present at a leaf node.

DecisionTreeRegressio (): It is the decision tree regressor function used to build a decision tree model in Machine Learning using Python. The DecisionTreeRegressor () function looks like this:

DecisionTreeRegressor (criterion = ‘mse’, random_state =None , max_depth=None, min_samples_leaf=1,)

  • criterion: This function is used to measure the quality of a split in the decision tree regression. By default, it is ‘mse’ (the mean squared error), and it also supports ‘mae’ (the mean absolute error).
  • max_depth: This is used to add maximum depth to the decision tree after the tree is expanded.
  • min_samples_leaf: This function is used to add the minimum number of samples required to be present at a leaf node.

Creating and Visualizing a Decision Tree Regression Model in Machine Learning Using Python

Problem Statement: Use Machine Learning to predict the selling prices of houses based on some economic factors. Build a model using decision tree in Python.

Dataset: Boston Housing Dataset

Let us have a quick look at the dataset:

Model Building

Let us build the regression model of decision tree in Python.

Step 1: Load required packages

Step 2: Load the Boston dataset

Learn Machine Learning from experts, click here to more in this Macnine Learning Training in Hyderabad!

Take a look at the first five entries of the dataset

Step 3: Visualize the dataset using a scatter plot

Step 4: Define the features and the target

Step 5: Split the dataset into train and test sets

Here, ‘test_size = 0.2’ means that the test set will be 20 percent of the whole dataset and the training set’s size will be 80 percent of the entire dataset.

Step 6: Build the model with the decision tree regressor function

Step 7: Visualize the tree using Graphviz

After executing this step, the ‘reg_tree.dot’ file will be saved in your system. Now to visualize the tree, open this file with the ‘.dot’ extension.

Become Master of Machine Learning by going through this online Machine Learning course in Sydney.

Now, copy the graphviz data, go to the site ‘http://www.webgraphviz.com/’ and paste the Graphviz data there, as shown below:

Having done all this, let us perform the decision tree analysis now.

Step 7: Predict the values

Step 8: Compare y_test and y_pred

Step 9: Finding the RMSE value


Check this Intellipaat Machine Learning video :

Interested in learning Machine Learning? Click here to learn more in this Machine Learning Training in New York!

Creating and Visualizing a Decision Tree Classification Model in Machine Learning Using Python

Problem Statement: Use Machine Learning to predict breast cancer cases using patient treatment history and health data. Build a model using decision tree in Python.

Dataset: Breast Cancer Wisconsin (Diagnostic) Dataset

Let us have a quick look at the dataset:

Model Building

Let us build the classification model of decision tree in Python.

Step 1: Load required packages and the dataset using Pandas

Step 2: Take a look at the shape of the dataset

Step 3: Define the features and the target

Step 4: Split the dataset into train and test sets using sklearn

Go through this Artificial Intelligence Interview Questions And Answers to excel in your Artificial Intelligence Interview.

Step 5: Build the model with the help of the decision tree classifier function

Step 6: Visualize the decision tree

After executing this step, the ‘clf_tree.dot’ file will be saved in your system. Now to visualize the tree, open this file with the ‘.dot’ extension and copy the graphviz data. Then, go to the site ‘http://www.webgraphviz.com/’ and paste the graphviz data there, as shown below:

Click on ‘Generate Graph’

After this step, let us perform the decision tree analysis now.

Step 7: Predict the values

For the best of career growth, check out Intellipaat’s Machine Learning Course and get certified.

Step 8: Compare y_test and y_pred

Step 9: Find the confusion matrix and other metric parameters for this decision tree classification model

What Did We Learn So Far?

In this decision tree tutorial blog, we answered the question, ‘what is decision tree algorithm?’ We also learned how to build decision tree classification models with the help of decision tree classifier and decision tree regressor, decision tree analysis, and also decision tree algorithm visualization in Machine Learning using Python, Scikit-Learn, and Graphviz tool.

 

 

Related Articles

1 thought on “Creating and Visualizing Decision Tree Algorithm in Machine Learning Using Sklearn”

Leave a Reply

Your email address will not be published. Required fields are marked *

Solve : *
28 + 15 =