 Updated on 18th Nov, 22 3751 Views

## What is Decision Tree? Decision Tree in Python and Scikit-Learn

Decision Tree algorithm is one of the simplest yet most powerful Supervised Machine Learning algorithms. Decision Tree algorithm can be used to solve both regression and classification problems in Machine Learning. That is why it is also known as CART or Classification and Regression Trees. As the name suggests, in Decision Tree, we form a tree-like model of decisions and their possible consequences.

Before we dive right into understanding this interesting algorithm, let us take a look at the concepts this blog has to offer.

### Check this Machine Learning Tutorial:

Without much delay, let’s get started!

## Decision Tree Algorithm Example

Monica’s cousin Marry is visiting Central Park this weekend. Now, Monica needs to make some plans for the weekend, whether to go out for shopping, go for a movie, spend time in the Central Park coffee shop, or just stay in and play a board game. Well, she decides to create a Decision Tree to make things easy. Let us see what she has created. Here, the interior nodes represent different tests on an attribute (for example, whether to go out or stay in), branches hold the outcomes of those tests, and leaf nodes represent a class label or some decision taken after measuring all attributes. Each path from the root node to the leaf nodes represents a decision tree classification rule.

Rule 1: If it’s not raining and not too sunny, then go out for shopping.

Rule 2: If it’s not raining but too sunny outside, then go for a movie.

Rule 3: If it’s raining outside and the cable has signal, then watch a TV show.

Rule 4: If it’s raining and the cable signal fails, then spend time in the coffee shop downstairs

That’s how a decision tree helps Monica to make the perfect weekend plan with her cousin. ## Types of Decision Tree Algorithms

There are two types of decision trees. They are categorized based on the type of the target variable they have. If the decision tree has a categorical target variable, then it is called a ‘categorical variable decision tree’. Similarly, if it has a continuous target variable, it is called a ‘continuous variable decision tree’.

• Root Node: This node gets divided into different homogeneous nodes. It represents the entire sample.
• Splitting: It is the process of splitting or dividing a node into two or more sub-nodes.
• Interior Nodes: They represent different tests on an attribute.
• Branches: They hold the outcomes of those tests.
• Leaf Nodes: When the nodes can’t be split further, they are called leaf nodes.
• Parent and Child Nodes: The node from which sub-nodes are created is called a parent node. And, the sub-nodes are called the child nodes.

Enroll in this amazing course to become an expert in AI and machine learning, M.Tech in Artificial Intelligence & Machine Learning by IIT Jammu!

## Advantages of Decision Tree Algorithms

• Easy to understand
• Requires minimum data cleaning
• No constraint on the data type

## Disadvantages of Decision Tree Algorithms

• Possibility of overfitting

## How Does a Decision Tree in Machine Learning Work?

The process of training and predicting the target features using a decision tree in Machine Learning is given below:

• Feed a dataset, containing a number of training instances, with a set of features and a target
• Train the decision tree classification or regression models with the help of DecisionTreeClassifier () or DecisionTreeRegressor () methods, and add the required criterion while building the decision tree model
• Use Graphviz to visualize the decision tree model

Interested to learn Data Science? Go through this Data Science course in Chennai.

Decision Tree in Machine Learning – DecisionTreeClassifier () and DecisionTreeRegressor ()

DecisionTreeClassifier (): It is nothing but the decision tree classifier function to build a decision tree model in Machine Learning using Python. The DecisionTreeClassifier() function looks like this:

DecisionTreeClassifier (criterion = ‘gini’, random_state = None, max_depth = None, min_samples_leaf =1)

Here are a few important parameters:

• criterion: It is used to measure the quality of a split in the decision tree classification. By default, it is ‘gini’; it also supports ‘entropy’.
• max_depth: This is used to add maximum depth to the decision tree after the tree is expanded.
• min_samples_leaf: This parameter is used to add the minimum number of samples required to be present at a leaf node.

DecisionTreeRegressio (): It is the decision tree regressor function used to build a decision tree model in Machine Learning using Python. The DecisionTreeRegressor () function looks like this:

DecisionTreeRegressor (criterion = ‘mse’, random_state =None , max_depth=None, min_samples_leaf=1,)

• criterion: This function is used to measure the quality of a split in the decision tree regression. By default, it is ‘mse’ (the mean squared error), and it also supports ‘mae’ (the mean absolute error).
• max_depth: This is used to add maximum depth to the decision tree after the tree is expanded.
• min_samples_leaf: This function is used to add the minimum number of samples required to be present at a leaf node. ## Creating and Visualizing a Decision Tree Regression Model in Machine Learning Using Python

Problem Statement: Use Machine Learning to predict the selling prices of houses based on some economic factors. Build a model using decision tree in Python.

Dataset: Boston Housing Dataset

Let us have a quick look at the dataset: Model Building

Let us build the regression model of decision tree in Python. Step 2: Load the Boston dataset Take a look at the first five entries of the dataset  Step 3: Visualize the dataset using a scatter plot  Step 4: Define the features and the target Step 5: Split the dataset into train and test sets Here, ‘test_size = 0.2’ means that the test set will be 20 percent of the whole dataset and the training set’s size will be 80 percent of the entire dataset.

Confused about choosing the best Data Set for your ML model? Refer to our blog on Choosing Dataset for Machine Learning.

Step 6: Build the model with the decision tree regressor function  Step 7: Visualize the tree using Graphviz After executing this step, the ‘reg_tree.dot’ file will be saved in your system. Now to visualize the tree, open this file with the ‘.dot’ extension. Become a Master of Machine Learning by going through this online Machine Learning course in Sydney.

Now, copy the Graphviz data, go to the site ‘http://www.webgraphviz.com/’ and paste the Graphviz data there, as shown below: Having done all this, let us perform the decision tree analysis now.

Step 7: Predict the values Step 8: Compare y_test and y_pred  Step 9: Finding the RMSE value  ## Creating and Visualizing a Decision Tree Classification Model in Machine Learning Using Python

Problem Statement: Use Machine Learning to predict breast cancer cases using patient treatment history and health data. Build a model using decision tree in Python.

Dataset: Breast Cancer Wisconsin (Diagnostic) Dataset

Let us have a quick look at the dataset: Model Building

Let us build the classification model of decision tree in Python.

Step 1: Load required packages and the dataset using Pandas  Step 2: Take a look at the shape of the dataset  Step 3: Define the features and the target Step 4: Split the dataset into train and test sets using sklearn Go through these Top 40 Machine Learning Interview Questions and Answers to crack your interviews.

Step 5: Build the model with the help of the decision tree classifier function  Step 6: Visualize the decision tree After executing this step, the ‘clf_tree.dot’ file will be saved in your system. Now to visualize the tree, open this file with the ‘.dot’ extension and copy the graphviz data. Then, go to the site ‘http://www.webgraphviz.com/’ and paste the graphviz data there, as shown below: Click on ‘Generate Graph’ After this step, let us perform the decision tree analysis now.

Step 7: Predict the values For the best of career growth, check out Intellipaat’s Machine Learning Course and get certified.

Step 8: Compare y_test and y_pred  Step 9: Find the confusion matrix and other metric parameters for this decision tree classification model  ## What Did We Learn So Far?

In this decision tree tutorial blog, we answered the question, ‘what is decision tree algorithm?’ We also learned how to build decision tree classification models with the help of decision tree classifier and decision tree regressor, decision tree analysis, and also decision tree algorithm visualization in Machine Learning using Python, Scikit-Learn, and Graphviz tool.

Course Schedule

Name Date Details
Machine Learning Course 04 Feb 2023(Sat-Sun) Weekend Batch
View Details
Machine Learning Course 11 Feb 2023(Sat-Sun) Weekend Batch
View Details
Machine Learning Course 18 Feb 2023(Sat-Sun) Weekend Batch
View Details