Finance, medicine and telecommunication are some examples where classification problem is quite helpful. It is also used in spam filtering. With classification one can train models after assigning labels to instances to predict the pre-assigned labels of a set of examples. Of all the Machine Learning algorithms selection of a method depends on business needs. Each has its own ups and downs. In this article we’ll visit on the boosting and random forest classifiers.
Boosting
In boosting, n number of samples are made utilizing bootstrapping which is sampling with substitution. Boosting gets the work done on weak classifiers that have high bias. On weak learners low variation works iteratively and misclassified learners in subsequent iteration are given more weightage. By combining prediction of every classifier the final classification is done. Boosting utilizes base model as decision tree algorithms usually. Be that as it may, linear regression or logistic regression can be utilized as a base model as well. It is best to grow a tree with no pruning and trees with 2-8 leaves function admirably. For a large number of small models with low variance boosting reduces bias. On the bias of each level the underlying elements are chained somehow. As each model improved upon all its previous models there is no such thing as independent parallel models here.
Random Forest Classifiers
Random Forest Classifiers is an ensemble method which works on uncorrelated classifiers and bootstrapped samples. A final decision is made based on the majority as this model learns from various over grown trees. For every node predictors are sampled. It is ideal for models that have high variance and low bias and is overfitted. Random Forest Classifiers diminishes variance of an extensive number of complex models with low bias. The models are excessively complex and not weak as per the composition elements. The underlying trees are planted as large as possible. The underlying trees can therefore be said as independent parallel models. To make them even more independent a random variable selection is added in addition. The performance is therefore better than ordinary bagging.
Let’s visit on the aspects in which boosting and random forest differ :
| Boosting |
Random Forest Classifiers |
| Boosting is used normally when the aim is to train and test. |
Random Forest Classifiers is preferred when the aim is to train and test as well as for prediction |
| While boosting has a high accuracy it does not rival that of the random forest |
Random Forest Classifiers is more precise and better explainable than boosting on the various predictors |
| The algorithm for boosted trees tries to discover ideal linear combination of trees in connection to given training data. |
Trained with random sample of data, Random Forest Classifiers relies on randomization to give improved generalization performance from the training set. |
| When we have a practical view on the tuning stage there are some fine observations. Volatile and constantly changing data causes many variations in how boosted trees work. |
In the tuning stage, Random Forest Classifiers are better to strain on overfitting. |
| As boosting is only concerned with training and testing it uses nearest neighbour, k- nearest neighbour, decision tree. |
Random Forest Classifiers uses decision trees for prediction and bootstrapping for training & testing |
| In a more thoughtful way random samples are produced by the boosting methods. Consider there are 50 samples in the boosting method. The final prediction is the weighted average of all 50 samples. |
There is an out-of-bag sample for every bootstrapped sample used to test. Consider you get a set of predictions for each sample to the total of 50 bootstrapped samples. The average of all the 50 predictions is the final prediction |
Conclusion
The straight forward interpretation is one of the main reasons why boosting algorithms are so extensively used. The interpretation of gradient boosting is the same as the one provided by classical regression analysis. Boosting will have a huge application in the field of biomedical research in the years to come. The increase in the number of predictors and candidate variables for biomedical research is the main reason for this. Random forest is also used in many areas like ecology, medicine, astronomy, autopsy, agriculture, traffic and transport planning, Bioinformatics and many more. Random forests are successfully classifying astronomical objects. The research in Machine Learning is improving day by day because of which all these progress can be seen. The one you’re most familiar about is that of self-driving cars. But there are many areas where the sub branch of artificial intelligence pervades itself. The humankind is yet to witness the wonders of Machine Learning in the years to come.