Adaptive Boosting: Correct its predecessor by paying a bit more attention to the training instance that the predecessor underfitted. This results in new predictors focusing more and more on the hard cases.
2020-11-07
TL;DR Bootstrap Aggregating (Boosting): Sampling with replacement Pasting: Sampling without replacement Explaination Ensemble methods work best when the predictors are as independent from one another as possible. One way to get a diverse set of classifiers: use the same training algorithm for every predictor, but to train them on different random subsets of the training set
2020-11-07
Boosting Refers to any Ensemble method that can combine serval weak learners into a strong learner 💡 General idea: train predictors sequentially, each trying to correct its predecessor. Popular boosting methods:
2020-11-07
Why emsemble learners? Lower error Each learner (model) has its own bias. It we put them together, the bias tend to be reduced (they fight against each other in some sort of way) Less overfitting Tastes great
2020-11-07
Train a group of Decision Tree classifiers (generally via the bagging method (or sometimes pasting)), each on a different random subset of the training set To make predictions, just obtain the preditions of all individual trees, then predict the class that gets the most votes.
2020-11-07
Suppose we have trained a few classifiers, each one achieving about 80% accuracy. A very simple way to create an even better classifier is to aggregate the predictions of each classifier and predict the class that gets the most votes.
2020-11-07
wisdom of the crowd : In many cases you will find that this aggregated answer is better than an expert’s answer. Similarly, if you aggregate the predictions of a group of predictors (such as classifiers or regressors), you will often get better predictions than with the best individual predictor.
2020-11-07
Use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone.
2020-09-07