Ensemble Method
Ensemble Method
Ensemble
Definition:
A set of learned models whose individual decisions are combined in some way to make predictions for new instances.
When ensemble does a better job?
The errors made by the individual predictors are (somewhat) uncorrelated, and the predictors’ error rates are better than guessing (< 0.5 for 2-class pro...
Support Vector Machine
Support Vector Machines
Motivation
Linear classification usually assume perfect separation between the two classes. However, if a new data point is introduced to the system, the current linear model may not be able to capture it. Therefore, the margin is introduced to address this problem.
Margin
Lemma 1
$x$ has distance $\frac{|f_w(x)|}{||w|...
Decision Tree
Decision Tree
Overview
Simple Structure
Each internal node tests one features $x_i$.
Each branch from an internal node represents one outcome of the test.
Each leaf predicts $y$ or $P(y|x)$
Top-down decision tree learning
Pseudocode
MakeSubtree(set of training instances $D$)
$\quad\quad C$ = DetermineCandidateSplits($D$)
$\quad\quad$if stoppin...
13 post articles, 2 pages.