ML

Tree-based Methods CART: Classification And Regression Tree Grow a binary tree At each node, “split” the data into two “daughter” nodes. Splits are chosen using a splitting criterion. Bottom nodes are “terminal” nodes.

2020-10-27

Classification models.

2020-07-13

Kernel regression Kernel identities Let $$\boldsymbol{\Phi}\_{X}=\left[\begin{array}{c} \boldsymbol{\phi}\left(\boldsymbol{x}\_{1}\right)^{T} \\\\ \vdots \\\\ \boldsymbol{\phi}\left(\boldsymbol{x}\_{N}\right)^{T} \end{array}\right] \in \mathbb{R}^{N \times d} , \qquad \left( \boldsymbol{\Phi}\_{X}^T = \left[ \boldsymbol{\phi}(x\_1), \dots, \boldsymbol{\phi}(x\_N)\right] \in \mathbb{R}^{d \times N} \right)$$ then the following identities hold:

2020-07-13

💡Idea Use a linear model to fit nonlinear data: add powers of each feature as new features, then train a linear model on this extended set of features. Generalize Linear Regression to Polynomial Regression In Linear Regression $f$ is modelled as linear in $\boldsymbol{x}$ and $\boldsymbol{w}$

2020-07-13

Linear Regression Model A linear model makes a prediction $\hat{y}_i$ by simply computing a weighted sum of the input $\boldsymbol{x}_i$, plus a constant $w_0$ called the bias term: For single sample/instances $$\hat{y}_i = f \left( \boldsymbol{x} \right) = w_0 + \sum\_{j=1}^{D}w\_{j} x\_{i, j}$$ In matrix-form:

2020-07-06

Objective function overview

2020-07-06

TL;DR Resaon Example affect Model's complexity ⬆️ Model's complexity ⬇️ Bias wrong assumption assume a quadratic model to be linear underfitting ⬇️ ⬆️ Variance excessive sensitivity to small variations high-degree polynomial model overfitting ⬆️ ⬇️ Inreducible error noisy data Explaination A model’s generalization error can be expressed as the sum of three very different errors:

2020-07-06