How to properly select a Machine Learning model.
2020-09-07
Objective function overview
2020-07-06
TL;DR Resaon Example affect Model's complexity ⬆️ Model's complexity ⬇️ Bias wrong assumption assume a quadratic model to be linear underfitting ⬇️ ⬆️ Variance excessive sensitivity to small variations high-degree polynomial model overfitting ⬆️ ⬇️ Inreducible error noisy data Explaination A model’s generalization error can be expressed as the sum of three very different errors:
2020-07-06
How does the objective function look like? Objective function: $$ \operatorname{Obj}(\Theta)= \overbrace{L(\Theta)}^{\text {Training Loss}} + \underbrace{\Omega(\Theta)}_{\text{Regularization}} $$ Training loss: measures how well the model fit on training data $$ L=\sum_{i=1}^{n} l\left(y_{i}, g_{i}\right) $$ Square loss: $$ l(y_i, \hat{y}_i) = (y_i - \hat{y}_i)^2 $$ Logistic loss: $$ l(y_i, \hat{y}_i) = y_i \log(1 + e^{-\hat{y}_i}) + (1 - y_i) \log(1 + e^{\hat{y}_i}) $$ Regularization: How complicated is the model?
2020-07-06