Motivation Overfitting happens because of having too few examples to train on, resulting in a model that has poor generalization performance 😢. If we had infinite training data, we wouldn’t overfit because we would see every possible instance.
2020-08-16
Model Overfitting In order to give more “capacity” to capture different features, we give neural nets a lot of neurons. But this can cause overfitting. Reason: Co-adaptation Neurons become dependent on others Imagination: neuron $H\_i$ captures a particular feature $X$ which however, is very frequenly seen with some inputs.
2020-08-16