SVM: Kernel Methods
Kernel function
Given a mapping function ϕ:X→V, the function
K:x→v,K(x,x′)=⟨ϕ(x),ϕ(x′)⟩Vis called a kernel function.
“A kernel is a function that returns the result of a dot product performed in another space.”
Kernel trick
Applying the kernel trick simply means replacing the dot product of two examples by a kernel function.
Typical kernels
| Kernel Type | Definition |
|---|
| Linear kernel | k(x,x′)=⟨x,x′⟩ |
| Polynomial kernel | k(x,x′)=⟨x,x′⟩d |
| Gaussian / Radial Basis Function (RBF) kernel | k(x,y)=exp(−2σ2∥x−y∥2) |
Why do we need kernel trick?
Kernels can be used for all feature based algorithms that can be rewritten such that they contain inner products of feature vectors
- This is true for almost all feature based algorithms (Linear regression, SVMs, …)
Kernels can be used to map the data x in an infinite dimensional feature space (i.e., a function space)
- The feature vector never has to be represented explicitly
- As long as we can evaluate the inner product of two feature vectors
➡️ We can obtain a more powerful representation than standard linear feature models.