Choose the Length Scale: Gaussian Process Regression
A Gaussian process (GP) with mean 0 and squared exponential kernel
\[ k(x, x') = \exp\!\left( -\frac{(x - x')^2}{2\ell^2} \right) \]is used to learn an unknown function \(f\) on \([0, 1]\) from five observations. The length scale \(\ell\) controls how quickly the GP believes \(f\) can change. Given the observations, the GP gives a posterior mean (its best guess of \(f\)) and a 95% credible band around it.
Your task: the true \(f\) is hidden. Looking only at the observations, choose a length scale so that the 95% band covers \(f\) everywhere on \([0, 1]\), while keeping the band as narrow as possible. Then click Reveal f to see how you did: any part of \(f\) outside the band is shown in red. A tiny \(\ell\) covers almost anything, but its band is too wide to be useful.