Gaussian process
We have the follow impractical definition:
A Gaussian process is a collection of random variables, any finite number of which have consistent Gaussian distributions.
It is completely defined by its mean function \(m(x)\) and covariance function \(\Sigma(x, x')\). We write:
\begin{equation} f(x) \sim \mathcal{G}\left(m(x), \Sigma(x, x')\right) \end{equation}
A very import special case of Gaussian processes is Gaussian Markov processes
Kernels #
We often express the covariance function \(\Sigma(x, x')\) as a function of a kernel \(K(x,x')\)):
\begin{align*} \displaystyle \Sigma(x,x') = K(x, x') + \sigma^{2}_{y}\;\mathbb{I} \end{align*}
We can combine kernels. The multiplication of two covariance functions is a valid covariance functions, the addition of two correlation functions is a correlation function.
Sample #
How to Sample from a Gaussian Process predictive distribution
Learn #
- Visual exploration of gaussian processes is a very good introduction
- Gaussian Process, not quite for dummies is excellent, starts from gaussians to gaussian in high dimensions to get to gaussian process. One of the best things I’ve read on the topic.
- is a useful reference, but more advanced
Use #
Here is a list of libraries implementating different flavors of Gaussian processes. The catch is that gaussian processes can quick get computation-intensive.
- BayesNewton is written in JAX
- GPytorch is built atop the Pytorch autodiff framework
- PyMC3 also has an implementation of gaussian processes