Covariance function estimation in Gaussian process regression

Résumé

Gaussian process regression consists in predicting a continuous realization of a Gaussian process, given a finite number of observations of it. When the covariance function of the Gaussian process is known, or when the statistician selects and fix a given covariance function, this prediction is made explicitly thanks to Gaussian conditioning. Thus, most classically, the covariance function is estimated in a first step, and kept fixed to its estimate in a second step, where prediction is carried out ("plug-in approach"). In this presentation, we address parametric estimation, and we consider the Maximum Likelihood and Cross Validation estimators of the covariance parameters. We analyze these two estimators in two cases. 1) Well-specified case where the true covariance function belongs to the parametric set of covariance functions used for estimation. We consider an increasing-domain asymptotic framework, based on a randomly-perturbed regular grid of observation points. We show that both estimators are consistent and asymptotically Gaussian with a square-root-of-n rate of convergence. It is observed that the Maximum Likelihood estimator has a smaller asymptotic variance. 2) Misspecified case where the true covariance function does not belong to the parametric set of covariance functions used for estimation. A finite-sample analysis shows that, for design of observation points that are not too regular, Cross Validation is more robust than Maximum Likelihood. Furthermore, an increasing-domain asymptotic result supports this conclusion. More precisely, for randomly located observation points, the Cross Validation estimator converges to the covariance parameter minimizing the integrated square prediction error.