Seminar

Optimal Estimation of Sparse High-Dimensional Additive Models

Enno Mammen (University of Heidelberg)

November 10, 2016, 11:00–12:15

Toulouse

Room MS 001

MAD-Stat. Seminar

Abstract

In this paper we discuss the estimation of a nonparametric component f1 of a nonparametric additive model Y = f1(X1) + ... + fq(Xq) + ". We allow the number q of additive components to grow to infinity and we make sparsity assumptions about the number of nonzero additive components. We compare this estimation problem with that of estimating f1 in the oracle model Z = f1(X1) + ", for which the additive components f2, . . . , fq are known. We construct a two-step presmoothing-andresmoothing estimator of f1 in the additive model and state finitesample bounds for the di↵erence between our estimator and some smoothing estimators ˜ foracle 1 in the oracle model which satisfy mild conditions. In an asymptotic setting these bounds can be used to show asymptotic equivalence of our estimator and the oracle estimators; the paper thus shows that, asymptotically, under strong enough sparsity conditions, knowledge of f2, . . . , fq has no e↵ect on estimation efficiency. Our first step is to estimate all of the components in the additive model with undersmoothing using a group-Lasso estimator. We then construct pseudo responses ˆ Y by evaluating a desparsified modification of our undersmoothed estimator of f1 at the design points. In the second step the smoothing method of the oracle estimator ˜ foracle 1 is applied to a nonparametric regression problem with “responses” ˆ Y and covariates X1. Our mathematical exposition centers primarily on establishing properties of the presmoothing estimator. We also present simulation results demonstrating close-to-oracle performance of our estimator in practical applications. The main results of the paper are also important for understanding the behavior of the presmoothing estimator when the resmoothing step is omitted.