First order methods beyond convexity and Lipschitz gradient continuity with applications to quadratic inverse problems

Abstract

We focus on nonconvex and nonsmooth minimization problems with a composite objective, where the differentiable part of the objective is freed from the usual and restrictive global Lipschitz gradient continuity assumption. This longstanding smoothness restriction is pervasive in first order methods (FOM), and was recently circumvent for convex composite optimization by Bauschke, Bolte and Teboulle, through a simple and elegant framework which captures, all at once, the geometry of the function and of the feasible set. Building on this work, we tackle genuine nonconvex problems. We first complement and extend their approach to derive a full extended descent lemma by introducing the notion of smooth adaptable functions. We then consider a Bregman-based proximal gradient methods for the nonconvex composite model with smooth adaptable functions, which is proven to globally converge to a critical point under natural assumptions on the problem's data. To illustrate the power and potential of our general framework and results, we consider a broad class of quadratic inverse problems with sparsity constraints which arises in many fundamental applications, and we apply our approach to derive new globally convergent schemes for this class.

Reference

Jérôme Bolte, Shoham Sabach, Marc Teboulle, and Y. Vaisbourd, “First order methods beyond convexity and Lipschitz gradient continuity with applications to quadratic inverse problems”, SIAM Journal on Optimization, vol. 28, n. 3, 2018, pp. 2131–2151.