The backtrack Hölder gradient method with application to min-max and min-min problems

Résumé

We present a new algorithm to solve min-max or min-min problems out of the convex world. We use rigidity assumptions, ubiquitous in learning, making our method – the backtrack Hölder algorithm applicable to many optimization problems. Our approach takes advantage of hidden regularity properties and allows us, in particular, to devise a simple algorithm of ridge type. An original feature of our method is to come with automatic step size adaptation which departs from the usual overly cautious backtracking methods. In a general framework, we provide convergence theoretical guarantees and rates. We apply our findings on simple Generative Adversarial Network (GAN) problems obtaining promising numerical results. It is worthwhile mentioning that a byproduct of our approach is a simple recipe for general Hölderian backtracking optimization.

Mots-clés

Référence

Jérôme Bolte, Lilian Glaudin, Edouard Pauwels et Matthieu Serrurier, « The backtrack Hölder gradient method with application to min-max and min-min problems », Open Journal of Mathematical Optimization, vol. 4, n° 8, 2023, 17 pages.