A nonlinear conjugate gradient method with complexity guarantees and its application to nonconvex regression
Chan-Renous-Legoubin, Rémi; Royer, Clément W. (2022), A nonlinear conjugate gradient method with complexity guarantees and its application to nonconvex regression, EURO Journal on Computational Optimization, 10, p. 100044. 10.1016/j.ejco.2022.100044
Voir/Ouvrir
Type
Article accepté pour publication ou publiéLien vers un document non conservé dans cette base
https://arxiv.org/pdf/2201.08568.pdfDate
2022Nom de la revue
EURO Journal on Computational OptimizationVolume
10Éditeur
Springer
Pages
100044
Identifiant publication
Métadonnées
Afficher la notice complèteAuteur(s)
Chan-Renous-Legoubin, RémiRoyer, Clément W.
Laboratoire d'analyse et modélisation de systèmes pour l'aide à la décision [LAMSADE]
Résumé (EN)
Nonlinear conjugate gradients are among the most popular techniques for solving continuous optimization problems. Although these schemes have long been studied from a global convergence standpoint, their worst-case complexity properties have yet to be fully understood, especially in the nonconvex setting. In particular, it is unclear whether nonlinear conjugate gradient methods possess better guarantees than first-order methods such as gradient descent. Meanwhile, recent experiments have shown impressive performance of standard nonlinear conjugate gradient techniques on certain nonconvex problems, even when compared with methods endowed with the best known complexity guarantees.In this paper, we propose a nonlinear conjugate gradient scheme based on a simple line-search paradigm and a modified restart condition. These two ingredients allow for monitoring the properties of the search directions, which is instrumental in obtaining complexity guarantees. Our complexity results illustrate the possible discrepancy between nonlinear conjugate gradient methods and classical gradient descent. A numerical investigation on nonconvex robust regression problems as well as a standard benchmark illustrate that the restarting condition can track the behavior of a standard implementation.Mots-clés
Optimization and ControlPublications associées
Affichage des éléments liés par titre et auteur.
-
Curtis, F. E.; Robinson, D. P.; Royer, Clément; Wright, S. J. (2019) Article accepté pour publication ou publié
-
Bergou, E. H.; Diouane, Y.; Kungurtsev, V.; Royer, Clément W. (2022) Article accepté pour publication ou publié
-
Zaag, Hatem; Nouaili, Nejla (2010) Article accepté pour publication ou publié
-
Lewin, Mathieu; Séré, Eric (2009) Article accepté pour publication ou publié
-
Bartier, Jean-Philippe; Laurençot, Philippe (2007) Article accepté pour publication ou publié