Show simple item record

A nonlinear conjugate gradient method with complexity guarantees and its application to nonconvex regression

dc.contributor.authorChan-Renous-Legoubin, Rémi
hal.structure.identifierLaboratoire d'analyse et modélisation de systèmes pour l'aide à la décision [LAMSADE]
dc.contributor.authorRoyer, Clément W.
dc.date.accessioned2023-03-03T11:21:57Z
dc.date.available2023-03-03T11:21:57Z
dc.date.issued2022
dc.identifier.issn2192-4406
dc.identifier.urihttps://basepub.dauphine.psl.eu/handle/123456789/24521
dc.language.isoenen
dc.subjectOptimization and Control
dc.subject.ddc003en
dc.titleA nonlinear conjugate gradient method with complexity guarantees and its application to nonconvex regression
dc.typeArticle accepté pour publication ou publié
dc.description.abstractenNonlinear conjugate gradients are among the most popular techniques for solving continuous optimization problems. Although these schemes have long been studied from a global convergence standpoint, their worst-case complexity properties have yet to be fully understood, especially in the nonconvex setting. In particular, it is unclear whether nonlinear conjugate gradient methods possess better guarantees than first-order methods such as gradient descent. Meanwhile, recent experiments have shown impressive performance of standard nonlinear conjugate gradient techniques on certain nonconvex problems, even when compared with methods endowed with the best known complexity guarantees.In this paper, we propose a nonlinear conjugate gradient scheme based on a simple line-search paradigm and a modified restart condition. These two ingredients allow for monitoring the properties of the search directions, which is instrumental in obtaining complexity guarantees. Our complexity results illustrate the possible discrepancy between nonlinear conjugate gradient methods and classical gradient descent. A numerical investigation on nonconvex robust regression problems as well as a standard benchmark illustrate that the restarting condition can track the behavior of a standard implementation.
dc.relation.isversionofjnlnameEURO Journal on Computational Optimization
dc.relation.isversionofjnlvol10
dc.relation.isversionofjnldate2022
dc.relation.isversionofjnlpages100044
dc.relation.isversionofdoi10.1016/j.ejco.2022.100044
dc.identifier.urlsitehttps://arxiv.org/pdf/2201.08568.pdf
dc.relation.isversionofjnlpublisherSpringer
dc.subject.ddclabelRecherche opérationnelleen
dc.relation.forthcomingnonen
dc.description.ssrncandidatenon
dc.description.halcandidateoui
dc.description.readershiprecherche
dc.description.audienceInternational
dc.relation.Isversionofjnlpeerreviewedoui
dc.date.updated2023-03-03T13:05:30Z
hal.export.arxivnonen
hal.export.pmcnonen
hal.hide.repecnonen
hal.hide.oainonen
hal.author.functionaut
hal.author.functionaut


Files in this item

Thumbnail
Name:
1-s2.0-S219244062200020X-main.pdf
Size:
2.265Mb
Format:
PDF
View/Open

This item appears in the following Collection(s)

Show simple item record