Show simple item record

Stochastic Runge-Kutta methods and adaptive SGD-G2 stochastic gradient descent

hal.structure.identifierCEntre de REcherches en MAthématiques de la DEcision [CEREMADE]
dc.contributor.authorAyadi, Imen
hal.structure.identifierCEntre de REcherches en MAthématiques de la DEcision [CEREMADE]
dc.contributor.authorTurinici, Gabriel
HAL ID: 16
ORCID: 0000-0003-2713-006X
dc.date.accessioned2021-11-04T09:28:22Z
dc.date.available2021-11-04T09:28:22Z
dc.date.issued2021
dc.identifier.urihttps://basepub.dauphine.psl.eu/handle/123456789/22167
dc.language.isoenen
dc.subjectSGDen
dc.subjectstochastic gradient descenten
dc.subjectMachine Learningen
dc.subjectadaptive stochastic gradienten
dc.subjectdeep learning optimizationen
dc.subjectneural networks optimizationen
dc.subject.ddc519en
dc.titleStochastic Runge-Kutta methods and adaptive SGD-G2 stochastic gradient descenten
dc.typeCommunication / Conférence
dc.description.abstractenThe minimization of the loss function is of paramount importance in deep neural networks. On the other hand, many popular optimization algorithms have been shown to correspond to some evolution equation of gradient flow type. Inspired by the numerical schemes used for general evolution equations we introduce a second order stochastic Runge Kutta method and show that it yields a consistent procedure for the minimization of the loss function. In addition it can be coupled, in an adaptive framework, with a Stochastic Gradient Descent (SGD) to adjust automatically the learning rate of the SGD, without the need of any additional information on the Hessian of the loss functional. The adaptive SGD, called SGD-G2, is successfully tested on standard datasets.en
dc.identifier.citationpages1-16en
dc.relation.ispartoftitle25th International Conference on Pattern Recognition (ICPR 2020)en
dc.relation.ispartofpublnameIEEE - Institute of Electrical and Electronics Engineersen
dc.relation.ispartofpublcityPiscataway, NJen
dc.relation.ispartofdate2021
dc.relation.ispartofurl10.1109/ICPR48806.2021en
dc.subject.ddclabelProbabilités et mathématiques appliquéesen
dc.relation.conftitle25th International Conference on Pattern Recognition (ICPR)en
dc.relation.confdate2021-01
dc.relation.confcityMilanen
dc.relation.confcountryItalyen
dc.relation.forthcomingnonen
dc.identifier.doi10.1109/ICPR48806.2021.9412831en
dc.description.ssrncandidatenon
dc.description.halcandidatenonen
dc.description.readershiprechercheen
dc.description.audienceInternationalen
dc.relation.Isversionofjnlpeerreviewednonen
dc.date.updated2021-11-04T09:23:35Z
hal.author.functionaut
hal.author.functionaut


Files in this item

Thumbnail
Name:
turinici_ayadi2020-rk-adaptive ...
Size:
449.1Kb
Format:
PDF
View/Open

This item appears in the following Collection(s)

Show simple item record