Show simple item record

hal.structure.identifierCEntre de REcherches en MAthématiques de la DEcision [CEREMADE]
dc.contributor.authorAyadi, Imen
hal.structure.identifierCEntre de REcherches en MAthématiques de la DEcision [CEREMADE]
dc.contributor.authorTurinici, Gabriel
HAL ID: 16
ORCID: 0000-0003-2713-006X
dc.date.accessioned2020-05-12T11:40:01Z
dc.date.available2020-05-12T11:40:01Z
dc.date.issued2020
dc.identifier.urihttps://basepub.dauphine.fr/handle/123456789/20719
dc.language.isoenen
dc.subjectSGDen
dc.subjectstochastic gradient descenten
dc.subjectMachine Learningen
dc.subjectadaptive stochastic gradienten
dc.subjectdeep learning optimizationen
dc.subjectneural networks optimizationen
dc.subject.ddc515en
dc.titleStochastic Runge-Kutta methods and adaptive SGD-G2 stochastic gradient descenten
dc.typeDocument de travail / Working paper
dc.description.abstractenThe minimization of the loss function is of paramount importance in deep neural networks. On the other hand, many popular optimization algorithms have been shown to correspond to some evolution equation of gradient flow type. Inspired by the numerical schemes used for general evolution equations we introduce a second order stochastic Runge Kutta method and show that it yields a consistent procedure for the minimization of the loss function. In addition it can be coupled, in an adaptive framework, with a Stochastic Gradient Descent (SGD) to adjust automatically the learning rate of the SGD, without the need of any additional information on the Hessian of the loss functional. The adaptive SGD, called SGD-G2, is successfully tested on standard datasets.en
dc.publisher.nameCahier de recherche CEREMADE, Université Paris-Dauphineen
dc.publisher.cityParisen
dc.identifier.citationpages16en
dc.identifier.urlsitehttps://hal.archives-ouvertes.fr/hal-02483988en
dc.subject.ddclabelAnalyseen
dc.identifier.citationdate2020-02
dc.description.ssrncandidatenonen
dc.description.halcandidatenonen
dc.description.readershiprechercheen
dc.description.audienceInternationalen
dc.date.updated2020-05-12T11:37:30Z
hal.author.functionaut
hal.author.functionaut


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record