Show simple item record

Depth-Adaptive Neural Networks from the Optimal Control viewpoint

hal.structure.identifierCEntre de REcherches en MAthématiques de la DEcision [CEREMADE]
dc.contributor.authorAghili, Joubine
HAL ID: 2773
hal.structure.identifierLaboratoire Jacques-Louis Lions [LJLL]
hal.structure.identifierCEntre de REcherches en MAthématiques de la DEcision [CEREMADE]
dc.contributor.authorMula, Olga
HAL ID: 1531
ORCID: 0000-0002-3017-6598
dc.date.accessioned2020-10-21T09:17:10Z
dc.date.available2020-10-21T09:17:10Z
dc.date.issued2020
dc.identifier.urihttps://basepub.dauphine.fr/handle/123456789/21136
dc.language.isoenen
dc.subjectNeural Networksen
dc.subjectDeep Learningen
dc.subjectContinuous-Depth Neural Networksen
dc.subjectOptimal Controlen
dc.subject.ddc515en
dc.titleDepth-Adaptive Neural Networks from the Optimal Control viewpointen
dc.typeDocument de travail / Working paper
dc.description.abstractenIn recent years, deep learning has been connected with optimal control as a way to define a notion of a continuous underlying learning problem. In this view, neural networks can be interpreted as a discretization of a parametric Ordinary Differential Equation which, in the limit, defines a continuous-depth neural network. The learning task then consists in finding the best ODE parameters for the problem under consideration, and their number increases with the accuracy of the time discretization. Although important steps have been taken to realize the advantages of such continuous formulations, most current learning techniques fix a discretization (i.e. the number of layers is fixed). In this work, we propose an iterative adaptive algorithm where we progressively refine the time discretization (i.e. we increase the number of layers). Provided that certain tolerances are met across the iterations, we prove that the strategy converges to the underlying continuous problem. One salient advantage of such a shallow-to-deep approach is that it helps to benefit in practice from the higher approximation properties of deep networks by mitigating over-parametrization issues. The performance of the approach is illustrated in several numerical examples.en
dc.identifier.citationpages40en
dc.relation.ispartofseriestitleCahier de recherche CEREMADEen
dc.identifier.urlsitehttps://hal.archives-ouvertes.fr/hal-02897466en
dc.subject.ddclabelAnalyseen
dc.description.ssrncandidatenonen
dc.description.halcandidatenonen
dc.description.readershiprechercheen
dc.description.audienceInternationalen
dc.date.updated2020-10-21T09:14:15Z
hal.author.functionaut
hal.author.functionaut


Files in this item

Thumbnail
Name:
AM2020.pdf
Size:
1.440Mb
Format:
PDF
View/Open

This item appears in the following Collection(s)

Show simple item record