Thumbnail

Mean-field Langevin System, Optimal Control and Deep Neural Networks

Hu, Kaitong; Kazeykina, Anna; Ren, Zhenjie (2019-09), Mean-field Langevin System, Optimal Control and Deep Neural Networks. https://basepub.dauphine.fr/handle/123456789/19860

View/Open
Type
Document de travail / Working paper
Date
2019-09
Publisher
Cahier de recherche CEREMADE, Université Paris-Dauphine
Series title
Cahier de recherche CEREMADE, Université Paris-Dauphine
Published in
Paris
Pages
25
Metadata
Show full item record
Author(s)
Hu, Kaitong
Centre de Mathématiques Appliquées - Ecole Polytechnique [CMAP]
Kazeykina, Anna
Laboratoire de Mathématiques d'Orsay [LMO]
Ren, Zhenjie
CEntre de REcherches en MAthématiques de la DEcision [CEREMADE]
Abstract (EN)
In this paper, we study a regularised relaxed optimal control problem and, in particular, we are concerned with the case where the control variable is of large dimension. We introduce a system of mean-field Langevin equations, the invariant measure of which is shown to be the optimal control of the initial problem under mild conditions. Therefore, this system of processes can be viewed as a continuous-time numerical algorithm for computing the optimal control. As an application, this result endorses the solvability of the stochastic gradient descent algorithm for a wide class of deep neural networks.
Subjects / Keywords
Mean-Field Langevin Dynamics; Gradient Flow; Neural Networks