Convergence dynamics of Generative Adversarial Networks: the dual metric flows
Turinici, Gabriel (2021), Convergence dynamics of Generative Adversarial Networks: the dual metric flows, in Alberto Del BimboRita CucchiaraStan SclaroffGiovanni Maria FarinellaTao MeiMarco BertiniHugo Jair EscalanteRoberto Vezzani, Pattern Recognition. ICPR International Workshops and Challenges, Springer : Berlin Heidelberg, p. 619-634. 10.1007/978-3-030-68763-2_47
TypeCommunication / Conférence
Conference titleICPR: International Conference on Pattern Recognition
Conference cityMilan (Virtual Event)
Book titlePattern Recognition. ICPR International Workshops and Challenges
Book authorAlberto Del BimboRita CucchiaraStan SclaroffGiovanni Maria FarinellaTao MeiMarco BertiniHugo Jair EscalanteRoberto Vezzani
Number of pages741
MetadataShow full item record
CEntre de REcherches en MAthématiques de la DEcision [CEREMADE]
Abstract (EN)Fitting neural networks often resorts to stochastic (or similar) gradient descent which is a noise-tolerant (and efficient) resolution of a gradient descent dynamics. It outputs a sequence of networks parameters, which sequence evolves during the training steps. The gradient descent is the limit, when the learning rate is small and the batch size is infinite, of this set of increasingly optimal network parameters obtained during training. In this contribution, we investigate instead the convergence in the Generative Adversarial Networks used in machine learning. We study the limit of small learning rate, and show that, similar to single network training, the GAN learning dynamics tend, for vanishing learning rate to some limit dynamics. This leads us to consider evolution equations in metric spaces (which is the natural framework for evolving probability laws)that we call dual flows. We give formal definitions of solutions and prove the convergence. The theory is then applied to specific instances of GANs and we discuss how this insight helps understand and mitigate the mode collapse.
Subjects / KeywordsGAN; Metric flow; Generative network
Showing items related by title and author.
Buffa, Annalisa; Maday, Yvon; Patera, Anthony T.; Prud'Homme, Christophe; Turinici, Gabriel (2012) Article accepté pour publication ou publié