Show simple item record

Convergence dynamics of Generative Adversarial Networks: the dual metric flows

hal.structure.identifierCEntre de REcherches en MAthématiques de la DEcision [CEREMADE]
dc.contributor.authorTurinici, Gabriel
HAL ID: 16
ORCID: 0000-0003-2713-006X
dc.date.accessioned2021-11-04T09:36:54Z
dc.date.available2021-11-04T09:36:54Z
dc.date.issued2021
dc.identifier.urihttps://basepub.dauphine.psl.eu/handle/123456789/22168
dc.language.isoenen
dc.subjectGANen
dc.subjectMetric flowen
dc.subjectGenerative networken
dc.subject.ddc519en
dc.titleConvergence dynamics of Generative Adversarial Networks: the dual metric flowsen
dc.typeCommunication / Conférence
dc.description.abstractenFitting neural networks often resorts to stochastic (or similar) gradient descent which is a noise-tolerant (and efficient) resolution of a gradient descent dynamics. It outputs a sequence of networks parameters, which sequence evolves during the training steps. The gradient descent is the limit, when the learning rate is small and the batch size is infinite, of this set of increasingly optimal network parameters obtained during training. In this contribution, we investigate instead the convergence in the Generative Adversarial Networks used in machine learning. We study the limit of small learning rate, and show that, similar to single network training, the GAN learning dynamics tend, for vanishing learning rate to some limit dynamics. This leads us to consider evolution equations in metric spaces (which is the natural framework for evolving probability laws)that we call dual flows. We give formal definitions of solutions and prove the convergence. The theory is then applied to specific instances of GANs and we discuss how this insight helps understand and mitigate the mode collapse.en
dc.identifier.citationpages619-634en
dc.relation.ispartoftitlePattern Recognition. ICPR International Workshops and Challengesen
dc.relation.ispartofeditorAlberto Del BimboRita CucchiaraStan SclaroffGiovanni Maria FarinellaTao MeiMarco BertiniHugo Jair EscalanteRoberto Vezzani
dc.relation.ispartofpublnameSpringeren
dc.relation.ispartofpublcityBerlin Heidelbergen
dc.relation.ispartofdate2021
dc.relation.ispartofpages741en
dc.relation.ispartofurl10.1007/978-3-030-68763-2en
dc.subject.ddclabelProbabilités et mathématiques appliquéesen
dc.relation.ispartofisbn978-3-030-68762-5en
dc.relation.conftitleICPR: International Conference on Pattern Recognitionen
dc.relation.confdate2021-01
dc.relation.confcityMilan (Virtual Event)en
dc.relation.confcountryItalyen
dc.relation.forthcomingnonen
dc.identifier.doi10.1007/978-3-030-68763-2_47en
dc.description.ssrncandidatenon
dc.description.halcandidatenonen
dc.description.readershiprechercheen
dc.description.audienceInternationalen
dc.relation.Isversionofjnlpeerreviewednonen
dc.date.updated2021-11-04T09:33:25Z
hal.author.functionaut


Files in this item

Thumbnail
Name:
2012.10410.pdf
Size:
481.1Kb
Format:
PDF
View/Open

This item appears in the following Collection(s)

Show simple item record