About contrastive unsupervised representation learning for classification and its convergence
Merad, Ibrahim; Yu, Yiyang; Bacry, Emmanuel; Gaïffas, Stéphane (2021), About contrastive unsupervised representation learning for classification and its convergence. https://basepub.dauphine.psl.eu/handle/123456789/22718
TypeDocument de travail / Working paper
External document linkhttps://hal.archives-ouvertes.fr/hal-03438767
Series titleCahier de recherche CEREMADE, Université Paris Dauphine-PSL
MetadataShow full item record
CEntre de REcherches en MAthématiques de la DEcision [CEREMADE]
Centre de Mathématiques Appliquées - Ecole Polytechnique [CMAP]
Laboratoire de Probabilités, Statistiques et Modélisations [LPSM (UMR_8001)]
Abstract (EN)Contrastive representation learning has been recently proved to be very efficient for selfsupervised training. These methods have been successfully used to train encoders which perform comparably to supervised training on downstream classification tasks. A few works have started to build a theoretical framework around contrastive learning in which guarantees for its performance can be proven. We provide extensions of these results to training with multiple negative samples and for multiway classification. Furthermore, we provide convergence guarantees for the minimization of the contrastive training error with gradient descent of an overparametrized deep neural encoder, and provide some numerical experiments that complement our theoretical findings.
Subjects / Keywordsunsupervised learning; constrative learning; deep neural networks; theoretical guarantees
Showing items related by title and author.
Kabeshova, Anastasiia; Yu, Yiyang; Lukacs, Bertrand; Bacry, Emmanuel; Gaïffas, Stéphane (2020) Article accepté pour publication ou publié
Screening anxiolytics, hypnotics, antidepressants and neuroleptics for bone fracture risk among elderly: a nation-wide dynamic multivariate self-control study using the SNDS claims database Morel, Maryan; Bouyer, Benjamin; Guilloux, Agathe; LAANANI, Moussa; Leroy, Fanny; Nguyen, Dinh Phong; Sebiat, Youcef; Bacry, Emmanuel; Gaïffas, Stéphane (2021) Document de travail / Working paper
Morel, Maryan; Bacry, Emmanuel; Gaïffas, Stéphane; Guilloux, Agathe; Leroy, Fanny (2019) Article accepté pour publication ou publié