Show simple item record

dc.contributor.authorAraújo, Alexandre
dc.contributor.authorNegrevergne, Benjamin
HAL ID: 172154
ORCID: 0000-0002-7074-8167
dc.contributor.authorChevaleyre, Yann
dc.contributor.authorAtif, Jamal
HAL ID: 15689
dc.date.accessioned2020-05-06T09:52:07Z
dc.date.available2020-05-06T09:52:07Z
dc.date.issued2019
dc.identifier.urihttps://basepub.dauphine.fr/handle/123456789/20698
dc.language.isoenen
dc.subjectneural networks
dc.subject.ddc06.3en
dc.titleOn the Expressive Power of Deep Fully Circulant Neural Networks
dc.typeDocument de travail / Working paper
dc.description.abstractenIn this paper, we study deep fully circulant neural networks, that is deep neural networks in which all weight matrices are circulant ones. We show that these networks outperform the recently introduced deep networks with other types of structured layers. Besides introducing principled techniques for training these models, we provide theoretical guarantees regarding their expressivity. Indeed, we prove that the function space spanned by circulant networks of bounded depth includes the one spanned by dense networks with specific properties on their rank. We conduct a thorough experimental study to compare the performance of deep fully circulant networks with state of the art models based on structured matrices and with dense models. We show that our models achieve better accuracy than their structured alternatives while required 2x fewer weights as the next best approach. Finally we train deep fully circulant networks to build a compact and accurate models on a real world video classification dataset with over 3.8 million training examples.
dc.publisher.cityParisen
dc.relation.ispartofseriestitlePreprint Lamsade
dc.subject.ddclabelAraujoen
dc.description.ssrncandidatenon
dc.description.halcandidatenon
dc.description.readershiprecherche
dc.description.audienceInternational
dc.date.updated2020-12-17T08:50:12Z


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record