Show simple item record

dc.contributor.authorAraujo, Alexandre
dc.contributor.authorNegrevergne, Benjamin
dc.contributor.authorChevaleyre, Yann
dc.contributor.authorAtif, Jamal
dc.date.accessioned2020-05-06T09:52:07Z
dc.date.available2020-05-06T09:52:07Z
dc.date.issued2019
dc.identifier.urihttps://basepub.dauphine.fr/handle/123456789/20698
dc.language.isoenen
dc.subjectneural networksen
dc.subject.ddc06.3en
dc.titleOn the Expressive Power of Deep Fully Circulant Neural Networksen
dc.typeDocument de travail / Working paper
dc.description.abstractenIn this paper, we study deep fully circulant neural networks, that is deep neural networks in which all weight matrices are circulant ones. We show that these networks outperform the recently introduced deep networks with other types of structured layers. Besides introducing principled techniques for training these models, we provide theoretical guarantees regarding their expressivity. Indeed, we prove that the function space spanned by circulant networks of bounded depth includes the one spanned by dense networks with specific properties on their rank. We conduct a thorough experimental study to compare the performance of deep fully circulant networks with state of the art models based on structured matrices and with dense models. We show that our models achieve better accuracy than their structured alternatives while required 2x fewer weights as the next best approach. Finally we train deep fully circulant networks to build a compact and accurate models on a real world video classification dataset with over 3.8 million training examples.en
dc.publisher.namePreprint Lamsadeen
dc.publisher.cityParisen
dc.relation.ispartofseriestitlePreprint Lamsadeen
dc.subject.ddclabelAraujoen
dc.identifier.citationdate2019
dc.description.ssrncandidatenonen
dc.description.halcandidatenonen
dc.description.readershiprechercheen
dc.description.audienceInternationalen
dc.date.updated2020-05-06T09:40:39Z
hal.person.labIds989
hal.person.labIds989
hal.person.labIds989
hal.person.labIds989


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record