Show simple item record

dc.contributor.authorYao, Y.
dc.contributor.authorTarres, Pierre
dc.date.accessioned2014-07-17T11:24:40Z
dc.date.available2014-07-17T11:24:40Z
dc.date.issued2014
dc.identifier.urihttps://basepub.dauphine.fr/handle/123456789/13735
dc.language.isoenen
dc.subjectUpper bounden
dc.subjectProbabilistic logicen
dc.subjectKernelen
dc.subjectHilbert spaceen
dc.subjectEducational institutionsen
dc.subjectConvergenceen
dc.subjectApproximation methodsen
dc.subject.ddc519en
dc.titleOnline Learning as Stochastic Approximation of Regularization Paths: Optimality and Almost-sure Convergenceen
dc.typeArticle accepté pour publication ou publié
dc.description.abstractenIn this paper, an online learning algorithm is proposed as sequential stochastic approximation of a regularization path converging to the regression function in reproducing kernel Hilbert spaces (RKHSs). We show that it is possible to produce the best known strong (RKHS norm) convergence rate of batch learning, through a careful choice of the gain or step size sequences, depending on regularity assumptions on the regression function. The corresponding weak (mean square distance) convergence rate is optimal in the sense that it reaches the minimax and individual lower rates in the literature. In both cases we deduce almost sure convergence, using Bernstein-type inequalities for martingales in Hilbert spaces. To achieve this we develop a bias-variance decomposition similar to the batch learning setting; the bias consists in the approximation and drift errors along the regularization path, which display the same rates of convergence, and the variance arises from the sample error analysed as a (reverse) martingale difference sequence. The rates above are obtained by an optimal trade-off between the bias and the variance.en
dc.relation.isversionofjnlnameIEEE Transactions on Information Theory
dc.relation.isversionofjnlvolPPen
dc.relation.isversionofjnlissue99en
dc.relation.isversionofjnldate2014
dc.relation.isversionofdoihttp://dx.doi.org/10.1109/TIT.2014.2332531en
dc.relation.isversionofjnlpublisherIEEEen
dc.subject.ddclabelProbabilités et mathématiques appliquéesen
dc.relation.forthcomingnonen
dc.relation.forthcomingprintnonen


Files in this item

FilesSizeFormatView

There are no files associated with this item.

This item appears in the following Collection(s)

Show simple item record