检索结果-内蒙古大学图书馆

Duality for neural networks through reproducing kernel banach spaces

APPLIED AND COMPUTATIONAL HARMONIC ANALYSIS 2025年 78卷

作者： Spek, Len Heeringa, Tjeerd Jan Schwenninger, Felix Brune, Christoph Univ Twente Math Imaging & AI POB 217 NL-7500 AE Enschede Netherlands Univ Twente Math Syst Theory POB 217 NL-7500 AE Enschede Netherlands

reproducing kernel Hilbert spaces (RKHS) have been a very successful tool in various areas of machine learning. Recently, Barron spaces have been used to prove bounds on the generalisation error for neural networks. Unfortunately, Barron spaces cannot be understood in terms of RKHS due to the strong nonlinear coupling of the weights. This can be solved by using the more general reproducing kernel banach spaces (RKBS). We show that these Barron spaces belong to a class of integral RKBS. This class can also be understood as an infinite union of RKHS spaces. Furthermore, we show that the dual space of such RKBSs, is again an RKBS where the roles of the data and parameters are interchanged, forming an adjoint pair of RKBSs including a reproducing kernel. This allows us to construct the saddle point problem for neural networks, which can be used in the whole field of primal-dual optimisation.

关键词： reproducing kernel banach spaces Neural networks Duality Functional analysis Theory of machine learning

来源：评论

学校读者我要写书评

暂无评论

Understanding neural networks with reproducing kernel banach spaces

引用

APPLIED AND COMPUTATIONAL HARMONIC ANALYSIS 2023年 62卷 194-236页

作者： Bartolucci, Francesca De Vito, Ernesto Rosasco, Lorenzo Vigogna, Stefano Swiss Fed Inst Technol Dept Math SAM Zurich Switzerland Univ Genoa MaLGa DIMA Genoa Italy Univ Genoa MaLGa DIBRIS Genoa Italy CBMM MIT Cambridge MA USA IIT Genoa Italy Univ Roma Tor Vergata RoMaDS Dept Math Rome Italy

Characterizing the function spaces corresponding to neural networks can provide a way to understand their properties. In this paper we discuss how the theory of reproducing kernel banach spaces can be used to tackle this challenge. In particular, we prove a representer theorem for a wide class of reproducing kernel banach spaces that admit a suitable integral representation and include one hidden layer neural networks of possibly infinite width. Further, we show that, for a suitable class of ReLU activation functions, the norm in the corresponding reproducing kernel banach space can be characterized in terms of the inverse Radon transform of a bounded real measure, with norm given by the total variation norm of the measure. Our analysis simplifies and extends recent results in [45,36,37]. (c) 2022 Elsevier Inc. All rights reserved.

关键词： Neural networks Representer theorem Radon transform reproducing kernel banach spaces

来源：评论

学校读者我要写书评

暂无评论

On reproducing kernel banach spaces:Generic Definitions and Unified Framework of Constructions

引用

Acta Mathematica Sinica,English Series 2022年第8期38卷 1459-1483页

作者： Rong Rong LIN Hai Zhang ZHANG Jun ZHANG School of Mathematics and Statistics Guangdong University of TechnologyGuangzhou 510520P.R.China School of Mathematics(Zhuhai) and Guangdong Province Key Laboratory of Computational ScienceSun Yat-sen UniversityZhuhai 519082P.R.China Department of Psychology and Department of Statistics University of MichiganAnn ArborMI48109USA

Recently,there has been emerging interest in constructing reproducing kernel banach spaces(RKBS)for applied and theoretical purposes such as machine learning,sampling reconstruction,sparse approximation and functional *** constructions include the reflexive RKBS via a bilinear form,the semi-inner-product RKBS,the RKBS with?~1 norm,the p-norm RKBS via generalized Mercer kernels,*** definitions of RKBS and the associated reproducing kernel in those references are dependent on the ***,relations among those constructions are *** explore a generic definition of RKBS and the reproducing kernel for RKBS that is independent of ***,we propose a framework of constructing RKBSs that leads to new RKBSs based on Orlicz spaces and unifies existing constructions mentioned above via a continuous bilinear form and a pair of feature ***,we develop representer theorems for machine learning in RKBSs constructed in our framework,which also unifies representer theorems in existing RKBSs.

关键词： reproducing kernel banach spaces feature maps reproducing kernels machine learning the representer theorem

来源：评论

学校读者我要写书评

暂无评论

A converse sampling theorem in reproducing kernel banach spaces

引用

SAMPLING THEORY SIGNAL PROCESSING AND DATA ANALYSIS 2022年第2期20卷 1-19页

作者： Centeno, Hernan Medina, Juan Miguel Univ Buenos Aires Fac Ingn Dept Matemat Paseo Colon 850 RA-1063 Buenos Aires Argentina Univ Buenos Aires Fac Ingn Dept Matemat Paseo Colon 850 RA-1063 Buenos Aires Argentina Consejo Nacl Invest Cient & Tecn IAM Paseo Colon 850 RA-1063 Buenos Aires Argentina Pontificia Univ Catolica Argentina LaBIS Buenos Aires Argentina

We present a converse Kramer type sampling theorem over semi-inner product reproducing kernel banach spaces. Assuming that a sampling expansion holds for every f belonging to a semi-inner product reproducing kernel banach space B for a fixed sequence of interpolating functions {a(j)(-1)S(j)(t)}(j )and a subset of sampling points {t(j)}(j), it results that such sequence must be a X-d(& lowast;)-Riesz basis and a sampling basis for the space. Moreover, there exists an equivalent (in norm) reproducing kernel banach space with a reproducing kernel G(samp) such that {a(-1)(j)G(samp)(t(j),.)}(j) and {a(j)(-1)S(j)(.)}(j) are biorthogonal. These results are a generalization of some known results over reproducing kernel Hilbert spaces.

关键词： Sampling basis Non-uniform sampling reproducing kernel Hilbert spaces reproducing kernel banach spaces Kramer sampling theorems Semi-inner products Kramer sampling theorems Semi-inner products X-d-frames X-d-Riesz basis

来源：评论

学校读者我要写书评

暂无评论

SPARSE REGULARIZED LEARNING IN THE reproducing kernel banach spaces WITH THE l¹ NORM

引用

MATHEMATICAL FOUNDATIONS OF COMPUTING 2020年第3期3卷 205-218页

作者： Lin, Ying Lin, Rongrong Ye, Qi South China Normal Univ Sch Math Sci Guangzhou 510631 Guangdong Peoples R China Sun Yat Sen Univ Sch Data & Comp Sci Guangzhou 510006 Guangdong Peoples R China

We present a sparse representer theorem for regularization networks in a reproducing kernel banach space with the l(1) norm by the theory of convex analysis. The theorem states that extreme points of the solution set of regularization networks in such a sparsity-promoting space belong to the span of kernel functions centered on at most n adaptive points of the input space, where n is the number of training data. Under the Lebesgue constant assumptions on reproducing kernels, we can recover the relaxed representer theorem and the exact representer theorem in that space in the literature. Finally, we perform numerical experiments for synthetic data and real-world benchmark data in the reproducing kernel banach spaces with the l(1) norm and the reproducing kernel Hilbert spaces both with Laplacian kernels. The numerical performance demonstrates the advantages of sparse regularized learning.

关键词： Sparse representer theorems reproducing kernel banach spaces Laplacian kernels l(1) norm

来源：评论

学校读者我要写书评

暂无评论

Multi-task Learning in vector-valued reproducing kernel banach spaces with the l¹ norm

引用

JOURNAL OF COMPLEXITY 2021年 63卷 101514-101514页

作者： Lin, Rongrong Song, Guohui Zhang, Haizhang Sun Yat Sen Univ Sch Data & Comp Sci Guangzhou Peoples R China Old Dominion Univ Dept Math & Stat 2300 Engn & Computat Sci Bldg Norfolk VA 23529 USA Sun Yat Sen Univ Guangdong Prov Key Lab Computat Sci Guangzhou Peoples R China

Targeting at sparse multi-task learning, we consider regularization models with an l(1) penalty on the coefficients of kernel functions. In order to provide a kernel method for this model, we construct a class of vector-valued reproducing kernel banach spaces with the l(1) norm. The notion of multi-task admissible kernels is proposed so that the constructed spaces could have desirable properties including the crucial linear representer theorem. Such kernels are related to bounded Lebesgue constants of a kernel interpolation question. We study the Lebesgue constant of multi-task kernels and provide examples of admissible kernels. Furthermore, we present numerical experiments for both synthetic data and real-world benchmark data to demonstrate the advantages of the proposed construction and regularization models. (C) 2020 Elsevier Inc. All rights reserved.

关键词： reproducing kernel banach spaces Admissible multi-task kernels Lebesgue constants Representer theorems

来源：评论

学校读者我要写书评

暂无评论

reproducing kernel banach spaces with the l¹ norm

引用

APPLIED AND COMPUTATIONAL HARMONIC ANALYSIS 2013年第1期34卷 96-116页

作者： Song, Guohui Zhang, Haizhang Hickernell, Fred J. Arizona State Univ Sch Math & Stat Sci Tempe AZ 85287 USA Sun Yat Sen Univ Sch Math & Computat Sci Guangzhou 510275 Guangdong Peoples R China Sun Yat Sen Univ Guangdong Prov Key Lab Computat Sci Guangzhou 510275 Guangdong Peoples R China IIT Dept Appl Math Chicago IL 60616 USA

Targeting at sparse learning, we construct banach spaces B of functions on an input space X with the following properties: (1) B possesses an l(1) norm in the sense that B is isometrically isomorphic to the banach space of integrable functions on X with respect to the counting measure;(2) point evaluations are continuous linear functionals on B and are representable through a bilinear form with a kernel function;and (3) regularized learning schemes on B satisfy the linear representer theorem. Examples of kernel functions admissible for the construction of such spaces are given. (C) 2012 Elsevier Inc. All rights reserved.

关键词： reproducing kernel banach spaces Sparse learning Lasso Basis pursuit Regularization The representer theorem The Brownian bridge kernel The exponential kernel

来源：评论

学校读者我要写书评

暂无评论

Sampling in reproducing kernel banach spaces on Lie groups

引用

JOURNAL OF APPROXIMATION THEORY 2012年第1期164卷 179-203页

作者： Christensen, Jens Gerlach Tufts Univ Dept Math Medford MA 02155 USA

We present sampling theorems for reproducing kernel banach spaces on Lie groups. Recent approaches to this problem rely on integrability of the kernel and its local oscillations. In this paper, we replace these integrability conditions by requirements on the derivatives of the reproducing kernel and, in particular, oscillation estimates are found using derivatives of the reproducing kernel. This provides a convenient path to sampling results on reproducing kernel banach spaces. Finally, these results are used to obtain frames and atomic decompositions for banach spaces of distributions stemming from a cyclic representation. It is shown that this process is particularly easy, when the cyclic vector is a Garding vector for a square integrable representation. (C) 2011 Elsevier Inc. All rights reserved.

关键词： Sampling reproducing kernel banach spaces Lie groups Coorbits

来源：评论

学校读者我要写书评

暂无评论

Generalized support vector regression: Duality and tensor-kernel representation

引用

ANALYSIS AND APPLICATIONS 2020年第1期18卷 149-183页

作者： Salzo, Saverio Suykens, Johan A. K. Ist Italiano Tecnol CSML Via E Melen 83 I-16152 Genoa Italy Katholieke Univ Leuven ESAT STADIUS Kasteelpk Arenberg 10 B-3001 Heverlee Belgium

In this paper, we study the variational problem associated to support vector regression in banach function spaces. Using the Fenchet-Roclatfellar duality theory, we give an explicit formulation of the dual problem as well as of the related optimality conditions. Moreover, we provide a new computational framework for solving the problem which relies on a tensor-kernel representation. This analysis overcomes the typical difficulties connected to learning in banach spaces. We finally present a large class of tensor-kernels to which our theory fully applies: power series tensor kernels. This type of kernels describes banach spaces of analytic functions and includes generalizations of the exponential and polynomial kernels as well as, in the complex case, generalizations of the Szego and Bergman kernels.

关键词： Support vector regression regularized empirical risk reproducing kernel banach spaces tensors Fenchel-Rockatfellar duality

来源：评论

学校读者我要写书评

暂无评论

When is there a representer theorem? Reflexive banach spaces

引用

ADVANCES IN COMPUTATIONAL MATHEMATICS 2021年第4期47卷 54-54页

作者： Schlegel, Kevin UCL Dept Math 25 Gordon St London WC1H 0AY England

We consider a general regularised interpolation problem for learning a parameter vector from data. The well-known representer theorem says that under certain conditions on the regulariser there exists a solution in the linear span of the data points. This is at the core of kernel methods in machine learning as it makes the problem computationally tractable. Most literature deals only with sufficient conditions for representer theorems in Hilbert spaces and shows that the regulariser being norm-based is sufficient for the existence of a representer theorem. We prove necessary and sufficient conditions for the existence of representer theorems in reflexive banach spaces and show that any regulariser has to be essentially norm-based for a representer theorem to exist. Moreover, we illustrate why in a sense reflexivity is the minimal requirement on the function space. We further show that if the learning relies on the linear representer theorem, then the solution is independent of the regulariser and in fact determined by the function space alone. This in particular shows the value of generalising Hilbert space learning theory to banach spaces.

关键词： Representer theorem Regularised interpolation Regularisation kernel methods reproducing kernel banach spaces

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：