检索结果-内蒙古大学图书馆

Stability of randomized learning algorithms

JOURNAL OF MACHINE learning RESEARCH 2005年第1期6卷 55-79页

作者： Elisseeff, A Evgeniou, T Pontil, M IBM Zurich Res Lab CH-8803 Ruschlikon Switzerland INSEAT Technol Management F-77300 Fontainebleau France UCL Dept Comp Sci London WC1E England

We extend existing theory on stability, namely how much changes in the training data influence the estimated models, and generalization performance of deterministic learning algorithms to the case of randomized algorithms. We give formal definitions of stability for randomized algorithms and prove non-asymptotic bounds on the difference between the empirical and expected error as well as the leave-one-out and expected error of such algorithms that depend on their random stability. The setup we develop for this purpose can be also used for generally studying randomized learning algorithms. We then use these general results to study the effects of bagging on the stability of a learning method and to prove non-asymptotic bounds on the predictive performance of bagging which have not been possible to prove with the existing theory of stability for deterministic learning algorithms.(1)

关键词： stability randomized learning algorithms sensitivity analysis bagging bootstrap methods generalization error leave-one-out error

来源：评论

学校读者我要写书评

暂无评论

Strong and Weak Stability of randomized learning algorithms

Strong and Weak Stability of Randomized Learning Algorithms

引用

14th IEEE International Conference on Communication Technology (ICCT)

作者： Luo, Ke Jia, Zhiyang Gao, Wei Shaoyang Univ Shaoyang Univ Lib Shaoyang Hunan Peoples R China Yunnan Univ Tourism & Culture Coll Lijiang Yunnan Peoples R China Yunnan Normal Univ Sch Informat Kunming Yunnan Peoples R China

ISBN: (纸本)9781467321013;9781467321006

An algorithm is called stable at a training set S if any change of a single point in S yields only a small change in the output. Stability of the learning algorithm is necessary for learnability in the supervised classification and regression setting. In this paper, we give formal definitions of strong and weak stability for randomized algorithms and prove non-asymptotic bounds on the difference between the empirical and expected error.

关键词： strong stability weak stability randomized learning algorithms generalization error empirical error

来源：评论

学校读者我要写书评

暂无评论

Strong and Weak Stability of randomized learning algorithms

Strong and Weak Stability of Randomized Learning Algorithms

引用

2012 IEEE 14th International Conference on Communication Technology

作者： Ke Luo Zhiyang Jia Wei Gao Shaoyang University Library Shaoyang UniversityShaoyang Tourism and Culture College Yunnan University School of Information Yunnan Normal University

ISBN: (纸本)9781467321006

An algorithm is called stable at a training set S if any change of a single point in S yields only a small change in the *** of the learning algorithm is necessary for learnability in the supervised classification and regression *** this paper,we give formal definitions of strong and weak stability for randomized algorithms and prove non-asymptotic bounds on the difference between the empirical and expected error.

关键词： strong stability weak stability randomized learning algorithms generalization error empirical error

来源：评论

学校读者我要写书评

暂无评论

randomized learning: Generalization performance of old and new theoretically grounded algorithms

引用

NEUROCOMPUTING 2018年 298卷 21-33页

作者： Oneto, Luca Cipollini, Francesca Ridella, Sandro Anguita, Davide Univ Genoa DIBRIS Via Opera Pia 13 I-16145 Genoa Italy Univ Genoa DITEN Via Opera Pia 11a I-16145 Genoa Italy

In the context of assessing the generalization abilities of a randomized model or learning algorithm, PAC-Bayes and Differential Privacy (DP) theories are the state-of-the-art tools. For this reason, in this paper, we will develop tight DP-based generalization bounds, which improve over the current state-of-the-art ones both in terms of constants and rate of convergence. Moreover, we will also prove that some old and new randomized algorithm, show better generalization performances with respect to their non private counterpart, if the DP is exploited for assessing their generalization ability. Results on a series of algorithms and real world problems show the practical validity of the achieved theoretical results. (C) 2018 Elsevier B.V. All rights reserved.

关键词： Generalization performances randomized models randomized learning algorithms Differential privacy PAC-Bayes Distribution dependent prior Data dependent posterior

来源：评论

学校读者我要写书评

暂无评论

Improving randomized learning of Feedforward Neural Networks by Appropriate Generation of Random Parameters 15th

Improving Randomized Learning of Feedforward Neural Networks...

引用

15th International Work-Conference on Artificial Neural Networks (IWANN)

作者： Dudek, Grzegorz Czestochowa Tech Univ Dept Elect Engn Czestochowa Poland

ISBN: (纸本)9783030205218;9783030205201

In this work, a method of random parameters generation for randomized learning of a single-hidden-layer feedforward neural network is proposed. The method firstly, randomly selects the slope angles of the hidden neurons activation functions from an interval adjusted to the target function, then randomly rotates the activation functions, and finally distributes them across the input space. For complex target functions the proposed method gives better results than the approach commonly used in practice, where the random parameters are selected from the fixed interval. This is because it introduces the steepest fragments of the activation functions into the input hypercube, avoiding their saturation fragments.

关键词： Function approximation Feedforward Neural Networks Neural networks with random hidden nodes randomized learning algorithms

来源：评论

学校读者我要写书评

暂无评论

Autoencoder based randomized learning of Feedforward Neural Networks for Regression

Autoencoder based Randomized Learning of Feedforward Neural ...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Dudek, Grzegorz Czestochowa Tech Univ Dept Elect Engn Czestochowa Poland

ISBN: (纸本)9780738133669

Feedforward neural networks are widely used as universal predictive models to fit data distribution. Common gradient-based learning, however, suffers from many drawbacks making the training process ineffective and time-consuming. Alternative randomized learning does not use gradients but selects hidden node parameters randomly. This makes the training process extremely fast. However, the problem in randomized learning is how to determine the random parameters. A recently proposed method uses autoencoders for unsupervised parameter learning. This method showed superior performance on classification tasks. In this work, we apply this method to regression problems, and, finding that it has some drawbacks, we show how to improve it. We propose a learning method of autoencoders that controls the produced random weights. We also propose how to determine the biases of hidden nodes. We empirically compare autoencoder based learning with other randomized learning methods proposed recently for regression and find that despite the proposed improvement of the autoencoder based learning, it does not outperform its competitors in fitting accuracy. Moreover, the method is much more complex than its competitors.

关键词： autoencoder feedforward neural networks randomized learning algorithms

来源：评论

学校读者我要写书评

暂无评论

Data-Driven randomized learning of Feedforward Neural Networks

Data-Driven Randomized Learning of Feedforward Neural Networ...

引用

International Joint Conference on Neural Networks (IJCNN) held as part of the IEEE World Congress on Computational Intelligence (IEEE WCCI)

作者： Dudek, Grzegorz Czestochowa Tech Univ Dept Elect Engn Czestochowa Poland

ISBN: (纸本)9781728169262

randomized methods of neural network learning suffer from a problem with the generation of random parameters as they are difficult to set optimally to obtain a good projection space. The standard method draws the parameters from a fixed interval which is independent of the data scope and activation function type. This does not lead to good results in the approximation of the strongly nonlinear functions. In this work, a method which adjusts the random parameters, representing the slopes and positions of the sigmoids, to the target function features is proposed. The method randomly selects the input space regions, places the sigmoids in these regions and then adjusts the sigmoid slopes to the local fluctuations of the target function. A bias-variance tradeoff is controlled by the region size. This brings very good results in the approximation of the complex target functions when compared to the standard fixed interval method and other methods recently proposed in the literature.

关键词： data-driven randomized learning feedforward neural networks neural networks with random hidden nodes randomized learning algorithms

来源：评论

学校读者我要写书评

暂无评论

Sensitivity Analysis of the Neural Networks randomized learning 18th

Sensitivity Analysis of the Neural Networks Randomized Learn...

引用

18th International Conference on Artificial Intelligence and Soft Computing (ICAISC)

作者： Dudek, Grzegorz Czestochowa Tech Univ Elect Engn Fac Czestochowa Poland

ISBN: (纸本)9783030209124;9783030209117

randomized algorithms for learning feedforward neural networks are increasingly used in practice. They offer very speed training because the only parameters that are learned are the output weights. Parameters of hidden neurons are generated randomly once and need not to be adjusted. The key issue in randomized learning algorithms is to generate parameters in a right way to ensure good approximation and generalization properties of the network. Recently the method of generating hidden nodes parameters was proposed [1], which ensures better adjustment of the random parameters to the target function and better distribution of neurons in the input space, when comparing to the previous approaches. In this work the new method is tested in terms of sensitivity to the number of neurons, noise in data and data deficit. Experiments shows better results for the new method in comparison to the existing approach of generating random parameters of the network.

关键词： randomized learning algorithms Neural networks with random hidden nodes Feedforward neural networks

来源：评论

学校读者我要写书评

暂无评论

A constructive approach to data-driven randomized learning for feedforward neural networks

引用

APPLIED SOFT COMPUTING 2021年 112卷 107797-107797页

作者： Dudek, Grzegorz Czestochowa Tech Univ Fac Elect Engn 17 Armii Krajowej Ave PL-42200 Czestochowa Poland

There is an issue with the way in which feedforward neural networks with random hidden nodes generate random parameters in order to obtain a good projection space. Typically, random weights and biases are both drawn from the same interval, which is misguided as they have different functions. Recently, more sophisticated methods of random parameters generation have been developed, such as the data-driven approach, where the sigmoids are placed in randomly selected regions of the input space and then their slopes are adjusted to the local fluctuations of the target function. In this work, we propose a new constructive data-driven method that builds iteratively the network architecture. This method successively generates new candidate hidden nodes and accepts them when the training error falls significantly. The threshold of acceptance is adapted throughout training, accepting at the beginning of the training process only those nodes which lead to the largest reductions in error. In the next stages, the threshold is successively reduced to accept only those nodes which model the target function details more accurately. This leads to a more compact network architecture, as it includes only "significant" nodes. It is worth noting that redundant, random nodes, which are usually generated by existing randomized learning methods, are not accepted by the proposed method. We empirically compared our approach with several alternative methods, including its predecessor, competitive randomized learning solutions, a gradient-based network and a generalized additive model. We found that our proposed approach outperformed its competitors in terms of fitting accuracy. (C) 2021 Elsevier B.V. All rights reserved.

关键词： Data-driven randomized learning Feedforward neural networks Neural networks with random hidden nodes randomized learning algorithms

来源：评论

学校读者我要写书评

暂无评论

Generating random weights and biases in feedforward neural networks with random hidden nodes

引用

INFORMATION SCIENCES 2019年 481卷 33-56页

作者： Dudek, Grzegorz Czestochowa Tech Univ Fac Elect Engn 17 Armii Krajowej Ave PL-42200 Czestochowa Poland

Neural networks with random hidden nodes have gained increasing interest from researchers and practical applications. This is due to their unique features such as very fast training and universal approximation property. In these networks the weights and biases of hidden nodes determining the nonlinear feature mapping are set randomly and are not learned. Appropriate selection of the intervals from which weights and biases are selected is extremely important. This topic has not yet been sufficiently explored in the literature. In this work a method of generating random weights and biases is proposed. This method generates the parameters of the hidden nodes in such a way that nonlinear fragments of the activation functions are located in the input space regions with data and can be used to construct the surface approximating a nonlinear target function. The weights and biases are dependent on the input data range and activation function type. The proposed methods allows us to control the generalization degree of the model. These all lead to improvement in approximation performance of the network. Several experiments show very promising results. (C) 2018 Elsevier Inc. All rights reserved.

关键词： Activation functions Function approximation Feedforward neural networks Neural networks with random hidden nodes randomized learning algorithms

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：