检索结果-内蒙古大学图书馆

Convergence of Constant Step Stochastic Gradient Descent for Non-Smooth Non-Convex Functions

SET-VALUED AND VARIATIONAL ANALYSIS 2022年第3期30卷 1117-1147页

作者： Bianchi, Pascal Hachem, Walid Schechtman, Sholom IP Paris Telecom Paris LTCI Paris France Univ Gustave Eiffel CNRS LIGM F-77454 Marne La Vallee France

This paper studies the asymptotic behavior of the constant step Stochastic Gradient Descent for the minimization of an unknown function, defined as the expectation of a non convex, non smooth, locally Lipschitz random function. As the gradient may not exist, it is replaced by a certain operator: a reasonable choice is to use an element of the Clarke subdifferential of the random function;another choice is the output of the celebrated backpropagation algorithm, which is popular amongst practioners, and whose properties have recently been studied by Bolte and Pauwels. Since the expectation of the chosen operator is not in general an element of the Clarke subdifferential of the mean function, it has been assumed in the literature that an oracle of the Clarke subdifferential of the mean function is available. As a first result, it is shown in this paper that such an oracle is not needed for almost all initialization points of the algorithm. Next, in the small step size regime, it is shown that the interpolated trajectory of the algorithm converges in probability (in the compact convergence sense) towards the set of solutions of a particular differential inclusion: the subgradient flow. Finally, viewing the iterates as a Markov chain whose transition kernel is indexed by the step size, it is shown that the invariant distribution of the kernel converge weakly to the set of invariant distribution of this differential inclusion as the step size tends to zero. These results show that when the step size is small, with large probability, the iterates eventually lie in a neighborhood of the critical points of the mean function.

关键词： Clarke subdifferential backpropagation algorithm Differential inclusions Non convex and non smooth optimization Stochastic approximation

来源：评论

学校读者我要写书评

暂无评论

Complex Function Approximation Based on Multi-ANN Approach

引用

Journal of Systems Engineering and Electronics 1995年第2期6卷 22-31页

作者： Li Renhou & Gao Feng (Institute of Systems Engineering, Xi’an Jiaotong University, Xi’an 710049, China) Institute of Systems Engineering Xi'an Jiaotong University Xi'an 710049 China

This paper presents a multi-ANN approximation approach to approximate complex non-linear function. Comparing with single-ANN methods the proposed approach improves and increases the approximation and generalization ability, and adaptability greatly in learning processes of networks. The simulation results have been shown that the method can be applied to the modeling and identification of complex dynamic control systems.

关键词： backpropagation algorithm Multilayer perception Complex function approximation Input space partitioning.

来源：评论

学校读者我要写书评

暂无评论

Novel approaches to determine age and gender from dental x-ray images by using multiplayer perceptron neural networks and image processing techniques

引用

CHAOS SOLITONS & FRACTALS 2019年 120卷 127-138页

作者： Avuclu, Emre Basciftci, Fatih Aksaray Univ Dept Comp Technol & Comp Programming Aksaray Turkey Selcuk Univ Dept Comp Engn Technol Fac TR-42003 Selcuklu Konya Turkey

It may be necessary to determine the identity or gender of a person for any reason (disasters, inheritance etc.). In such cases, forensic medical institutions are asked for help. Forensic science institutions try to estimate the age of people's teeth and bones. In this study, a novel algorithm was developed to keep these predictions at the highest level and to obtain definite results. The data base of 162 different tooth classes is created manually. All image sizes are 150x150 pixels. First, image preprocessing techniques have been applied to teeth images. These preprocessing techniques were first applied to teeth images. After this process, the segmentation process of the teeth images was performed to extract the feature by novel segmentation algorithm. Segmentation can be done automatically and dynamically. Numerical data obtained as a result of feature extraction from dental images is presented as an inputs to Multi layer perceptron neural network. In application, feature reduction can be performed. Thanks to the originally developed algorithm, the highest success rates were obtained with the highest 99.9% (full segment) and 100% (notfull segment) classification. After classification, for many dental groups the age estimate is performed with zero error. Application was developed as a multidisciplinary study. (C) 2019 Elsevier Ltd. All rights reserved.

关键词： Age estimation backpropagation algorithm Multilayer perceptron Panoramic x-ray Image processing techniques

来源：评论

学校读者我要写书评

暂无评论

End-to-end training algorithm for conceptor-based neural networks

引用

ELECTRONICS LETTERS 2018年第15期54卷 924-925页

作者： Qian, Guangwu Zhang, Lei Zhang, Qianjun Sichuan Univ Coll Comp Sci Machine Intelligence Lab Chengdu Sichuan Peoples R China

Since a conceptor, achieving direction-selective damping of high-dimensional network signals, usually takes the form of a projection matrix and is deduced analytically, a conceptor-based neural network is thought to be untrainable with backpropagation and gradient-descent algorithms from end to end. It limits the application of conceptors. To address this issue, an algorithm is proposed to train conceptor-based neural networks from end to end with gradient-descent algorithms. To the best of the authors' knowledge, it is the first work of such an end-to-end training algorithm. To develop this algorithm, a softmax-like loss function involved with conceptors is constructed empirically. Based on this loss function, corresponding gradients are deduced by using backpropagation method so that it is possible to train conceptor neural networks from end to end with a gradient-descent algorithm. Several experiments are conducted to show the feasibility and effectiveness of the proposed training algorithm.

关键词： recurrent neural nets backpropagation gradient methods matrix algebra end-to-end training algorithm conceptor-based neural networks direction-selective damping high-dimensional network signals projection matrix backpropagation algorithm gradient-descent algorithm softmax-like loss function recurrent neural networks

来源：评论

学校读者我要写书评

暂无评论

A novel Chebyshev neural network approach for solving singular arbitrary order Lane-Emden equation arising in astrophysics

引用

NETWORK-COMPUTATION IN NEURAL SYSTEMS 2020年第1-4期31卷 142-165页

作者： Mall, Susmita Chakraverty, S. Natl Inst Technol Rourkela Dept Math Rourkela 769008 Odisha India

The motivation of this investigation is to develop a single-layer Chebyshev Neural Network (ChNN) model to handle singular fractional (arbitrary)-order Lane-Emden type equations. These equations are well-known application problems of astrophysics and quantum mechanics. Fractional Lane-Emden equations are singular so it is very difficult to solve analytically. Thus, an efficient method is required to handle the above equations. Here, our main aim is to use a single-layer ChNN model for solving fractional Lane-Emden equations. ChNN model is one kind of Functional Link Neural Network (FLNN) in which the hidden layer is replaced by a functional expansion block of the input pattern using orthogonalshifted Chebyshev polynomials (SChP). Thus, the network parameters of ChNN are less than the Multi-Layer Artificial Neural Network (MLANN). We have considered factional-order singular nonlinear problems of astrophysics to show the computational effort of the proposed method. Back Propagation algorithm of the unsupervised version has been considered for minimizing the error function and updating the weights of the ChNN model. Computed results are displayed in terms of tables and graphs.

关键词： Chebyshev neural network fractional Lane-Emden equation backpropagation algorithm conformable fractional derivative

来源：评论

学校读者我要写书评

暂无评论

Nonlinear complex-valued extensions of Hebbian learning: An essay

引用

NEURAL COMPUTATION 2005年第4期17卷 779-838页

作者： Fiori, S Univ Perugia Fac Ingn Polo Sci & Didatt Ternano I-05100 Terni Italy

The Hebbian paradigm is perhaps the best-known unsupervised learning theory in connectionism. It has inspired wide research activity in the artificial neural network field because it embodies some interesting properties such as locality and the capability of being applicable to the basic weight-and-surn structure of neuron models. The plain Hebbian principle, however, also presents some inherent theoretical limitations that make it impractical in most cases. Therefore, modifications of the basic Hebbian learning paradigm have been proposed over the past 20 years in order to design profitable signal and data processing algorithms. Such modifications led to the principal component analysis type class of learning rules along with their nonlinear extensions. The aim of this review is primarily to present part of the existing fragmented material in the field of principal component learning within a unified view and contextually to motivate and present extensions of previous works on Hebbian learning to complex-weighted linear neural networks. This work benefits from previous studies on linear signal decomposition by artificial neural networks, nonquadratic component optimization and reconstruction error definition, neural parameters adaptation by constrained optimization of learning criteria of complex-valued arguments, and orthonormality expression via the insertion of topological elements in the networks or by modifying the network learning criterion. In particular, the learning principles considered here and their analysis concern complex-valued principal/minor component/subspace linear/nonlinear rules for complex-weighted neural structures, both feedforward and laterally connected.

关键词： INDEPENDENT COMPONENT ANALYSIS LINEAR NEURAL NETWORKS BLIND-DECONVOLUTION MINOR COMPONENTS LEAST-SQUARES backpropagation algorithm PRINCIPAL SEPARATION INFORMATION EXTRACTION

来源：评论

学校读者我要写书评

暂无评论

Water quality assessment using diatom assemblages and advanced modelling techniques

引用

FRESHWATER BIOLOGY 2004年第2期49卷 208-220页

作者： Gevrey, M Rimet, F Park, YS Giraudel, JL Ector, L Lek, S Univ Toulouse 3 LADYBIO CNRS UMR F-31062 Toulouse France CREBS Ctr Rech Publ Gabriel Lippmann Luxembourg Luxembourg

1. Two types of artificial neural networks procedures were used to define and predict diatom assemblage structures in Luxembourg streams using environmental data. 2. Self-organising maps (SOM) were used to classify samples according to their diatom composition, and multilayer perceptron with a backpropagation learning algorithm (BPN) was used to predict these assemblages using environmental characteristics of each sample as input and spatial coordinates (X and Y) of the cell centres of the SOM map identified as diatom assemblages as output. Classical methods (correspondence analysis and clustering analysis) were then used to identify the relations between diatom assemblages and the SOM cell number. A canonical correspondence analysis was also used to define the relationship between these assemblages and the environmental conditions. 3. The diatom-SOM training set resulted in 12 representative assemblages (12 clusters) having different species compositions. Comparison of observed and estimated sample positions on the SOM map were used to evaluate the performance of the BPN (correlation coefficients were 0.93 for X and 0.94 for Y). Mean square errors of 12 cells varied from 0.47 to 1.77 and the proportion of well predicted samples ranged from 37.5 to 92.9%. This study showed the high predictability of diatom assemblages using physical and chemical parameters for a small number of river types within a restricted geographical area.

关键词： backpropagation algorithm benthic diatoms Kohonen self-organising map stream ecology

来源：评论

学校读者我要写书评

暂无评论

Active cancellation system of acoustic noise in MR imaging

引用

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 1999年第2期46卷 186-191页

作者： Chen, GK Chiueh, TD Chen, JH Natl Taiwan Univ Dept Elect Engn Taipei 10617 Taiwan

In this paper, we introduce a new neural-network architecture fur reducing the acoustic noise level in magnetic resonance (MR) imaging processes. The proposed neural network (NN) consists of two cascaded time-delay NN's (TDNN's), This NN is used as the predictor of a feedback active noise control (ANC) system for reducing acoustic noises. Experimental results with real MR noises show that the proposed system achieved an average noise power attenuation of 18.75 dB, which compares favorably with previous studies. preliminary results also show that with the proposed ANC system installed, acoustic MR noises are greatly attenuated while verbal communication during MRI sessions is not affected.

关键词： active noise cancellation backpropagation algorithm magnetic resonance (MR) multilayer perceptrons time-delay neural networks (TDNN's)

来源：评论

学校读者我要写书评

暂无评论

A complex-valued RTRL algorithm for recurrent neural networks

引用

NEURAL COMPUTATION 2004年第12期16卷 2699-2713页

作者： Goh, SL Mandic, DP Univ London Imperial Coll Sci Technol & Med Dept Elect & Elect Engn London SW7 2AZ England

A complex-valued real-time recurrent learning (CRTRL) algorithm for the class of nonlinear adaptive filters realized as fully connected recurrent neural networks is introduced. The proposed CRTRL is derived for a general complex activation function of a neuron, which makes it suitable for nonlinear adaptive filtering of complex-valued nonlinear and nonstationary signals and complex signals with strong component correlations. In addition, this algorithm is generic and represents a natural extension of the real-valued RTRL. Simulations on benchmark and real-world complex-valued signals support the approach.

关键词： backpropagation algorithm

来源：评论

学校读者我要写书评

暂无评论

A new approach for the vanishing gradient problem on sigmoid activation

引用

PROGRESS IN ARTIFICIAL INTELLIGENCE 2020年第4期9卷 351-360页

作者： Roodschild, Matias Gotay Sardinas, Jorge Will, Adrian Univ Tecnol Nacl Grp Invest Tecnol Avanzadas GITIA Fac Reg Tucuman San Miguel De Tucuman Argentina

The vanishing gradient problem (VGP) is an important issue at training time on multilayer neural networks using the backpropagation algorithm. This problem is worse when sigmoid transfer functions are used, in a network with many hidden layers. However, the sigmoid function is very important in several architectures such as recurrent neural networks and autoencoders, where the VGP might also appear. In this article, we propose a modification of the backpropagation algorithm for the sigmoid neurons training. It consists of adding a small constant to the calculation of the sigmoid's derivative so that the proposed training direction differs slightly from the gradient while keeping the original sigmoid function in the network. This approach suggests that the derivative's modification produces the same accuracy in fewer training steps on most datasets. Moreover, due to VGP, the original derivative does not converge using sigmoid functions on more than five hidden layers. However, the modification allows backpropagation to train two extra hidden layers in feedforward neural networks.

关键词： Vanishing gradient problem Sigmoid function Feedforward neural networks backpropagation algorithm

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：