检索结果-内蒙古大学图书馆

第34届中国控制与决策会议

作者： Jie Chu Junhong Li Tiancheng Zong School of Electrical Engineering Nantong University

The Volterra model can approximate many nonlinear systems,and it is a typical nonlinear *** paper studies the parameter estimation problem of the Volterra *** the Levenberg-Marquardt optimization method and the recursive identification method,we propose a Levenberg-Marquardt recursive algorithm and apply it to the identification of the Volterra *** order to verify the feasibility of the above algorithm,the second-order Volterra system is simulated using the Levenberg-Marquardt recursive algorithm and the forgetting factor stochastic gradient algorithm respectively,and then we compare the simulation results of the Volterra system under the two *** simulation results show that the above two algorithms can identify the parameters of the Volterra *** with the forgetting factor stochastic gradient algorithm,the Levenberg-Marquardt recursive algorithm has faster convergence speed and higher convergence *** proves the effectiveness of the Levenberg-Marquardt recursive algorithm.

关键词： parameter estimation nonlinear systems Volterra model Levenberg-Marquardt recursive algorithm stochastic gradient algorithm

来源：评论

学校读者我要写书评

暂无评论

The Value of Collaboration in Convex Machine Learning with Differential Privacy 41

The Value of Collaboration in Convex Machine Learning with D...

引用

41st IEEE Symposium on Security and Privacy (SP)

作者： Wu, Nan Farokhi, Farhad Smith, David Kaafar, Mohamed Ali Macquarie Univ N Ryde NSW Australia CSIRO Data61 Canberra ACT Australia Univ Melbourne Melbourne Vic 3010 Australia Australian Natl Univ Canberra ACT Australia

ISBN: (数字)9781728134970

ISBN: (纸本)9781728134970

In this paper, we apply machine learning to distributed private data owned by multiple data owners, entities with access to non-overlapping training datasets. We use noisy, differentially-private gradients to minimize the fitness cost of the machine learning model using stochastic gradient descent. We quantify the quality of the trained model, using the fitness cost, as a function of privacy budget and size of the distributed datasets to capture the trade-off between privacy and utility in machine learning. This way, we can predict the outcome of collaboration among privacy-aware data owners prior to executing potentially computationally-expensive machine learning algorithms. Particularly, we show that the difference between the fitness of the trained machine learning model using differentially-private gradient queries and the fitness of the trained machine model in the absence of any privacy concerns is inversely proportional to the size of the training datasets squared and the privacy budget squared. We successfully validate the performance prediction with the actual performance of the proposed privacy-aware learning algorithms, applied to: financial datasets for determining interest rates of loans using regression;and detecting credit card frauds using support vector machines.

关键词： Machine learning Differential privacy stochastic gradient algorithm

来源：评论

学校读者我要写书评

暂无评论

On Large Batch Training and Sharp Minima: A Fokker-Planck Perspective

引用

JOURNAL OF STATISTICAL THEORY AND PRACTICE 2020年第3期14卷 1-31页

作者： Dai, Xiaowu Zhu, Yuhua Univ Calif Berkeley CDAR Berkeley CA 94720 USA Univ Calif Berkeley Simons Inst Theory Comp Berkeley CA 94720 USA Stanford Univ Dept Math Stanford CA 94305 USA

We study the statistical properties of the dynamic trajectory of stochastic gradient descent (SGD). We approximate the mini-batch SGD and the momentum SGD as stochastic differential equations. We exploit the continuous formulation of SDE and the theory of Fokker-Planck equations to develop new results on the escaping phenomenon and the relationship with large batch and sharp minima. In particular, we find that the stochastic process solution tends to converge to flatter minima regardless of the batch size in the asymptotic regime. However, the convergence rate is rigorously proven to depend on the batch size. These results are validated empirically with various datasets and models.

关键词： Large batch training Sharp minima Fokker-Planck equation stochastic gradient algorithm Deep neural network

来源：评论

学校读者我要写书评

暂无评论

A unified convergence analysis for shuffling-type gradient methods

The Journal of Machine Learning Research

引用

The Journal of Machine Learning Research 2021年第1期22卷 9397-9440页

作者： Lam M. Nguyen Quoc Tran-Dinh Dzung T. Phan Phuong Ha Nguyen Marten Van Dijk IBM Research Thomas J. Watson Research Center Yorktown Heights NY Department of Statistics and Operations Research The University of North Carolina at Chapel Hill Chapel Hill NC eBay Inc. San Jose CA Centrum Wiskunde & Informatica Amsterdam Netherlands

In this paper, we propose a unified convergence analysis for a class of generic shuffling-type gradient methods for solving finite-sum optimization problems. Our analysis works with any sampling without replacement strategy and covers many known variants such as randomized reshuffling, deterministic or randomized single permutation, and cyclic and incremental gradient schemes. We focus on two different settings: strongly convex and nonconvex problems, but also discuss the non-strongly convex case. Our main contribution consists of new non-asymptotic and asymptotic convergence rates for a wide class of shuffling-type gradient methods in both nonconvex and convex settings. We also study uniformly randomized shuffling variants with different learning rates and model assumptions. While our rate in the nonconvex case is new and significantly improved over existing works under standard assumptions, the rate on the strongly convex one matches the existing best-known rates prior to this paper up to a constant factor without imposing a bounded gradient condition. Finally, we empirically illustrate our theoretical results via two numerical examples: nonconvex logistic regression and neural network training examples. As byproducts, our results suggest some appropriate choices for diminishing learning rates in certain shuffling variants.

关键词： stochastic gradient algorithm shuffling-type gradient scheme sampling without replacement non-convex finite-sum minimization strongly convex minimization

来源：评论

学校读者我要写书评

暂无评论

PROJECTED stochastic gradientS FOR CONVEX CONSTRAINED PROBLEMS IN HILBERT SPACES

引用

SIAM JOURNAL ON OPTIMIZATION 2019年第3期29卷 2079-2099页

作者： Geiersbach, Caroline Pflug, Georg Ch Univ Vienna Dept Stat & Operat Res A-1030 Vienna Austria IIASA Schlosspl 1 A-2361 Laxenburg Austria

Convergence of a projected stochastic gradient algorithm is demonstrated for convex objective functionals with convex constraint sets in Hilbert spaces. In the convex case, the sequence of iterates u(n) converges weakly to a point in the set of minimizers with probability one. In the strongly convex case, the sequence converges strongly to the unique optimum with probability one. An application to a class of PDE constrained problems with a convex objective, convex constraint, and random elliptic PDE constraints is shown. Theoretical results are demonstrated numerically.

关键词： stochastic approximation stochastic gradient algorithm random elliptic PDEs as constraints PDE constrained optimization under uncertainty optimization in Hilbert spaces

来源：评论

学校读者我要写书评

暂无评论

Research on a learning rate with energy index in deep learning

引用

NEURAL NETWORKS 2019年 110卷 225-231页

作者： Zhao, Huizhen Liu, Fuxian Zhang, Han Liang, Zhibing Air Force Engn Univ Changle East Rd1 Jia Zi Xian Shaanxi Peoples R China

The stochastic gradient descent algorithm (SGD) is the main optimization solution in deep learning. The performance of SGD depends critically on how learning rates are tuned over time. In this paper, we propose a novel energy index based optimization method (EIOM) to automatically adjust the learning rate in the backpropagation. Since a frequently occurring feature is more important than a rarely occurring feature, we update the features to different extents according to their frequencies. We first define an energy neuron model and then design an energy index to describe the frequency of a feature. The learning rate is taken as a hyperparameter function according to the energy index. To empirically evaluate the EIOM, we investigate different optimizers with three popular machine learning models: logistic regression, multilayer perceptron, and convolutional neural network. The experiments demonstrate the promising performance of the proposed EIOM compared with that of other optimization algorithms. (C) 2018 Elsevier Ltd. All rights reserved.

关键词： Deep learning Convolutional neural network stochastic gradient algorithm Learning rate Energy index

来源：评论

学校读者我要写书评

暂无评论

Adaptive Filtering Under the Maximum Correntropy Criterion With Variable Center

引用

IEEE ACCESS 2019年 7卷 105902-105908页

作者： Zhu, Lingfei Song, Chengtian Pan, Lizhi Li, Jili Beijing Inst Technol Sch Mechatron Engn Beijing 100081 Peoples R China Huaihai Ind Grp Changzhi 046012 Peoples R China

Recently, an extended version of correntropy, whose center can locate at any position has been proposed and applied in a new optimization criterion called maximum correntropy criterion with variable center (MCC-VC). In order to optimize the performance of adaptive filtering in non-Gaussian and non-zero mean noise environments, in this paper, we propose a stochastic gradient adaptive filtering algorithm for online learning based on MCC-VC and analyze its stability and convergence performance. Moreover, we also extend an online learning approach to estimate the kernel width and the center location, in which two parameters have a great influence on the accuracy of the algorithm. The simulation results of the online learning model have verified the superiority and robustness of the new method.

关键词： Adaptive filtering maximum correntropy criterion with variable center(MCC-VC) stochastic gradient algorithm steady-state excess mean square error

来源：评论

学校读者我要写书评

暂无评论

Parameter estimation algorithm for d-step time delay systems

引用

INTERNATIONAL JOURNAL OF MODELLING IDENTIFICATION AND CONTROL 2019年第3-4期32卷 199-203页

作者： Gu, Ya Zhu, Peiyi Li, Xiangli Gu, Jianfei Changshu Inst Technol Sch Elect Engn & Automat Changshu Jiangsu Peoples R China

This article proposes the methods of parameter estimation and state estimation to calculate state space systems with delay. Associating the properties of linear conversion and shift operators, the state space model can be equal to the standard state space model, which can then be converted into the recognition model. The proposed stochastic gradient method is brought to calculate the system and converge to the recursive least squares estimates for state space models. Finally, one example is proposed to certify the theorems of this paper.

关键词： state estimation stochastic gradient algorithm time-delay

来源：评论

学校读者我要写书评

暂无评论

ICI Suppression of Underwater Acoustic OFDM Signal Transmission by Differential Multichannel Detection 2

ICI Suppression of Underwater Acoustic OFDM Signal Transmiss...

引用

IEEE 2nd International Conference on Electronic Information and Communication Technology (ICEICT)

作者： Ma, Xuefei Wang, Bo Li, Yang Hu, Pengpeng Wang, Tingling Wang, Guan Harbin Engn Univ Coll Underwater Acoust Engn Harbin Peoples R China Beijing Great Wall Elect & Equipment Co Ltd Beijing Peoples R China Syst Engn Res Inst Syst Engn Innovat Ctr Beijing Peoples R China

ISBN: (纸本)9781538692981

In this paper, we propose an underwater communication method that can realize real-time communication, which can be used for communication between high-speed UUV and underwater intelligent nodes. This method of Doppler compensation of orthogonal frequency division multiplexed signals for underwater communication does not utilize the conventional channel estimation method, but uses the concept of decomposing channel variations based on a set of known functions. OFDM symbol reconstruction is performed after the T (Taylor)-FFT decomposition channel. A stochastic gradient adaptive algorithm is designed to learn the combination of detection weights. Thus, the two steps of channel matching filtering and channel balancing are implemented by one step. The validity and correctness of this method are verified by the recent experimental data (several kilometers in shallow water, 12-15 kHz band) of mobile underwater acoustic communication provide technical support for high-speed mobile object communication.

关键词： OFDM Doppler stochastic gradient algorithm Underwater communication

来源：评论

学校读者我要写书评

暂无评论

Minimum deviation distribution machine for large scale regression

引用

KNOWLEDGE-BASED SYSTEMS 2018年 146卷 167-180页

作者： Liu, Ming-Zeng Shao, Yuan-Hai Wang, Zhen Li, Chun-Na Chen, Wei-Jie Dalian Univ Technol Sch Math & Phys Sci Panjin 124221 Peoples R China Hainan Univ Sch Econ & Management Haikou 570228 Hainan Peoples R China Inner Mongolia Univ Sch Math Sci Hohhot 010021 Peoples R China Zhejiang Univ Technol Zhijiang Coll Hangzhou 310024 Zhejiang Peoples R China

In this paper, by introducing the statistics of training data into support vector regression (SVR), we propose a minimum deviation distribution regression (MDR). Rather than just minimizing the structural risk, MDR also minimizes both the regression deviation mean and the regression deviation variance, which is able to deal with the different distribution of boundary data and noises. The formulation of minimizing the first and second order statistics in MDR leads to a strongly convex quadratic programming problem (QPP). An efficient dual coordinate descend algorithm is adopted for small sample problem, and an average stochastic gradient algorithm for large scale one. Both theoretical analysis and experimental results illustrate the efficiency and effectiveness of the proposed method. (C) 2018 Elsevier B.V. All rights reserved.

关键词： Regression Support vector machine Minimum deviation distribution machine Dual coordinate descend algorithm stochastic gradient algorithm

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：