检索结果-内蒙古大学图书馆

Convergence in quadratic mean of averaged stochastic gradient algorithms without strong convexity nor bounded gradient

引用

Series Statistics 2023年第3期57卷

作者： Antoine Godichon-Baggioni Laboratoire de Probabilités Statistique et Modélisation Sorbonne-Université Paris France

Online averaged stochastic gradient algorithms are more and more studied since (i) they can deal quickly with large sample taking values in high-dimensional spaces, (ii) they enable to treat data sequentially, (iii) they are known to be asymptotically efficient. In this paper, we focus on giving explicit bounds of the quadratic mean error of the estimates, and this, without supposing that the function we would like to minimize is strongly convex or admits a bounded gradient.

关键词： stochastic optimization stochastic gradient algorithm averaging online learning non-asymptotic convergence 62L12 62L20

来源：评论

学校读者我要写书评

暂无评论

Adaptive Filtering under Minimum Information Divergence Criterion

引用

INTERNATIONAL JOURNAL OF CONTROL AUTOMATION AND SYSTEMS 2009年第2期7卷 157-164页

作者： Chen, Badong Zhu, Yu Hu, Jinchun Sun, Zengqi Tsinghua Univ Inst Mfg Engn Dept Precis Instruments & Mechanol Beijing 100084 Peoples R China Tsinghua Univ Dept Comp Sci & Technol State Key Lab Intelligent Technol & Syst Beijing 100084 Peoples R China

Traditional filtering theory is always based on optimization of the expected value of a suitably chosen function of error, Such as the minimum mean-square error (MMSE) criterion, the minimum error entropy (MEE) criterion, and so oil. None of those criteria could capture all the probabilistic information about the error distribution. In this work, we propose a novel approach to shape the probability density function (PDF) of the errors in adaptive filtering. As the PDF contains all the probabilistic information, the proposed approach can be used to obtain the desired variance or entropy, and is expected to be useful in the complex signal processing and learning systems. In our method, the information divergence between the actual errors and the desired errors is chosen as the cost function, which is estimated by kernel approach. Some important properties of the estimated divergence are presented. Also, for the finite impulse response (FIR) Filter, a stochastic gradient algorithm is derived. Finally, simulation examples illustrate the effectiveness of this algorithm in adaptive system training.

关键词： Adaptive filtering information divergence kernel method stochastic gradient algorithm

来源：评论

学校读者我要写书评

暂无评论

Minimum deviation distribution machine for large scale regression

引用

KNOWLEDGE-BASED SYSTEMS 2018年 146卷 167-180页

作者： Liu, Ming-Zeng Shao, Yuan-Hai Wang, Zhen Li, Chun-Na Chen, Wei-Jie Dalian Univ Technol Sch Math & Phys Sci Panjin 124221 Peoples R China Hainan Univ Sch Econ & Management Haikou 570228 Hainan Peoples R China Inner Mongolia Univ Sch Math Sci Hohhot 010021 Peoples R China Zhejiang Univ Technol Zhijiang Coll Hangzhou 310024 Zhejiang Peoples R China

In this paper, by introducing the statistics of training data into support vector regression (SVR), we propose a minimum deviation distribution regression (MDR). Rather than just minimizing the structural risk, MDR also minimizes both the regression deviation mean and the regression deviation variance, which is able to deal with the different distribution of boundary data and noises. The formulation of minimizing the first and second order statistics in MDR leads to a strongly convex quadratic programming problem (QPP). An efficient dual coordinate descend algorithm is adopted for small sample problem, and an average stochastic gradient algorithm for large scale one. Both theoretical analysis and experimental results illustrate the efficiency and effectiveness of the proposed method. (C) 2018 Elsevier B.V. All rights reserved.

关键词： Regression Support vector machine Minimum deviation distribution machine Dual coordinate descend algorithm stochastic gradient algorithm

来源：评论

学校读者我要写书评

暂无评论

Recursive coupled projection algorithms for multivariable output-error-like systems with coloured noises

引用

IET SIGNAL PROCESSING 2020年第7期14卷 455-466页

作者： Pan, Jian Ma, Hao Zhang, Xiao Liu, Qinyao Ding, Feng Chang, Yufang Sheng, Jie Hubei Univ Technol Sch Elect & Elect Engn Hubei Key Lab High Efficiency Utilizat Solar Ener Wuhan 430068 Peoples R China Jiangnan Univ Sch Internet Things Engn Wuxi 214122 Jiangsu Peoples R China Univ Washington Sch Engn & Technol Tacoma WA 98402 USA

By combining the coupling identification concept with the gradient search, this study develops a partially coupled generalised extended projection algorithm and a partially coupled generalised extended stochastic gradient algorithm to estimate the parameters of a multivariable output-error-like system with autoregressive moving average noise from input-output data. The key is to divide the identification model into several submodels based on the hierarchical identification principle and to establish the parameter estimation algorithm by using the coupled relationship between these submodels. The simulation test results indicate that the proposed algorithms are effective.

关键词： identification parameter estimation least squares approximations gradient methods autoregressive moving average processes state-space methods stochastic processes multivariable control systems recursive coupled projection algorithms multivariable output-error-like system coloured noises coupling identification concept gradient search projection algorithm stochastic gradient algorithm autoregressive moving average noise input-output data identification model hierarchical identification principle parameter estimation algorithm coupled relationship

来源：评论

学校读者我要写书评

暂无评论

Adaptive error-constrained method for LMS algorithms and applications

引用

SIGNAL PROCESSING 2005年第10期85卷 1875-1897页

作者： Choi, S Lee, TW Hong, D Yonsei Univ Ctr IT Seoul 120749 South Korea Univ Calif San Diego INC La Jolla CA 92093 USA

An adaptive error-constrained least mean square (AECLMS) algorithm is derived and proposed using adaptive error-constrained optimization techniques. This is accomplished by modifying the cost function of the LMS algorithm using augmented Lagrangian multipliers. Theoretical analyses of the proposed method are presented in detail. The method shows improved performance in terms of convergence speed and misadjustment. This proposed adaptive errorconstrained method can easily be applied to and combined with other LMS-type stochastic algorithms. Therefore, we also apply the method to constant modulus criterion for blind method and backpropagation algorithm for multilayer perceptrons. Simulation results show that the proposed method can accelerate the convergence speed by 2 to 20 times depending on the complexity of the problem. (c) 2005 Elsevier B.V. All rights reserved.

关键词： least mean square Lagrangian multiplier stochastic gradient algorithm constant modulus criterion multilayer perceptrons backpropagation

来源：评论

学校读者我要写书评

暂无评论

Low-Rate-Feedback-Assisted Beamforming and Power Control for MIMO-OFDM Systems

引用

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY 2010年第1期59卷 225-234页

作者： Merli, Filippo Zuccardi Wang, Xiaodong Vitetta, Giorgio Matteo Univ Modena & Reggio Emilia Dept Informat Engn I-41100 Modena Italy Columbia Univ Dept Elect Engn New York NY 10027 USA

This paper proposes a novel solution to the problem of beamforming and power control in the downlink of a multiple-input multiple-output ( MIMO) orthogonal frequency-division multiplexing (OFDM) system. This solution is developed in two steps. First, we describe an adaptive beamforming technique that, using a stochastic gradient method, maximizes the power delivered to a mobile terminal. In the proposed solution, perturbed precoding matrices are time multiplexed in the information signal transmitted to a mobile terminal;then, the mobile terminal informs the transmitter, via a single feedback bit, about the perturbation delivering the larger power. This approach does not need pilot symbols and uses quasi-Monte Carlo methods to generate the required perturbations with the relevant advantages of improving the downlink spectral efficiency and reducing the system complexity with respect to other competing solutions. Then, we propose a novel power-control algorithm that, selecting a proper transmission energy level from a set of possible values, aims to minimize the average bit error rate. This set of levels is generated on the basis of the channel statistics and a long-term constraint on the average transmission power. Numerical results evidence the robustness of the proposed algorithms in a dynamic fading environment.

关键词： Adaptive transmissions beamforming low-rate feedback multiple-input-multiple-output (MIMO) orthogonal frequency-division multiplexing (OFDM) power control quasi-Monte Carlo (QMC) stochastic gradient algorithm

来源：评论

学校读者我要写书评

暂无评论

Identifying Cognitive Radars-Inverse Reinforcement Learning Using Revealed Preferences

引用

IEEE TRANSACTIONS ON SIGNAL PROCESSING 2020年 68卷 4529-4542页

作者： Krishnamurthy, Vikram Angley, Daniel Evans, Robin Moran, Bill Cornell Univ Sch Elect & Comp Engn Ithaca NY 14853 USA Univ Melbourne Dept Elect & Elect Engn Parkville Vic 3010 Australia

We consider an inverse reinforcement learning problem involving "us" versus an "enemy" radar equipped with a Bayesian tracker. By observing the emissions of the enemy radar, how can we identify if the radar is cognitive (constrained utility maximizer)? Given the observed sequence of actions taken by the enemy's radar, we consider three problems: (i) Are the enemy radar's actions (waveform choice, beam scheduling) consistent with constrained utility maximization? If so how can we estimate the cognitive radar's utility function that is consistent with its actions. We formulate, and solve the problem in terms of the spectra (eigenvalues) of the state, and observation noise covariance matrices, and the algebraic Riccati equation. (ii) How to construct a statistical test for detecting a cognitive radar (constrained utility maximization) when we observe the radar's actions in noise or the radar observes our probe signal in noise? We propose a statistical detector with a tight Type-II error bound. (iii) How can we optimally probe (interrogate) the enemy's radar by choosing our state to minimize the Type-II error of detecting if the radar is deploying an economic rational strategy, subject to a constraint on the Type-I detection error? We present a stochastic optimization algorithm to optimize our probe signal. The main analysis framework used in this paper is that of revealed preferences from microeconomics.

关键词： Probes Radar detection Cognitive radar Radar tracking Learning (artificial intelligence) Bayes methods Revealed preferences inverse reinforcement learning adversarial signal processing identifying cognitive behavior spectral revealed preferences Afriat's theorem stochastic gradient algorithm detection economics-based-rationality Kalman filter tracker algebraic Riccati equation waveform selection beam scheduling

来源：评论

学校读者我要写书评

暂无评论

A modified constrained constant modulus approach to blind adaptive multiuser detection

引用

IEEE TRANSACTIONS ON COMMUNICATIONS 2001年第9期49卷 1642-1648页

作者： Xu, CJ Feng, GZ Kwak, KS Nanjing Univ Posts & Telecommun Dept Telecommun Engn Nanjing 21003 Peoples R China Inha Univ Sch Informat & Commun Engn Inchon 402751 South Korea

An alternative blind adaptive multiuser detection is investigated based on modified constrained constant modulus (CM) criterion. It has been shown that the performance of a CM-based receiver is limited by the received power of the desired user. In this paper, we show that the limitation can be avoided using noncanonical constraint CIM criterion and that in the presence of channel noise the modified CM criterion function is strictly convex by properly selecting some constant. With analyzing the extrema of the cost function, we point out how to select the constant. Moreover, a simple stochastic gradient algorithm for implementing our scheme is presented, and the convergence properties of the algorithm are analyzed. Simulation examples are given to demonstrate the performance of the proposed scheme.

关键词： blind multiuser detection CDMA constant modulus approach multiple-access interference stochastic gradient algorithm

来源：评论

学校读者我要写书评

暂无评论

Research on a learning rate with energy index in deep learning

引用

NEURAL NETWORKS 2019年 110卷 225-231页

作者： Zhao, Huizhen Liu, Fuxian Zhang, Han Liang, Zhibing Air Force Engn Univ Changle East Rd1 Jia Zi Xian Shaanxi Peoples R China

The stochastic gradient descent algorithm (SGD) is the main optimization solution in deep learning. The performance of SGD depends critically on how learning rates are tuned over time. In this paper, we propose a novel energy index based optimization method (EIOM) to automatically adjust the learning rate in the backpropagation. Since a frequently occurring feature is more important than a rarely occurring feature, we update the features to different extents according to their frequencies. We first define an energy neuron model and then design an energy index to describe the frequency of a feature. The learning rate is taken as a hyperparameter function according to the energy index. To empirically evaluate the EIOM, we investigate different optimizers with three popular machine learning models: logistic regression, multilayer perceptron, and convolutional neural network. The experiments demonstrate the promising performance of the proposed EIOM compared with that of other optimization algorithms. (C) 2018 Elsevier Ltd. All rights reserved.

关键词： Deep learning Convolutional neural network stochastic gradient algorithm Learning rate Energy index

来源：评论

学校读者我要写书评

暂无评论

Convergence of the recursive identification algorithms for multivariate pseudo-linear regressive systems

引用

INTERNATIONAL JOURNAL OF ADAPTIVE CONTROL AND SIGNAL PROCESSING 2016年第6期30卷 824-842页

作者： Wang, Xuehai Ding, Feng Jiangnan Univ Key Lab Adv Proc Control Light Ind Minist Educ Wuxi 214122 Peoples R China Henan Univ Urban Construct Sch Math & Phys Pingdingshan 467036 Peoples R China

The performance analysis of the recursive algorithms for the multivariate systems with an autoregressive moving average noise process is still open. This paper analyzes the convergence of two recursive identification algorithms, the multivariate recursive generalized extended least squares algorithm and the multivariate generalized extended stochastic gradient algorithm, for pseudo-linear multivariate systems and proves that the parameter estimation errors consistently converge to zero under persistent excitation conditions. The simulation results show that the proposed algorithms work well. Copyright (C) 2015 JohnWiley & Sons, Ltd.

关键词： parameter estimation least squares algorithm stochastic gradient algorithm pseudo-linear regressive model martingale convergence theorem

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：