We investigated two commonly used momentum algorithms, Classical momentum(CM) and Nesterov momentum(NM). We found that, when used in Restricted Boltzmann machine(RBM), they have two main problems: The first one is the...
详细信息
We investigated two commonly used momentum algorithms, Classical momentum(CM) and Nesterov momentum(NM). We found that, when used in Restricted Boltzmann machine(RBM), they have two main problems: The first one is their performances are not obvious and not as good as expected. The second one is they may lose accelerating ability in the later stage of training process. Aiming at these two problems, we proposed the Weight momentum algorithm and evaluated our approach on four datasets. It has been demonstrated that our methods can achieve better performance under both reconstruction error and classification rate criterions.
Nowadays, the digital world shows the development trend of high-dimensional mass, and most data can be structured or sparsely represented, so it is convenient to use Compressive Sensing (CS) to reduce the storage and ...
详细信息
ISBN:
(纸本)9798350387605;9798350387599
Nowadays, the digital world shows the development trend of high-dimensional mass, and most data can be structured or sparsely represented, so it is convenient to use Compressive Sensing (CS) to reduce the storage and transmission costs of big data. The key data reconstruction process in CS is usually formalized as a minimization problem, but it is difficult for existing algorithms to achieve high data reconstruction speed and accuracy at the same time. Heavy-ball momentum algorithm is the most representative acceleration algorithm in the field of optimization. In this paper, it is introduced into the reconstruction process of CS, and the convergence of the momentum algorithm to decompress the CS reconstruction optimization problem is guaranteed through theoretical analysis. The results of comparative experiments also show that the momentum algorithm can reconstruct the image with higher quality and consume the least computing time.
Aiming at the problem that the linear regression of the traditional linear water quality prediction model is not robust enough and the prediction accuracy cannot be guaranteed in the presence of interference, this pap...
详细信息
Aiming at the problem that the linear regression of the traditional linear water quality prediction model is not robust enough and the prediction accuracy cannot be guaranteed in the presence of interference, this paper proposes an agricultural water quality prediction model based on the momentum algorithm to optimize the logistic regression algorithm (LRM algorithm). The model uses the momentum algorithm to optimize the logistic regression algorithm to quickly adjust the misclassified samples. When the object encounters a local optimum in the process of falling, the introduction of momentum makes it easy for the next update to jump out of the local optimum with the help of the last large gradient. In this paper, the performance of the proposed model is evaluated on 4 real data sets. The experimental results show that the LRM algorithm proposed in this paper improves the prediction accuracy of the existing algorithm by an average of 1.11 percentage *** with KNN and other traditional prediction algorithms, LRM not only speeds up the convergence rate of the algorithm, but also reduces the steady-state error and improves the prediction accuracy of water quality, suitable for data mining of complex water quality data, The experiment verifies the feasibility of this method in predicting the actual agricultural water quality and even in predicting and warning the residents drinking water.
Wind power forecasting (WPF) is crucial for grid dispatch and effective collection of wind power. In recent years, deep learning such as multi-layer perception (MLP) has been widely adopted in WPF. This paper proposes...
详细信息
ISBN:
(纸本)9781665489577
Wind power forecasting (WPF) is crucial for grid dispatch and effective collection of wind power. In recent years, deep learning such as multi-layer perception (MLP) has been widely adopted in WPF. This paper proposes a novel ultra-short-term and short-term WPF approach based on optimized artificial neural network (ANN) with time series reconstruction (TSR) method. Z-scored method is adopted to preprocess the dataset measured from a wind farm. TSR is proposed and used to generate model inputs. An optimized ANN-based model with MLP architecture is introduced, and momentum algorithm and decaying learning rate are proposed for model optimization. Repeated trainings are performed to obtain the optimal model. The proposed approach is validated on the test case of a wind farm in China. The results obtained prove that the optimized ANN-based model and TSR can both effectively improve the accuracy of ultra-short-term and short-term WPF.
In ICLR's (2018) best paper "On the Convergence of Adam and Beyond", the author points out the shortcomings in Adam's convergence proof, proposes an AMSGRAD algorithm that can guarantee convergence a...
详细信息
ISBN:
(纸本)9781728137988
In ICLR's (2018) best paper "On the Convergence of Adam and Beyond", the author points out the shortcomings in Adam's convergence proof, proposes an AMSGRAD algorithm that can guarantee convergence as the number of iterations increases. However, through some comparative experiments, this paper finds that there are two problems in the convergence process of AMSGRAD algorithm. Firstly, the AMSGRAD algorithm is easy to oscillate;Secondly, the AMSGRAD algorithm converges slowly. After analysis, the above two problems can be solved by the following ways. When g(t-1) g(t) > 0, this paper adds the momentum term in momentum algorithm to the AMSGRAD algorithm to accelerate convergence. When g(t-1) g(t) <= 0, this paper use SGD algorithm instead of AMSGRAD algorithm to update the model weights. In order to eliminate some negative effects of the previous parameter gradient on the current parameter gradient and reduce the oscillation amplitude of the objective function, the first-order and second-order moment estimations of the parameter gradient are recalculated when g(t-1) g(t) <= 0. Therefore, this paper proposes the ACADG algorithm, which not only can improve the convergence speed, suppress the oscillation amplitude of the objective function, but also can improve the accuracy of training and test data sets.
Most of existing blind source separation (BSS) algorithms are suitable for real signals but difficult for complex signals, while transmission signals usually belong to complex domain in actual communication systems. T...
详细信息
ISBN:
(纸本)9781509006656
Most of existing blind source separation (BSS) algorithms are suitable for real signals but difficult for complex signals, while transmission signals usually belong to complex domain in actual communication systems. This paper proposes a new online algorithm for blind complex source separation (BCSS). The proposed algorithm process in-phase and quadrature component of communication signal jointly to realize BCSS based on momentum EASI algorithm. Simulation results show that the proposed algorithm separates complex sources effectively. When applied to separate complex communication signal and jamming signals, the new algorithm can increases the anti-jamming margin of communication system greatly, and both of the symbol error ratio (SER) and the bit error ratio (BER) of communication signal is close to the theoretical bound.
In this paper, a new gradient optimized blind source separation algorithm (GOA) which aims at improving the convergence performance is proposed. This algorithm modifies the gradient of cost function to make the iterat...
详细信息
ISBN:
(纸本)9781467376877
In this paper, a new gradient optimized blind source separation algorithm (GOA) which aims at improving the convergence performance is proposed. This algorithm modifies the gradient of cost function to make the iteration process of separation matrix closer to its change pattern. To be more specific, by adding the difference value between current time gradient and previous time gradient to the adaptive iterations of separation matrix, the proposed algorithm effectively improves convergence rate. Simulation experiment results show that the GOA has faster convergence rate when compared with the traditional momentum EASI algorithm. In the small step size conditions, the advantage of GOA is even more obvious.
Spectrum prediction in the cognitive radio system attracts more and more attention. It can predict future spectrum holes to save energy of spectrum sensing and to improve the efficiency of spectrum access. The current...
详细信息
ISBN:
(纸本)9781467376792
Spectrum prediction in the cognitive radio system attracts more and more attention. It can predict future spectrum holes to save energy of spectrum sensing and to improve the efficiency of spectrum access. The current research on spectrum prediction is to use the prediction model such as back propagation (BP) neural network to predict. However, the performance of conventional spectrum prediction is not satisfied to meet the real system for its using inaccurate spectrum states and defects of the BP neural network. Therefore, we propose a spectrum prediction based on improved-BP neural networks. In the proposed model, the channel power values information instead of the channel states are used as the inputs of the spectrum prediction, the BP neural network optimized by the genetic algorithm and momentum algorithm is utilized in the prediction process, and the threshold interval is applied to determine predicted channel states. Our experimental results demonstrate that the predictive accuracy of the proposed spectrum prediction based on the improved-BP neural network is higher than spectrum prediction based on conventional BP neural network.
Spectrum prediction in the cognitive radio system attracts more and more attention. It can predict future spectrum holes to save energy of spectrum sensing and to improve the efficiency of spectrum access. The current...
详细信息
ISBN:
(纸本)9781467376808
Spectrum prediction in the cognitive radio system attracts more and more attention. It can predict future spectrum holes to save energy of spectrum sensing and to improve the efficiency of spectrum access. The current research on spectrum prediction is to use the prediction model such as back propagation (BP) neural network to predict. However, the performance of conventional spectrum prediction is not satisfied to meet the real system for its using inaccurate spectrum states and defects of the BP neural network. Therefore, we propose a spectrum prediction based on improved-BP neural networks. In the proposed model, the channel power values information instead of the channel states are used as the inputs of the spectrum prediction, the BP neural network optimized by the genetic algorithm and momentum algorithm is utilized in the prediction process, and the threshold interval is applied to determine predicted channel states. Our experimental results demonstrate that the predictive accuracy of the proposed spectrum prediction based on the improved-BP neural network is higher than spectrum prediction based on conventional BP neural network.
In this study, nonlinear neural network controller will be developed to control plasma radial motion in Damavand Tokamak. It is essential to have a good model in order to design a proper controller for plasma radial m...
详细信息
ISBN:
(纸本)9781457720727
In this study, nonlinear neural network controller will be developed to control plasma radial motion in Damavand Tokamak. It is essential to have a good model in order to design a proper controller for plasma radial motion. To achieve this goal, actuator circuits are simulated and in consequence based on simulator model and simulated actuator circuits nonlinear neural network controller will be designed in Damavand Tokamak. Comparison between neural network controller output and PD controller output shows the efficiency of proposed approach.
暂无评论