Deep neural network (DNN) is trained according to a mini-batch optimization based on the stochastic gradient descent algorithm. Such a stochastic learning suffers from instability in parameter updating and may easily ...
详细信息
ISBN:
(纸本)9781509007462
Deep neural network (DNN) is trained according to a mini-batch optimization based on the stochastic gradient descent algorithm. Such a stochastic learning suffers from instability in parameter updating and may easily trap into local optimum. This study deals with the stability of stochastic learning by reducing the variance of gradients in optimization procedure. We upgrade the optimization from the stochastic dual coordinated ascent (SDCA) to the accelerated SDCA without duality (or dual-free ASDCA). This optimization incorporates the momentum method to accelerate the updating rule where the variance of gradients can be reduced. Using dual-free ASDCA, the optimization of dual function of SDCA in a form of convex loss is implemented by directly optimizing the primal function with respect to pseudo-dual parameters. The non-convex optimization in DNN training can be resolved and accelerated. Experimental results illustrate the reduction of training loss, variance of gradients and word error rate by using the proposed optimization for DNN speech recognition.
This article presents a newly proposed selection process for genetic algorithms on a class of unconstrained optimization problems. The k-means genetic algorithm selection process (KGA) is composed of four essential st...
详细信息
This article presents a newly proposed selection process for genetic algorithms on a class of unconstrained optimization problems. The k-means genetic algorithm selection process (KGA) is composed of four essential stages: clustering, membership phase, fitness scaling and selection. Inspired from the hypothesis that clustering the population helps to preserve a selection pressure throughout the evolution of the population, a membership probability index is assigned to each individual following the clustering phase. Fitness scaling converts the membership scores in a range suitable for the selection function which selects the parents of the next generation. Two versions of the KGA process are presented: using a fixed number of clusters K (KGA(f)) and via an optimal partitioning K-opt (KGA(o)) determined by two different internal validity indices. The performance of each method is tested on seven benchmark problems.
The routing strategy plays a very important role in complex networks such as Internet system and Peer-to-Peer networks. However, most of the previous work concentrates only on the path selection, e.g. Flooding and Ran...
详细信息
The routing strategy plays a very important role in complex networks such as Internet system and Peer-to-Peer networks. However, most of the previous work concentrates only on the path selection, e.g. Flooding and Random Walk, or finding the shortest path (SP) and rarely considering the local load information such as SP and Distance Vector Routing. Flow-based Routing mainly considers load balance and still cannot achieve best optimization. Thus, in this paper, we propose a novel dynamic routing strategy on complex network by incorporating the local load information into SP algorithm to enhance the traffic flow routing optimization. It was found that the flow in a network is greatly affected by the waiting time of the network, so we should not consider only choosing optimized path for package transformation but also consider node congestion. As a result, the packages should be transmitted with a global optimized path with smaller congestion and relatively short distance. Analysis work and simulation experiments show that the proposed algorithm can largely enhance the network flow with the maximum throughput within an acceptable calculating time. The detailed analysis of the algorithm will also be provided for explaining the efficiency.
The adaptive niche quantum-inspired immune clonal algorithm (ANQICA) is proposed by combining the quantum coding, immune clone and niche mechanism together to solve the multi-modal function optimization more effective...
详细信息
The adaptive niche quantum-inspired immune clonal algorithm (ANQICA) is proposed by combining the quantum coding, immune clone and niche mechanism together to solve the multi-modal function optimization more effectively and make the function converge to as many as possible extreme value points. The quantum coding can better explore the solution space, the niche mechanism ensures the algorithm to converge to multi-extremum and the adaptive mechanism is introduced according to the characteristics of each procedure of the algorithm to improve the effect of the algorithm. Example analysis shows that the ANQICA is better in exploration and convergence. Therefore, the ANQICA can be used to solve the problem of multi-modal function optimization effectively.
Classification problems are significant because they constitute a meta-model for multiple theoretical and practical applications from a wide range of fields. The belief rule based (BRB) expert system has shown potenti...
详细信息
Classification problems are significant because they constitute a meta-model for multiple theoretical and practical applications from a wide range of fields. The belief rule based (BRB) expert system has shown potentials in dealing with both quantitative and qualitative information under uncertainty. In this study, a BRB classifier is proposed to solve the classification problem. However, two challenges must be addressed. First, the size of the BRB classifier must be controlled within a feasible range for better expert involvement. Second, the initial parameters of the BRB classifier must be optimized by learning from the experts' knowledge and/or historic data. Therefore, new rule activation and weight calculation procedures are proposed to downsize the BRB classifier while maintaining the matching degree calculation procedure. Moreover, the optimal algorithm using the evidential reasoning (ER) algorithm as the inference engine and the differential evolution (DE) algorithm as the optimization engine is proposed to identify the fittest parameters, including the referenced values of the antecedent attributes, the weights of the rules and the beliefs of the degrees in the conclusion. Five benchmarks, namely, iris, wine, glass, cancer and pima, are studied to validate the efficiency of the proposed BRB classifier. The result shows that all five benchmarks could be precisely modeled with a limited number of rules. The proposed BRB classifier has also shown superior performance in comparing it with the results in the literature. (C) 2015 Elsevier Inc. All rights reserved.
This paper treats the problem of quadratically constrained least squares with positive semidefinite weight matrices. A new method of solution is presented that searches directly over the constraint set, and does not r...
详细信息
This paper treats the problem of quadratically constrained least squares with positive semidefinite weight matrices. A new method of solution is presented that searches directly over the constraint set, and does not require the determination of Lagrange multipliers. Global convergence of the algorithm is rigorously proven. In addition, a covariance analysis is performed for the constrained optimal solution. Two aerospace applications are presented: 1) quadratically constrained Kalman filtering similar in form to the norm-constrained Kalman filter from the literature-it is shown that the optimal quadratically constrained update is simply an orthogonal projection of the optimal unconstrained update onto the constraint set, and 2) a new quadratically constrained Kalman filter using the covariance expression developed in this paper, yielding a statistically more consistent constrained filter. The new filter is demonstrated numerically with a spacecraft attitude estimation example.
The purpose of this paper is to present an evaluation study of a pyroelectric sensor model implemented in MATLAB (R)/SIMULINK (R) environment. The sensor model consists of some cascaded transfer functions taking into ...
详细信息
The purpose of this paper is to present an evaluation study of a pyroelectric sensor model implemented in MATLAB (R)/SIMULINK (R) environment. The sensor model consists of some cascaded transfer functions taking into account the material characteristics and geometrical parameters which have been synthesized in the form of thermal and electrical time constants and a global multiplying coefficient. The model was proposed by Odon to serve as an excellent basis for the analysis of the dynamic behavior of a pyroelectric sensor. The study performed is relevant to evaluate the validity of the model by comparing its simulated response to measurement obtained on a pyroelectric sensor prototype. To achieve this evaluation, an optimization algorithm is used to estimate the parameters values of the transfer function model using a succession of tests and adjustments so that the algorithm converges to and reaches the optimal solution giving an acceptable correlation between simulated and measured responses. The semi-experimental evaluation approach has been applied to two prototype sensors with pyroelectric material thicknesses of 9 and 25 mu m, respectively. In the two cases, the estimated values of the parameters were very interesting in terms of uncertainties (between 1.846 and 6.726% at 1 sigma level) and setting evidence for the existence of two global solutions, i.e., two deep minima for each prototype sensor.
The optimization of a palladium-catalyzed Heck Matsuda reaction using an optimization algorithm is presented. We modified and implemented the Nelder-Mead method in order to perform constrained optimizations in a multi...
详细信息
The optimization of a palladium-catalyzed Heck Matsuda reaction using an optimization algorithm is presented. We modified and implemented the Nelder-Mead method in order to perform constrained optimizations in a multidimensional space. We illustrated the power of our modified algorithm through the optimization of a multivariable reaction involving the arylation of a deactivated olefin with an arenediazonium salt. The great flexibility of our optimization method allows to fine-tune experimental conditions according to three different objective functions: maximum yield, highest throughput, and lowest production cost. The beneficial properties of flow reactors associated with the power of intelligent algorithms for the fine-tuning of experimental parameters allowed the reaction to proceed in astonishingly simple conditions unable to promote the coupling through traditional batch chemistry.
This paper investigates the combination of optimal feedback control with the dynamical structure of the three-body problem. The results provide new insights for the design of continuous low-thrust spacecraft trajector...
详细信息
This paper investigates the combination of optimal feedback control with the dynamical structure of the three-body problem. The results provide new insights for the design of continuous low-thrust spacecraft trajectories. Specifically, the attracting set of an equilibrium point or a periodic orbit (represented as a fixed point) under optimal control with quadratic cost is obtained. The analysis reveals the relation between the attractive set and original dynamics. In particular, it is found that the largest dimensions of the set are found along the stable manifold and the least extent is along the left eigenvector of the unstable manifold. The asymptotic behavior of the structure of the attractive set when time tends to infinity is analytically revealed. The results generalize the use of manifolds for transfers to equilibrium points and periodic orbits in astrodynamic problems. The result is theoretical and developed for a linearized system, but it can be extended to nonlinear systems in the future.
A novel method to handle the hidden coupling terms that naturally arise in the control of a nonlinear plant when an endogenous variable is used as a scheduling parameter was proposed. The proposed approach includes th...
详细信息
A novel method to handle the hidden coupling terms that naturally arise in the control of a nonlinear plant when an endogenous variable is used as a scheduling parameter was proposed. The proposed approach includes them in the design process with an a priori selection of the scheduling functions. This procedure is generic enough to be applied to a great variety of nonlinear systems in a systematic manner. The efficiency of the proposed approach has been demonstrated on the control of a pitch-axis missile benchmark problem, which has been solved by a nonsmooth optimization algorithm to yield a robust and self-scheduled controller. Numerical simulations show that the resulting controller can achieve a significant performance improvement while satisfying the robustness requirements when faced with a variety of uncertainties.
暂无评论