Training algorithms in the domain of deep learning, have led to significant breakthroughs across diverse and subsequent domains including speech, text, images, and video processing. While the research around deeper ne...
详细信息
Training algorithms in the domain of deep learning, have led to significant breakthroughs across diverse and subsequent domains including speech, text, images, and video processing. While the research around deeper network architectures, notably exemplified by ResNet's expansive 152-layer structures, has yielded remarkable outcomes, the exploration of computationally simpler shallow Convolutional Neural Networks (CNN) remains an area for further exploration. Activation functions, crucial in introducing non-linearity within neural networks, have driven substantial advancements. In this paper, we delve into hidden layer activations, particularly examining their complex piece-wise linear attributes. Our comprehensive experiments showcase the superior efficacy of these piece-wise linear activations over traditional Rectified Linear Units across various architectures. We propose a novel Adaptive Activation algorithm, AdAct, exhibiting promising performance improvements in diverse CNN and multilayer perceptron configurations, thereby presenting compelling results to support its usage.
An algorithm is developed for automated training of a multilayer perceptron with two non-linear layers. The initial algorithm approximately minimizes validation error with respect to the numbers of both hidden units a...
详细信息
An algorithm is developed for automated training of a multilayer perceptron with two non-linear layers. The initial algorithm approximately minimizes validation error with respect to the numbers of both hidden units and training epochs. A median filtering approach is added to reduce deviations between validation and testing errors. Next, the mean-squared error objective function is modified for use with classifiers using a method similar to Ho-Kashyap. Then, both theoretical and practical reasons are provided for introducing growing steps into the algorithm. Lastly, a sigmoidal input layer is added to limit the effects of input outliers and further improve the method. Using widely available datasets, the final network's average testing error is shown to be less than that of several other competing algorithms reported in the literature.
In this paper we present the outline of a novel electrostatic, secondorder Particle-in-Cell (PIC) algorithm, that makes use of 'ghost particle& located around true particle positions in order to represent a c...
详细信息
In this paper we present the outline of a novel electrostatic, secondorder Particle-in-Cell (PIC) algorithm, that makes use of 'ghost particle& located around true particle positions in order to represent a charge distribution. We implement our algorithm within EMPIRE-PIC, a PIC code developed at Sandia National Laboratories. We test the performance of our algorithm on a variety of many-core architectures including NVIDIA GPUs, conventional CPUs, and Intel's Knights Landing. Our preliminary results show the viability of secondorder methods for PIC applications on these architectures when compared to previous generations of many-core hardware. Specifically, we see an order of magnitude improvement in performance for secondorder methods between the Tesla K20 and Tesla P100 GPU devices, despite only a 4x improvement in the theoretical peak performance between the devices. Although these initial results show a large increase in runtime over first order methods, we hope to be able to show improved scaling behaviour and increased simulation accuracy in the future.
In this work a double step algorithm is presented for the integration of equations governing the behavior of von Mises material in plastic limit. The isotropic and kinematic hardenings employed are of general type and...
详细信息
In this work a double step algorithm is presented for the integration of equations governing the behavior of von Mises material in plastic limit. The isotropic and kinematic hardenings employed are of general type and the evolution of back stress follows the Armstrong-Frederick rule. Theoretical and numerical aspects are discussed in detail and a comparison is made with the classical backward Euler method. (c) 2012 Sharif University of Technology. Production and hosting by Elsevier B.V. All rights reserved.
Given a centralized undirected graph with costs associated with its edges, the capacitated minimum spanning tree problem is to find a minimum cost spanning tree of the given graph, subject to a capacity constraint in ...
详细信息
Given a centralized undirected graph with costs associated with its edges, the capacitated minimum spanning tree problem is to find a minimum cost spanning tree of the given graph, subject to a capacity constraint in all subtrees incident in the central node. As the problem is NP-hard, we propose an enhanced version of the well-known secondorder algorithm, described in [Karnaugh M. A new class of algorithms for multipoint network optimization. IEEE Transactions on Communications 1976;COM-24:500-5.,]. The original version of this algorithm is based on a look-ahead strategy, used for a tentative inclusion of a constraint to the problem, performed in each iteration. In the new enhanced version, we propose the inclusion of look-behind steps, which can be seen as the reverse of the look-ahead procedure. Therefore and using some memory features, the method can continue even when facing the traditional stopping criterion of the original algorithm. Computational experiments showing the effectiveness of the new method on benchmark instances are reported. (c) 2005 Elsevier Ltd. All rights reserved.
It is shown that two algorithms obtained by simplifying a Kalman filter considered for a second-order Markov model are H-infinity suboptimal. Similar to least mean squares (LMS) and normalised LMS (NLMS) algorithms, t...
详细信息
It is shown that two algorithms obtained by simplifying a Kalman filter considered for a second-order Markov model are H-infinity suboptimal. Similar to least mean squares (LMS) and normalised LMS (NLMS) algorithms, these second order algorithms can be thought of as approximate solutions to stochastic or deterministic least squares minimisation. It is proved that second-order LMS and NLMS are exact solutions causing the maximum energy gain from the disturbances to the predicted and filtered errors to be less than one, respectively. These algorithms are implemented in two steps. Operation of the first step is like conventional LMS/NLMS algorithms and the second step consists of the estimation of the weight increment vector and prediction of weights for the next iteration. This step applies simple smoothing on the increment of the estimated weights to estimate the speed of the weights. Also they are cost-effective, robust and attractive for improving the tracking performance of smoothly time-varying models.
暂无评论