检索结果-内蒙古大学图书馆

Decoding β-decay systematics: A global statistical model for β− half-lives*

学校读者我要写书评

暂无评论

Physical Review C 2009年第4期80卷 044332-044332页

作者： N. J. Costiris E. Mavrommatis K. A. Gernoth J. W. Clark []Physics Department Division of Nuclear Physics & Particle Physics University of Athens GR-15771 Athens Greece

Statistical modeling of nuclear data provides a novel approach to nuclear systematics complementary to established theoretical and phenomenological approaches based on quantum theory. Continuing previous studies in which global statistical modeling is pursued within the general framework of machine learning theory, we implement advances in training algorithms designed to improve generalization, in application to the problem of reproducing and predicting the half-lives of nuclear ground states that decay 100% by the β− mode. More specifically, fully connected, multilayer feed-forward artificial neural network models are developed using the Levenberg-Marquardt optimization algorithm together with Bayesian regularization and cross-validation. The predictive performance of models emerging from extensive computer experiments is compared with that of traditional microscopic and phenomenological models as well as with the performance of other learning systems, including earlier neural network models as well as the support vector machines recently applied to the same problem. In discussing the results, emphasis is placed on predictions for nuclei that are far from the stability line, and especially those involved in r-process nucleosynthesis. It is found that the new statistical models can match or even surpass the predictive performance of conventional models for β-decay systematics and accordingly should provide a valuable additional tool for exploring the expanding nuclear landscape.

关键词： Decoding β-decay systematics global statistical model β- half-lives network model nuclear data Support Vector Network multilayers quantum theory Half Life r PROCESS PERFORMANCE MODELING Beta decay optimization algorithms statistical models (nuclear)

MetaOptimize: A Framework for Optimizing Step Sizes and Other Meta-parameters

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sharifnassab, Arsalan Salehkaleybar, Saber Sutton, Richard University of Alberta Canada Leiden University Netherlands Openmind Research Institute

We address the challenge of optimizing meta-parameters (i.e., hyperparameters) in machine learning algorithms, a critical factor influencing training efficiency and model performance. Moving away from the computationally expensive traditional meta-parameter search methods, we introduce MetaOptimize framework that dynamically adjusts meta-parameters, particularly step sizes (also known as learning rates), during training. More specifically, MetaOptimize can wrap around any first-order optimization algorithm, tuning step sizes on the fly to minimize a specific form of regret that accounts for long-term effect of step sizes on training, through a discounted sum of future losses. We also introduce low complexity variants of MetaOptimize that, in conjunction with its adaptability to multiple optimization algorithms, demonstrate performance competitive to those of best hand-crafted learning rate schedules across various machine learning applications. © 2024, CC BY.

关键词： optimization algorithms

Trajectory-based proofs for sampled-data extremum seeking control

学校读者我要写书评

暂无评论

Trajectory-based proofs for sampled-data extremum seeking co...

American Control Conference

作者： Sei Zhen Khong Dragan Nesic Ying Tan Chris Manzie Department of Electrical and Electronic Engineering The University of Melbourne VIC 3010 Australia Department of Mechanical Engineering The University of Melbourne VIC 3010 Australia

ISBN: (纸本)9781479901777

Extremum seeking of nonlinear systems based on a sampled-data control law is revisited. It is established that under some generic assumptions, semi-global practical asymptotically stable convergence to an extremum can be achieved. To this end, trajectory-based arguments are employed, by contrast with Lyapunov-function-type approaches in the existing literature. The proof is simpler and more straightforward;it is based on assumptions that are in general easier to verify. The proposed extremum seeking framework may encompass more general optimisation algorithms, such as those which do not admit a state-update realisation and/or Lyapunov functions. Multi-unit extremum seeking is also investigated within the context of accelerating the speed of convergence.

关键词： Data sampling optimization algorithms acceleration control law Converge

Regret Lower Bound and Optimal Algorithm in Finite Stochastic Partial Monitoring 15

学校读者我要写书评

暂无评论

Regret Lower Bound and Optimal Algorithm in Finite Stochasti...

Annual Conference on Neural Information Processing Systems

作者： Junpei Komiyama Junya Honda Hiroshi Nakagawa The University of Tokyo

ISBN: (纸本)9781510825024

Partial monitoring is a general model for sequential learning with limited feedback formalized as a game between two players. In this game, the learner chooses an action and at the same time the opponent chooses an outcome, then the learner suffers a loss and receives a feedback signal. The goal of the learner is to minimize the total loss. In this paper, we study partial monitoring with finite actions and stochastic outcomes. We derive a logarithmic distribution-dependent regret lower bound that defines the hardness of the problem. Inspired by the DMED algorithm (Honda and Takemura, 2010) for the multi-armed bandit problem, we propose PM-DMED, an algorithm that minimizes the distribution-dependent regret. PM-DMED significantly outperforms state-of-the-art algorithms in numerical experiments. To show the optimality of PM-DMED with respect to the regret bound, we slightly modify the algorithm by introducing a hinge function (PM-DMED-Hinge). Then, we derive an asymptotically optimal regret upper bound of PM-DMED-Hinge that matches the lower bound.

关键词： Regret optimization algorithms Lower bound Game theory Game animals optimality

A New Global optimization Algorithm for Mixed-Integer-Discrete-Continuous Variables based on Particles Swarm optimization

学校读者我要写书评

暂无评论

A New Global Optimization Algorithm for Mixed-Integer-Discre...

Biennial Institute of Electrical and Electronics Engineers Conference on Electomagnetic Field Computation

作者： Ziyan Ren Minh-Trien Pham Wei Li Chang Seop Koh School of Electrical Engineering Chungbuk National University

ISBN: (纸本)9781424470594

This paper presents a new global optimization algorithm for mixed-integer-discrete-continuous variables. In the algorithm, an augmented objective function is constructed by introducing a penalty function to treat both the integer and discrete variables as continuous ones. Particles swarm optimization (PSO) is, then, applied to the augmented objective function to find a global optimal point.

关键词： optimization algorithms Global optimization discrete transforms Integer Objective function global optimum algorithms penalty function

Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning 15

学校读者我要写书评

暂无评论

Variational Information Maximisation for Intrinsically Motiv...

Annual Conference on Neural Information Processing Systems

作者： Shakir Mohamed Danilo J. Rezende Google DeepMind London

ISBN: (纸本)9781510825024

The mutual information is a core statistical quantity that has applications in all areas of machine learning, whether this is in training of density models over multiple data modalities, in maximising the efficiency of noisy transmission channels, or when learning behaviour policies for exploration by artificial agents. Most learning algorithms that involve optimisation of the mutual information rely on the Blahut-Arimoto algorithm - an enumerative algorithm with exponential complexity that is not suitable for modern machine learning applications. This paper provides a new approach for scalable optimisation of the mutual information by merging techniques from variational inference and deep learning. We develop our approach by focusing on the problem of intrinsically-motivated learning, where the mutual information forms the definition of a well-known internal drive known as empowerment. Using a variational lower bound on the mutual information, combined with convolutional networks for handling visual input streams, we develop a stochastic optimisation algorithm that allows for scalable information maximisation and empowerment-based reasoning directly from pixels to actions.

关键词： optimization algorithms Machine Learning Variational Inference Transmission channels learning (artificial intelligence) Empowerment program area pixel Artificial Intelligence Scalability

A Markovian Model for Learning-to-Optimize

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sucker, Michael Ochs, Peter Department of Mathematics University of Tübingen Tübingen Germany Department of Mathematics and Computer Science Saarland University Saarbrücken Germany

We present a probabilistic model for stochastic iterative algorithms with the use case of optimization algorithms in mind. Based on this model, we present PAC-Bayesian generalization bounds for functions that are defined on the trajectory of the learned algorithm, for example, the expected (non-asymptotic) convergence rate and the expected time to reach the stopping criterion. Thus, not only does this model allow for learning stochastic algorithms based on their empirical performance, it also yields results about their actual convergence rate and their actual convergence time. We stress that, since the model is valid in a more general setting than learning-to-optimize, it is of interest for other fields of application, too. Finally, we conduct five practically relevant experiments, showing the validity of our claims. © 2024, CC BY.

关键词： optimization algorithms

EXADAM: THE POWER OF ADAPTIVE CROSS-MOMENTS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Adly, Ahmed M. Egypt

This paper introduces EXAdam (EXtended Adam), a novel optimization algorithm that builds upon the widely-used Adam [1] optimizer. EXAdam incorporates three key enhancements: (1) new debiasing terms for improved moment estimation, (2) a gradient-based acceleration mechanism for increased responsiveness to the current loss landscape, and (3) a dynamic step size formula that allows for continuous growth of the learning rate throughout training. These innovations work synergistically to address limitations of the original Adam algorithm, potentially offering improved convergence properties, enhanced ability to escape saddle points, and greater robustness to hyperparameter choices. I provide a theoretical analysis of EXAdam’s components and their interactions, highlighting the algorithm’s potential advantages in navigating complex optimization landscapes. Empirical evaluations demonstrate EXAdam’s superiority over Adam, achieving 48.07% faster convergence and yielding improvements of 4.6%, 4.13%, and 2.39% in training, validation, and testing accuracies, respectively, when applied to a CNN trained on the CIFAR-10 dataset [2]. While these results are promising, further empirical validation across diverse tasks is essential to fully gauge EXAdam’s efficacy. Nevertheless, EXAdam represents a significant advancement in adaptive optimization techniques, with promising implications for a wide range of machine learning applications. This work aims to contribute to the ongoing development of more efficient, adaptive, and universally applicable optimization methods in the field of machine learning and artificial intelligence. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

Simmering: Sufficient is better than optimal for training neural networks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Babayan, Irina Aliahmadi, Hazhir Anders, Greg van Department of Physics Engineering Physics and Astronomy Queen’s University KingstonONK7L 3N6 Canada

The broad range of neural network training techniques that invoke optimization but rely on ad hoc modification for validity [1? –4] suggests that optimization-based training is misguided. Shortcomings of optimization-based training are brought to particularly strong relief by the problem of overfitting, where naive optimization produces spurious outcomes.[5–7] The broad success of neural networks for modelling physical processes [8–12] has prompted advances that are based on inverting the direction of investigation and treating neural networks as if they were physical systems in their own right.[13–16] These successes raise the question of whether broader, physical perspectives could motivate the construction of improved training algorithms. Here, we introduce simmering, a physics-based method that trains neural networks to generate weights and biases that are merely "good enough", but which, paradoxically, outperforms leading optimization-based approaches. Using classification and regression examples we show that simmering corrects neural networks that are overfit by Adam [17], and show that simmering avoids overfitting if deployed from the outset. Our results question optimization as a paradigm for neural network training, and leverage information-geometric arguments to point to the existence of classes of sufficient training algorithms that do not take optimization as their starting point. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms