检索结果-内蒙古大学图书馆

Foxtsage vs. Adam: Revolution or Evolution in optimization?

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Aula, Sirwan A. Rashid, Tarik A. Soran University Computer Science Department Erbil Soran Iraq Computer Science & Engineering Department Artificial Intelligence & Innovation Centre University of Kurdistan Hewler Erbil Iraq

optimization techniques are pivotal in neural network training, shaping both predictive performance and convergence efficiency. This study introduces Foxtsage, a novel hybrid optimisation approach that integrates the Hybrid FOX-TSA with Stochastic Gradient Descent for training Multi-Layer Perceptron models. The proposed Foxtsage method is benchmarked against the widely adopted Adam optimizer across multiple standard datasets, focusing on key performance metrics such as training loss, accuracy, precision, recall, F1-score, and computational time. Experimental results demonstrate that Foxtsage achieves a 42.03% reduction in loss mean (Foxtsage: 9.508, Adam: 16.402) and a 42.19% improvement in loss standard deviation (Foxtsage: 20.86, Adam: 36.085), reflecting enhanced consistency and robustness. Modest improvements in accuracy mean (0.78%), precision mean (0.91%), recall mean (1.02%), and F1-score mean (0.89%) further underscore its predictive performance. However, these gains are accompanied by an increased computational cost, with a 330.87% rise in time mean (Foxtsage: 39.541 seconds, Adam: 9.177 seconds). By effectively combining the global search capabilities of FOX-TSA with the stability and adaptability of SGD, Foxtsage presents itself as a robust and viable alternative for neural network optimization tasks. © 2024, CC BY.

关键词： optimization algorithms

Global optimization Algorithm through High-Resolution Sampling

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cortild, Daniel Delplancke, Claire Oudjane, Nadia Peypouquet, Juan University of Groningen Netherlands Laboratoire de Finance des Marchés de l’Energie Dauphine CREST EDF R&D France EDF R&D Palaiseau France

We present an optimization algorithm that can identify a global minimum of a potentially nonconvex smooth function with high probability, assuming the Gibbs measure of the potential satisfies a logarithmic Sobolev inequality. Our contribution is twofold: on the one hand we propose said global optimization method, which is built on an oracle sampling algorithm producing arbitrarily accurate samples from a given Gibbs measure. On the other hand, we propose a new sampling algorithm, drawing inspiration from both overdamped and underdamped Langevin dynamics, as well as from the high-resolution differential equation known for its acceleration in deterministic settings. While the focus of the paper is primarily theoretical, we demonstrate the effectiveness of our algorithms on the Rastrigin function, where it outperforms recent approaches. © 2024, CC BY.

关键词： optimization algorithms

Momentum-Adapted Gradient Tracking Stochastic Algorithm for Distributed optimization

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Li, Yantao Hu, Hanqing Zhang, Keke Lü, Qingguo Deng, Shaojiang Li, Huaqing College of Computer Science Chongqing University Chongqing400044 China College of Electronic and Information Engineering Southwest University Chongqing400715 China

In this paper, we study distributed optimization problems, where each node owns a local convex cost function that is calculated as the average of amounts of constituent functions, and multiple nodes collaborate to minimize a finite sum of these local functions. Reviewing existing work, distributed optimization methods with adaptive momentum that consider reducing computation costs have not yet been explored. To this aim, we propose a gradient tracking stochastic distributed optimization algorithm with adaptive momentum, called GTSADAM. GTSADAM combines the distributed adaptive momentum method for faster convergence with the variance reduction mechanism to reduce computation costs. We provide a convergence analysis indicating that under certain conditions of step sizes, GTSADAM achieves convergence in the mean to the exact optimal solution. Meanwhile, a linear convergence rate can be attained when each constituent function exhibits strong convexity and smoothness. Moreover, GTSADAM maintains the acceleration efficiency of adaptive momentum while minimizing computation costs, which is confirmed by numerical simulations, and its performance is better than that of existing methods. © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

Automorphism-Assisted Quantum Approximate optimization Algorithm for efficient graph optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Prakash, Vaibhav

In this article we report on the application of the Quantum Approximate optimization Algorithm (QAOA) to solve the unweighted MaxCut problem on tree-structured graphs. Specifically, we utilize the Nauty (No Automorphisms, Yes?) package to identify graph automorphisms, focusing on determining edge equivalence classes. These equivalence classes also correspond to symmetries in the terms of the associated Ising Hamiltonian. By exploiting these symmetries, we achieve a significant reduction in the complexity of the Hamiltonian, thereby facilitating more efficient quantum simulations. We conduct benchmark experiments on graphs with up to 34 nodes on memory and CPU intensive TPU provided by google Colab, applying QAOA with a single layer (p=1). The approximation ratios obtained from both the full and symmetry-reduced Hamiltonians are systematically compared. Our results show that using automorphism-based symmetries to reduce the Pauli terms in the Hamiltonian can significantly decrease computational overhead without compromising the quality of the solutions obtained. © 2024, CC BY.

关键词： optimization algorithms

Adaptive Weighting Push-SUM for Decentralized optimization with Statistical Diversity

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhou, Yiming Cheng, Yifei Xu, Linli Chen, Enhong Hefei China

Statistical diversity is a property of data distribution and can hinder the optimization of a decentralized network. However, the theoretical limitations of the Push-SUM protocol reduce the performance in handling the statistical diversity of optimization algorithms based on it. In this paper, we theoretically and empirically mitigate the negative impact of statistical diversity on decentralized optimization using the Push-SUM protocol. Specifically, we propose the Adaptive Weighting Push-SUM protocol, a theoretical generalization of the original Push-SUM protocol where the latter is a special case of the former. Our theoretical analysis shows that, with sufficient communication, the upper bound on the consensus distance for the new protocol reduces to O(1/N), whereas it remains at O(1) for the Push-SUM protocol. We adopt SGD and Momentum SGD on the new protocol and prove that the convergence rate of these two algorithms to statistical diversity is O(N/T) on the new protocol, while it is O(Nd/T) on the Push-SUM protocol, where d is the parameter size of the training model. To address statistical diversity in practical applications of the new protocol, we develop the Moreau weighting method for its generalized weight matrix definition. This method, derived from the Moreau envelope, is an approximate optimization of the distance penalty of the Moreau envelope. We verify that the Adaptive Weighting Push-SUM protocol is practically more efficient than the Push-SUM protocol via deep learning experiments. © 2024, CC0.

关键词： optimization algorithms

Can optimization Trajectories Explain Multi-Task Transfer?

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Mueller, David Dredze, Mark Andrews, Nicholas Department of Computer Science Johns Hopkins University United States

Despite the widespread adoption of multi-task training in deep learning, little is understood about how multi-task learning (MTL) affects generalization. Prior work has conjectured that the negative effects of MTL are due to optimization challenges that arise during training, and many optimization methods have been proposed to improve multi-task performance. However, recent work has shown that these methods fail to consistently improve multi-task generalization. In this work, we seek to improve our understanding of these failures by empirically studying how MTL impacts the optimization of tasks, and whether this impact can explain the effects of MTL on generalization. We show that MTL results in a generalization gap—a gap in generalization at comparable training loss—between single-task and multi-task trajectories early into training. However, we find that factors of the optimization trajectory previously proposed to explain generalization gaps in single-task settings cannot explain the generalization gaps between single-task and multi-task models. Moreover, we show that the amount of gradient conflict between tasks is correlated with negative effects to task optimization, but is not predictive of generalization. Our work sheds light on the underlying causes for failures in MTL and, importantly, raises questions about the role of general purpose multi-task optimization algorithms. We release code for all of our experiments and analysis here: https://***/davidandym/Multi-Task-optimization © 2024, CC BY.

关键词： optimization algorithms

Design and optimization of an Eco-Efficient Vehicle Routing Model for Heterogeneous Fleets and Multi Depot Systems

学校读者我要写书评

暂无评论

Procedia Computer Science 2023年 230卷 629-640页

作者： Shreeyash Vyakarnam Gokulakrishnan Jayakumar Joseph Augustine Cherukara Department of Industrial and Operations Engineering University of Michigan Ann Arbor USA School for Environment and Sustainability University of Michigan Ann Arbor USA

In the dynamic landscape of vehicle routing problems (VRP), the traditional focus on cost minimization has significantly evolved due to the pressing need to address environmental concerns. With the logistics industry experiencing exponential growth and increased vehicle utilization, sustainability has become a central consideration both globally and in strategic planning. The Vehicle Routing Problem (VRP) remains a key challenge in logistics and transportation management, aiming to optimize routes and vehicle allocation to minimize transportation costs. This study introduces an innovative VRP model that intricately balances economic efficiency with ecological responsibility, addressing the urgent call for sustainable solutions amidst the ongoing environmental crisis. The proposed model features a diverse fleet of trucks, each designated for specific product types, creating a heterogeneous fleet. Originating from multiple depots, these trucks ensure comprehensive geographic coverage and enhance logistical efficiency. The core objective of the model is twofold: minimizing the financial transportation costs associated with supplying multiple products from multiple depots to customers and significantly reducing the carbon footprint, thereby addressing environmental sustainability. Two distinct solutions are proposed: one based on mixed-integer linear programming (MILP) and the other utilizing a heuristic approach. The choice of mixed-integer linear programming instead of linear programming is due to the need to make binary route decisions. Results are compared, focusing on outcome variation and computation time. Additionally, the solution demonstrates upto 100% variation in results when comparing the heuristic model with the optimal values derived from mixed-integer linear programming, though it requires less computation time. This research underscores the importance of seamlessly integrating economic and environmental considerations in the logistics industry. The propo

关键词： Environmental Sustainability Logistics Eco-Efficient Vehicle Routing Model optimization algorithms Heuristic models Clean Energy Greenhouse Gas Emission Reduction Carbon Footprint Reduction Green Transportation Solutions

PAMSO: Parametric Autotuning Multi-time Scale optimization Algorithm

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ramanujam, Asha Li, Can Davidson School of Chemical Engineering Purdue University 480 W. Stadium Ave West LafayetteIN47907 United States

optimization models with decision variables in multiple time scales are widely used across various fields such as integrated planning and scheduling. To address scalability challenges in these models, we present the Parametric Autotuning Multi-Time Scale optimization (PAMSO) algorithm. PAMSO tunes parameters in a low-fidelity model to assist in solving a higher-fidelity multi-time scale optimization model. These parameters represent the mismatch between the two models. PAMSO defines a black-box function with tunable parameters as inputs and multi-scale cost as output, optimized using Derivative-Free optimization methods. This scalable algorithm allows optimal parameters from one problem to be transferred to similar problems. Case studies demonstrate its effectiveness on an MINLP model for integrated design and scheduling in a resource task network with around 67,000 variables and an MILP model for integrated planning and scheduling of electrified chemical plants and renewable resources with around 26 million variables. © 2024, CC BY.

关键词： optimization algorithms

Optimal Rates for Robust Stochastic Convex optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Gao, Changyu Lowy, Andrew Zhou, Xingyu Wright, Stephen J. University of Wisconsin-Madison MadisonWI United States Wayne State University DetroitMI United States

Machine learning algorithms in high-dimensional settings are highly susceptible to the influence of even a small fraction of structured outliers, making robust optimization techniques essential. In particular, within the ϵ-contamination model, where an adversary can inspect and replace up to an ϵ-fraction of the samples, a fundamental open problem is determining the optimal rates for robust stochastic convex optimization (SCO) under such contamination. We develop novel algorithms that achieve minimax-optimal excess risk (up to logarithmic factors) under the ϵ-contamination model. Our approach improves over existing algorithms, which are not only suboptimal but also require stringent assumptions, including Lipschitz continuity and smoothness of individual sample functions. By contrast, our optimal algorithms do not require these restrictive assumptions, and can handle nonsmooth but Lipschitz population loss functions. We complement our algorithmic developments with a tight lower bound for robust SCO. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms