检索结果-内蒙古大学图书馆

Near-Optimal Rates for O(1)-Smooth DP-SCO with a Single Epoch and Large Batches 36

学校读者我要写书评

暂无评论

Near-Optimal Rates for O(1)-Smooth DP-SCO with a Single Epoc...

36th International Conference on Algorithmic Learning Theory, ALT 2025

作者： Choquette-Choo, Christopher A. Ganesh, Arun Thakurta, Abhradeep Mountain View CA United States Seattle WA United States

In this paper we revisit the DP stochastic convex optimization (SCO) problem. For convex smooth losses, it is well-known that the canonical DP-SGD (stochastic gradient descent) achieves the optimal rate of (Equation presented) under (Ε, δ)-DP (Bassily et al., 2014), and also well-known that variants of DP-SGD can achieve the optimal rate in a single epoch (Feldman et al., 2020). However, the batch gradient complexity (i.e., number of adaptive optimization steps), which is important in applications like federated learning, is less well-understood. In particular, all prior work on DP-SCO requires Ω(n) batch gradient steps, multiple epochs, or convexity for privacy. We propose an algorithm, Accelerated-DP-SRGD (stochastic recursive gradient descent), which bypasses the limitations of past work: it achieves the optimal rate for DP-SCO (up to polylog factors), in a single epoch using √n batch gradient steps with batch size √n, and can be made private for arbitrary (non-convex) losses via clipping. If the global minimizer is in the constraint set, we can further improve this to n1/4 batch gradient steps with batch size n3/4. To achieve this, our algorithm combines three key ingredients, a variant of stochastic recursive gradients (SRG), accelerated gradient descent, and correlated noise generation from DP continual counting. © 2025 C.A. Choquette-Choo, A. Ganesh & A. Thakurta.

关键词： optimization algorithms

An Interior-Point Algorithm for Continuous Nonlinearly Constrained optimization with Noisy Function and Derivative Evaluations

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Curtis, Frank E. Dezfulian, Shima Waechter, Andreas Department of Industrial and Systems Engineering Lehigh University United States Department of Industrial Engineering and Management Sciences Northwestern University United States

An algorithm based on the interior-point methodology for solving continuous nonlinearly constrained optimization problems is proposed, analyzed, and tested. The distinguishing feature of the algorithm is that it presumes that only noisy values of the objective and constraint functions and their first-order derivatives are available. The algorithm is based on a combination of a previously proposed interior-point algorithm that allows inexact subproblem solutions and recently proposed algorithms for solving bound- and equality-constrained optimization problems with only noisy function and derivative values. It is shown that the new interior-point algorithm drives a stationarity measure below a threshold that depends on bounds on the noise in the function and derivative values. The results of numerical experiments show that the algorithm is effective across a wide range of *** Codes 65K05, 49M37, 90C53, 90C30, 90C51 © 2025, CC BY.

关键词： optimization algorithms

Solution of Robust Linear optimization Problems

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Mondal, Parthasarathi Ojha, Akshay Kumar Department of Mathematics IIT Bhubaneswar India

Robust optimization(RO) is an important tool for handling optimization problem with uncertainty. The main objective of RO is to solve optimization problems due to uncertainty associated with constraints satisfying all realizations of uncertain values within a given uncertainty set. The challenge of RO is to reformulate the constraints so that the uncertain optimization problem is transformed into a tractable deterministic form. In this paper, we have given more emphasis to study the robust counterpart(RC) of the RO problems and have developed a mathematical model on the solution strategy for robust linear optimization problems, where the constraints only are associated with uncertainties. The box and ellipsoidal uncertainty sets are considered and some illustrative numerical examples have been solved in each corresponding case for validating our proposed method. Copyright © 2025, The Authors. All rights reserved.

关键词： optimization algorithms

Reward-aware Preference optimization: A Unified Mathematical Framework for Model Alignment

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Sun, Shengyang Zhang, Yian Bukharin, Alexander Mosallanezhad, David Zeng, Jiaqi Singhal, Soumye Shen, Gerald Renduchintala, Adithya Konuk, Tugrul Dong, Yi Wang, Zhilin Chichkov, Dmitry Delalleau, Olivier Kuchaiev, Oleksii NVIDIA United States

The rapid development of large language model (LLM) alignment algorithms has resulted in a complex and fragmented landscape, with limited clarity on the effectiveness of different methods and their inter-connections. This paper introduces Reward-Aware Preference optimization (RPO), a mathematical framework that unifies popular preference optimization techniques in LLM alignment, including DPO, IPO, SimPO, and REINFORCE (LOO), among others. RPO provides a structured approach to disentangle and systematically study the impact of various design choices—such as the optimization objective, the number of responses per prompt, and the use of implicit versus explicit reward models—on LLM preference optimization. We additionally propose a new experimental setup that enables the clean and direct ablation of such design choices. Through an extensive series of ablation studies within the RPO framework, we gain insights into the critical factors shaping model alignment, offering practical guidance on the most effective strategies for improving LLM alignment. © 2025, CC BY.

关键词： optimization algorithms

Marginal Price optimization

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Loesch, Stefan Richardson, Mark

We introduce a new framework for optimal routing and arbitrage in AMM driven markets. This framework improves on the original best-practice convex optimization by restricting the search to the boundary of the optimal space. We can parameterize this boundary using a set of prices, and a potentially very high dimensional optimization problem (2 optimization variables per curve) gets reduced to to a much lower dimensional root finding problem (1 optimization variable per token, irregardless of number of the curves). Our reformulation is similar to the dual problem of a reformulation of the original convex problem. We show our reformulation of the problem is equivalent to the original formulation except in the case of infinitely concentrated liquidity, where we provide a suitable approximation. Our formulation performs far better than the original one in terms of speed – we obtain an improvement of up to 200x against Clarabel, the new CVXPY default solver – and robustness, especially on levered curves. Copyright © 2025, The Authors. All rights reserved.

关键词： optimization algorithms

Controlled optimization with a Prescribed Finite-Time Convergence Using a Time Varying Feedback Gradient Flow

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Abdel Aal, Osama F. Özbek, Necdet Sinan Viola, Jairo Chen, YangQuan Dept. of Mechanical Engineering University of California Merced 5200 N. Lake Rd MercedCA95343 United States Adana Alparslan Türkeş Science and Technology University Faculty of Engineering Department of Electrical Electronics Engineering Adana Turkey

From the perspective of control theory, the gradient descent optimization methods can be regarded as a dynamic system where various control techniques can be designed to enhance the performance of the optimization method. In this paper, we propose a prescribed finite-time convergent gradient flow that uses time-varying gain nonlinear feedback that can drive the states smoothly towards the minimum. This idea is different from the traditional finite-time convergence algorithms that relies on fractional-power or signed gradient as a nonlinear feedback, that is proved to have finite/fixed time convergence satisfying strongly convex or the Polyak-Lojasiewicz (PL) inequality, where due to its nature, the proposed approach was shown to achieve this property for both strongly convex function, and for those satisfies Polyak-Lojasiewic inequality. Our method is proved to converge in a prescribed finite time via Lyapunov theory. Numerical experiments were presented to illustrate our results. © 2025, CC BY.

关键词： optimization algorithms

Systematic Design of Decentralized algorithms for Consensus optimization

学校读者我要写书评

暂无评论

IEEE CONTROL SYSTEMS LETTERS 2019年第4期3卷 966-971页

作者： Han, Shuo Univ Illinois Dept Elect & Comp Engn Chicago IL 60607 USA

We propose a separation principle that enables a systematic way of designing decentralized algorithms used in consensus optimization. Specifically, we show that a decentralized optimization algorithm can be constructed by combining a non-decentralized base optimization algorithm and decentralized consensus tracking. The separation principle provides modularity in both the design and analysis of algorithms under an automated convergence analysis framework using integral quadratic constraints (IQCs). We show that consensus tracking can be incorporated into the IQC-based analysis. The workflow is illustrated through the design and analysis of a decentralized algorithm based on the alternating direction method of multipliers.

关键词： optimization algorithms robust control

Advancements in Bat algorithm 4th

学校读者我要写书评

暂无评论

Advancements in Bat algorithm

4th International Conference on Computational Methods in Science and Technology, ICCMST 2024

作者： Kaur, Arvinder Kumar, Yugal Sidhu, Jagpreet Department of Information Technology Chandigarh Engineering College-CGC Landran Punjab Mohali India Computer Engineering/STME NMIMS Chandigarh Campus India

ISBN: (纸本)9781032911571

The modern times have led to the adoption of distinctive meta-heuristic procedures for solving distinct class of optimization-problems. The meta-heuristics procedures have benefit above conventional algorithms because of strong capabilities of exploration and exploitation. In a similar vein, a Bat algorithm was created based on the echolocation habits of bats to address optimisation difficulties. Several advancements have been made in bat algorithm since then to achieve the state-of art results. The purpose of the study is to investigate bat algorithm for the various advancements done. Henceforth, the study presents the miniature analysis on advancements in bat algorithm. The bat algorithm applications in different areas are also investigated. © 2025 the Author(s).

关键词： optimization algorithms

A Graph Clustering Algorithm Based on Gravitational Vector 16th

学校读者我要写书评

暂无评论

A Graph Clustering Algorithm Based on Gravitational Vector

16th International Conference on Genetic and Evolutionary Computing, ICGEC 2024

作者： Li, Jitao Yao, Yao Xu, Chugui Tian, Xue Department of Information Science and Technology Taishan University Tai’an271000 China

ISBN: (纸本)9789819615346

The traditional clustering algorithms mostly use simple one-dimensional distance measurement and overall optimization strategy for classification, one-dimensional measurement method can identify similarity matrix, but cannot keep the spatial location relationship between samples, because of the loss of data information between the sample which cause the clustering accuracy problem, this paper puts forward a graph clustering algorithm based on gravitational vector. This method combines the principle of gravity principle, uses the multi-dimensional vector instead of the traditional one-dimensional distance measurement method, and excavates the connection between the samples by establishing the local balance relationship, and finally achieves the purpose of improving the local information extraction and data distribution perception ability of the algorithm. Using the present algorithm in artificial datasets and UCI datasets can show that the present algorithm can cope well with multi-type, unbalanced, high complexity data and get good clustering results. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： optimization algorithms