检索结果-内蒙古大学图书馆

Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Urbán, Jorge F. Stefanou, Petros Pons, José A. Departament de Física Aplicada Universitat d’Alacant Ap. Correus 99 Comunitat Valenciana Alacant03830 Spain Departament d’Astronomia i Astrofísica Universitat de València Dr Moliner 50 Comunitat Valenciana València46100 Spain

This study investigates the potential accuracy boundaries of physics-informed neural networks, contrasting their approach with previous similar works and traditional numerical methods. We find that selecting improved optimization algorithms significantly enhances the accuracy of the results. Simple modifications to the loss function may also improve precision, offering an additional avenue for enhancement. Despite optimization algorithms having a greater impact on convergence than adjustments to the loss function, practical considerations often favor tweaking the latter due to ease of implementation. On a global scale, the integration of an enhanced optimizer and a marginally adjusted loss function enables a reduction in the loss function by several orders of magnitude across diverse physical problems. Consequently, our results obtained using compact networks (typically comprising 2 or 3 layers of 20-30 neurons) achieve accuracies comparable to finite difference schemes employing thousands of grid points. This study encourages the continued advancement of PINNs and associated optimization techniques for broader applications across various fields. © 2024, CC BY-NC-ND.

关键词： optimization algorithms

Variable reduction as a nonlinear preconditioning approach for optimization problems

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ciaramella, Gabriele Vanzan, Tommaso MOX Dipartimento di Matematica Politecnico di Milano Italy Dipartimento di Scienze Matematiche Politecnico di Torino Italy

When considering an unconstrained minimization problem, a standard approach is to solve the optimality system with a Newton method possibly preconditioned by, e.g., nonlinear elimination. In this contribution, we argue that nonlinear elimination could be used to reduce the number of optimization variables by artificially constraining them to satisfy a subset of the optimality conditions. Consequently, a reduced objective function is derived which can now be minimized with any optimization algorithm. By choosing suitable variables to eliminate, the conditioning of the reduced optimization problem is largely improved. We here focus in particular on a right preconditioned gradient descent and show theoretical and numerical results supporting the validity of the presented approach. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

Orthogonal Finetuning for Direct Preference optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yang, Chenxu Jia, Ruipeng Gu, Naibin Lin, Zheng Chen, Siyuan Pang, Chao Yin, Weichong Sun, Yu Wu, Hua Wang, Weiping Institute of Information Engineering Chinese Academy of Sciences Beijing China School of Cyber Security University of Chinese Academy of Sciences Beijing China Baidu Inc. Beijing China

DPO is an effective preference optimization algorithm. However, the DPO-tuned models tend to overfit on the dispreferred samples, manifested as overly long generations lacking diversity. While recent regularization approaches have endeavored to alleviate this issue by modifying the objective function, they achieved that at the cost of alignment performance degradation. In this paper, we innovatively incorporate regularization from the perspective of weight updating to curb alignment overfitting. Through the pilot experiment, we discovered that there exists a positive correlation between overfitting and the hyperspherical energy fluctuation. Hence, we introduce orthogonal finetuning for DPO via a weight-Rotated Preference optimization (RoPO) method, which merely conducts rotational and magnitude-stretching updates on the weight parameters to maintain the hyperspherical energy invariant, thereby preserving the knowledge encoded in the angle between neurons. Extensive experiments demonstrate that our model aligns perfectly with human preferences while retaining the original expressive capacity using only 0.0086% of the trainable parameters, suggesting an effective regularization against overfitting. Specifically, RoPO outperforms DPO by up to 10 points on MT-Bench and by up to 2.8 points on AlpacaEval 2, while enhancing the generation diversity by an average of 6 points. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

A SCALABLE K-MEDOIDS CLUSTERING VIA WHALE optimization ALGORITHM

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chenan, Huang Tsutsumida, Narumasa Graduate school of Science & Engineering Saitama University Japan

Unsupervised clustering has emerged as a critical tool for uncovering hidden patterns and insights from vast, unlabeled datasets. However, traditional methods like Partitioning Around Medoids (PAM) struggle with scalability due to their quadratic computational complexity. To address this limitation, we introduceWOA-kMedoids, a novel unsupervised clustering method that incorporates theWhale optimization Algorithm(WOA)-a nature-inspiredmetaheuristic inspired by the hunting strategies of humpback whales. By optimizing centroid selection, WOA-kMedoids reduces computational complexity of the k-medoids algorithm from quadratic to near-linear with respect to the number of observations. This improvement in efficiency enables WOA-kMedoids to be scalable to large datasets while maintaining high clustering accuracy. We evaluated the performance of WOAkMedoids on 25 diverse time series datasets from the UCR archive. Our empirical results demonstrate that WOA-kMedoids maintains clustering accuracy similar to PAM. While WOA-kMedoids exhibited slightly higher runtime than PAM on small datasets (less than 300 observations), it outperformed PAM in computational efficiency on larger datasets. The scalability of WOA-kMedoids, combined with its consistently high accuracy, positions it as a promising and practical choice for unsupervised clustering in big data applications. WOA-kMedoids has implications for efficient knowledge discovery in massive, unlabeled datasets across various domains. © 2024, CC BY-NC-ND.

关键词： optimization algorithms

A Fast Algorithm for Convex Composite Bi-Level optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Merchav, Roey Sabach, Shoham Teboulle, Marc Faculty of Industrial Engineering and Management Technion-Israel Institute of Technology Haifa3200003 Israel School of Mathematical Sciences Tel-Aviv University Ramat-Aviv69978 Israel

In this paper, we study convex bi-level optimization problems where both the inner and outer levels are given as a composite convex minimization. We propose the Fast Bi-level Proximal Gradient (FBi-PG) algorithm, which can be interpreted as applying FISTA to a dynamic regularized composite objective function. The dynamic nature of the regularization parameters allows to achieve an optimal fast convergence rate of O(1/k2) in terms of the inner objective function. This is the fastest known convergence rate under no additional restrictive assumptions. We also show that FBi-PG achieves sub-linear simultaneous rates in terms of both the inner and outer objective functions. Moreover, we show that under an Hölderian type error bound assumption on the inner objective function, the FBi-PG algorithm converges to an optimal solution of the bi-level optimization problem. Finally, we present numerical experiments demonstrating the performances of the proposed *** Codes 90C25, 65K05 Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

A Novel Identification Scheme of an Inverse Source Problem Based on Hilbert Reproducing Kernels 1

学校读者我要写书评

暂无评论

1st International Conference on New Trends of applied Mathematics, ICNTAM 2022

作者： Jauberteau, François Nachaoui, Mourad Zaroual, Sara Laboratoire de Mathématiques Jean Leray UMR6629 CNRS Université de Nantes 2 rue de la Houssinière BP92208 Nantes44322 France Equipe de Mathématiques et Interactions FST Béni-Mellal Université Sultan Moulay Slimane Béni-Mellal Morocco

ISBN: (数字)9783031330698

ISBN: (纸本)9783031330681

This work deals with an inverse problem of term source identification. An optimal control formulation is proposed. Thus, an existence result of an optimal solution is established. To solve the obtained optimization problem an approach based on the reproducing kernel Hilbert spaces is proposed. The proposed optimization method based on the gradient descent algorithm is examined through some numerical experiments. The effectiveness of the proposed approach is proven by the presented results. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.

关键词： optimization algorithms

The Distributionally Robust optimization Model of Sparse Principal Component Analysis

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Lei Liu, Xin Chen, Xiaojun Department of Applied Mathematics The Hong Kong Polytechnic University Hong Kong State Key Laboratory of Scientific and Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences University of Chinese Academy of Sciences Beijing China

We consider sparse principal component analysis (PCA) under a stochastic setting where the underlying probability distribution of the random parameter is uncertain. This problem is formulated as a distributionally robust optimization (DRO) model based on a constructive approach to capturing uncertainty in the covariance matrix, which constitutes a nonsmooth constrained min-max optimization problem. We further prove that the inner maximization problem admits a closed-form solution, reformulating the original DRO model into an equivalent minimization problem on the Stiefel manifold. This transformation leads to a Riemannian optimization problem with intricate nonsmooth terms, a challenging formulation beyond the reach of existing algorithms. To address this issue, we devise an efficient smoothing manifold proximal gradient algorithm. We prove the Riemannian gradient consistency and global convergence of our algorithm to a stationary point of the nonsmooth minimization problem. Moreover, we establish the iteration complexity of our algorithm. Finally, numerical experiments are conducted to validate the effectiveness and scalability of our algorithm, as well as to highlight the necessity and rationality of adopting the DRO model for sparse PCA. Copyright © 2025, The Authors. All rights reserved.

关键词： optimization algorithms

A survey on combinatorial optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Le, Phuong

This survey revisits classical combinatorial optimization algorithms and extends them to two-stage stochastic models, particularly focusing on client-element problems. We reformulate these problems to optimize element selection under uncertainty and present two key sampling algorithms: SSA and Boost-and-Sample, highlighting their performance guarantees. Additionally, we explore correlation-robust optimization, introducing the concept of the correlation gap, which enables approximations using independent distributions with minimal accuracy loss. This survey analyzes and presents foundational combinatorial optimization methods for researchers at the intersection of this field and reinforcement learning. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms

LPBSA: Enhancing optimization Efficiency through Learner Performance-based Behavior and Simulated Annealing

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hamad, Dana Rasul Rashid, Tarik A. Computer Science Department Faculty of Science Soran University Erbil Soran Iraq Computer Science and Engineering Department University of Kurdistan Hewler Erbil Iraq

This study introduces the LPBSA, an advanced optimization algorithm that combines Learner Performance-based Behavior (LPB) and Simulated Annealing (SA) in a hybrid approach. Emphasizing metaheuristics, the LPBSA addresses and mitigates the challenges associated with traditional LPB methodologies, enhancing convergence, robustness, and adaptability in solving complex optimization problems. Through extensive evaluations using benchmark test functions, the LPBSA demonstrates superior performance compared to LPB and competes favorably with established algorithms such as PSO, FDO, LEO, and GA. Real-world applications underscore the algorithm's promise, with LPBSA outperforming the LEO algorithm in two tested scenarios. Based on the study results many test function results such as TF5 by recording (4.76762333) and some other test functions provided in the result section prove that LPBSA outperforms popular algorithms. This research highlights the efficacy of a hybrid approach in the ongoing evolution of optimization algorithms, showcasing the LPBSA's capacity to navigate diverse optimization landscapes and contribute significantly to addressing intricate optimization challenges. Copyright © 2024, The Authors. All rights reserved.

关键词： optimization algorithms