检索结果-内蒙古大学图书馆

Paradox-Free Data Analysis in Numerical Comparisons of optimization algorithms: A Review and Case Study

学校读者我要写书评

暂无评论

INTERNATIONAL JOURNAL OF SWARM INTELLIGENCE RESEARCH 2024年第1期15卷 1页

作者： Zhi, Yunchao Liu, Qunfeng Yan, Yuan Jing, Yunpeng Dongguan Univ Technol Dongguan Peoples R China Univ Elect Sci & Technol China Shenzhen Peoples R China

Numerical comparison is essential for evaluating an optimization algorithm. Unfortunately, recent research has shown that two paradoxes may occur, namely the cycle ranking paradox and survival of the nonfittest paradox. Further exploitation reveals that these paradoxes stem from the method of data analysis, especially its comparison strategy. Therefore, the design and widespread use of paradox-free data analysis methods has become urgent. This paper is dedicated to reviewing the recent progress in paradox in numerical comparisons of optimization algorithms, especially the reasons for paradoxes and the ways to eliminate them. Specifically, significant progress has been made in eliminating paradoxes from two popular data analysis methods, including hypothesis testing and methods based on the cumulative distribution function. Furthermore, in this paper the authors provide case studies, aiming to show that paradoxes are common and how to eliminate them with paradox-free data analysis methods.

关键词： Cycle Ranking Paradox Data Profile Hypothesis Testing Numerical Comparison optimization algorithms Survival of The Nonfittest Paradox

Evaluating Convolution Neural Network optimization algorithms for Classification of Cervical Cancer Macro Images

学校读者我要写书评

暂无评论

Evaluating Convolution Neural Network optimization Algorithm...

International Conference on Electronics, Computer and Computation (ICECCO)

作者： Suleiman Mustafa Mohammed Dauda National Agency for Science and Engineering Infrastructure Abuja Nigeria

ISBN: (数字)9781728151601

ISBN: (纸本)9781728151618

In this study, Convolution Neural Network (CNN)-based learning method, which is well-used in deep learning is applied to evaluate optimization algorithms for improved accuracy in cervix cancer detection. The convolutional layer in this study is made up of numerous convolution kernels which are used to compute different feature maps for representations of the inputs. As a result, the architecture of the model consisting of convolutional layers, pooling layers, and fully-connected layers is designed by stacking the network layers. The model is evaluated firstly by increasing the amount of image data through image augmentation. Furthermore, the hyper parameters for optimum performance are chosen. Finally, analysis is performed for Stochastic gradient descent (SGD), Root Mean Square Propagation (RMSprop) and Adaptive Moment Estimation (Adam) optimizers to determine which improves the networks performance for the classification of cervix cancer.

关键词： cancer convolutional neural nets estimation theory gradient methods image classification learning (artificial intelligence) mean square error methods medical image processing optimisation Mean square error estimation theory Medical image processing gradient methods Image classification Cancer of the Uterine Cervix image data optimization algorithms Taxonomy Convolution

A Novel Pareto-optimal Ranking Method for Comparing Multi-objective optimization algorithms

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ibrahim, Amin Bidgoli, Azam Asilian Rahnamayan, Shahryar Deb, Kalyanmoy Faculty of Business and IT Ontario Tech University 2000 Simcoe Street North OshawaONL1G 0C5 Canada Computer Science Wilfrid Laurier University 75 University Avenue WaterlooONN2L 3C5 Canada Department of Engineering Brock University 1812 Sir Isaac Brock Way St. CatharinesONL2S 3A1 Canada Department of Electrical and Computer Engineering Michigan State University East LansingMI48824 United States

As the interest in multi- and many-objective optimization algorithms grows, the performance comparison of these algorithms becomes increasingly important. A large number of performance indicators for multi-objective optimization algorithms have been introduced, each of which evaluates these algorithms based on a certain aspect. Therefore, assessing the quality of multi-objective results using multiple indicators is essential to guarantee that the evaluation considers all quality perspectives. This paper proposes a novel multi-metric comparison method to rank the performance of multi-/ many-objective optimization algorithms based on a set of performance indicators. We utilize the Pareto optimality concept (i.e., non-dominated sorting algorithm) to create the rank levels of algorithms by simultaneously considering multiple performance indicators as criteria/objectives. As a result, four different techniques are proposed to rank algorithms based on their contribution at each Pareto level. This method allows researchers to utilize a set of existing/newly developed performance metrics to adequately assess/rank multi-/many-objective algorithms. The proposed methods are scalable and can accommodate in its comprehensive scheme any newly introduced metric. The method was applied to rank 10 competing algorithms in the 2018 CEC competition solving 15 many-objective test problems. The Pareto-optimal ranking was conducted based on 10 well-known multi-objective performance indicators and the results were compared to the final ranks reported by the competition, which were based on the inverted generational distance (IGD) and hypervolume indicator (HV) measures. The techniques suggested in this paper have broad applications in science and engineering, particularly in areas where multiple metrics are used for comparisons. Examples include machine learning and data mining. © 2024, CC BY.

关键词： optimization algorithms

Quantized Distributed Nonconvex optimization algorithms with Linear Convergence under the Polyak–Lojasiewicz Condition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Xu, Lei Yi, Xinlei Sun, Jiayue Shi, Yang Johansson, Karl H. Yang, Tao State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang110819 China School of Electrical Engineering and Computer Science KTH Royal Institute of Technology and Digital Futures Stockholm10044 Sweden Department of Mechanical Engineering University of Victoria VictoriaBCV8W 2Y2 Canada

This paper considers distributed optimization for minimizing the average of local nonconvex cost functions, by using local information exchange over undirected communication networks. To reduce the required communication capacity, we introduce an encoder–decoder scheme. By integrating them with distributed gradient tracking and proportional integral algorithms, respectively, we then propose two quantized distributed nonconvex optimization algorithms. Assuming the global cost function satisfies the Polyak–Lojasiewicz condition, which does not require the global cost function to be convex and the global minimizer is not necessarily unique, we show that our proposed algorithms linearly converge to a global optimal point and that larger quantization level leads to faster convergence speed. Moreover, we show that a low data rate is sufficient to guarantee linear convergence when the algorithm parameters are properly chosen. The theoretical results are illustrated by numerical examples. Copyright © 2022, The Authors. All rights reserved.

关键词： optimization algorithms

Enhancing Convergence of optimization algorithms for Logistic Regression: A Breast Cancer Diagnosis Application

学校读者我要写书评

暂无评论

Research Square

Research Square 2024年

作者： Borbor, Ariana Herrera, Priscilla Ashraf, Ahmed Delgado, Erwin Liut, Michael Suarez, Carlos Anibal Faculty of Natural Sciences and Mathematics Escuela Superior Politécnica del Litoral Campus Gustavo Galindo Km. 30.5 Vía Perimetral Guayas Guayaquil090902 Ecuador Department of Mathematical and Computational Sciences University of Toronto Mississauga 3359 Mississauga Road MississaugaONL5L 1C6 Canada

This paper explores strategies to enhance the convergence of Newton and Trust Region methods, which are employed to obtain optimal solutions for the optimization problems underlying the Logistic Regression technique. The optimization problems consist of maximizing the likelihood function and minimizing the sum of squared errors. These are addressed using the mentioned gradient-based algorithms. Improvements include using a local search heuristic to find a good initial solution and determining the optimal learning rate at each iteration. The heuristic-generated initial points significantly improve the convergence rate of these gradient-based methods. Additionally, instead of using a fixed learning rate or step size, the optimization process dynamically computes these parameters using the Golden Search method, ensuring a more adaptive and efficient search strategy. Extensive numerical experiments and a detailed case study validate the effectiveness of this approach. © 2024, CC BY.

关键词： optimization algorithms

Convergence of Descent optimization algorithms under Polyak-Lojasiewicz-Kurdyka Conditions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Bento, G.C. Mordukhovich, B.S. Mota, T.S. Nesterov, Yu. Federal University of Goiás Goiás Goiânia Brazil Wayne State University DetroitMI48202 United States Corvinus Centre for Operations Research Corvinus Institute for Advanced Studies Corvinus University of Budapest Hungary School of Data Sciences Chinese University of Hong Kong Shenzhen China

This paper develops a comprehensive convergence analysis for generic classes of descent algorithms in nonsmooth and nonconvex optimization under several conditions of the Polyak-Lojasiewicz-Kurdyka (PLK) type. Along other results, we prove the finite termination of generic algorithms under the PLK conditions with lower exponents. Specifications are given to establish new convergence rates for inexact reduced gradient methods and some versions of the boosted algorithm in DC programming. It is revealed, e.g., that the lower exponent PLK conditions for a broad class of difference programs are incompatible with the gradient Lipschitz continuity for the plus function around a local minimizer. On the other hand, we show that the above inconsistency observation may fail if the Lipschitz continuity is replaced by merely the gradient *** Codes 65K05, 65K10, 90C26, 47N10 © 2024, CC0.

关键词： optimization algorithms

Modeling of Missing Data Prediction: Computational Intelligence and optimization algorithms

学校读者我要写书评

暂无评论

Modeling of Missing Data Prediction: Computational Intellige...

IEEE International Conference on Systems, Man, and Cybernetics

作者： Collins Leke Bhekisipho Twala Tshilidzi Marwala Department of Electrical and Electronic Engineering Science University of Johannesburg

ISBN: (纸本)9781479938414

Four optimization algorithms (genetic algorithm, simulated annealing, particle swarm optimization and random forest) were applied with an MLP based auto associative neural network on two classification datasets and one prediction dataset. This work was undertaken to investigate the effectiveness of using auto associative neural networks and optimization algorithms in missing data prediction and classification tasks. If performed appropriately, computational intelligence and optimization algorithm systems could lead to consistent, accurate and trustworthy predictions and classifications resulting in more adequate decisions. The results reveal GA, SA and PSO to be more efficient when compared to RF in terms of predicting the forest area to be affected by fire. GA, SA, and PSO had the same accuracy of 93.3%, while RF showed 92.99% accuracy. For the classification problems, RF showed 93.66% and 92.11% accuracy on the German credit and Heart disease datasets respectively, outperforming GA, SA and PSO.

关键词： optimization algorithms Auto-associative neural networks Prediction Classification Missing data

On analysis of open optimization algorithms

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Eising, Jaap Dörfler, Florian ETH Zürich Switzerland

—We develop analysis results for optimization algorithms that are open, that is, with inputs and outputs. Such algorithms arise for instance, when analyzing the effect of noise or disturbance on an algorithm, or when an algorithm is part of control loop without timescale separation. To be precise, we consider an incremental small gain problem to analyze robustness. Moreover, we investigate the behaviors of the closed loop between incrementally dissipative nonlinear plants and optimization algorithms. The framework we develop is built upon the theories of incremental dissipativity and monotone operators, and yields tests in the form of linear matrix inequalities. © 2024, CC BY.

关键词： optimization algorithms

Optimal Restart Strategies For Parameter-Dependent optimization algorithms

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Schönenberger, Lisa Beyer, Hans-Georg Vorarlberg University of Applied Sciences Research Center Business Informatics Dornbirn6850 Austria

This paper examines restart strategies for algorithms whose performance and runtime depends on a parameter λ. After each restart, λ is incremented, until the algorithm terminates successfully. It is assumed that there is a unique, unknown, optimal value for λ. For the algorithm to run successfully, this value must be surpassed. The key question is whether there exists an optimal strategy for selecting λ after each restart taking into account that the computational costs (runtime) increases with λ. In this work, potential restart strategies are classified into parameter-dependent strategy types. A loss function is introduced to quantify the wasted computational cost relative to the optimal strategy. A crucial requirement for any efficient restart strategy is that its loss, relative to the optimal λ, remains bounded. To this end, upper and lower bounds of the loss are sought that can be expressed analytically. These bounds should be as sharp as possible. Using these bounds it will be shown that not all strategy types are bounded. However, for a particular strategy type, where λ is increased multiplicatively by a constant factor ρ, the relative loss function is bounded. Furthermore, it will be demonstrated that within this strategy type, there exists an optimal value for ρ that minimizes the maximum relative loss. In the asymptotic limit, this optimal choice of ρ does not depend on the unknown optimal λ. © 2024, The Authors. All rights reserved.

关键词： optimization algorithms