On the Representativeness Metric of Benchmark Problems in Numerical Optimization

作     者：Chen, Caifeng Liu, Qunfeng Jing, Yunpeng Zhang, Mingming Cheng, Shi Li, Yun 

作者机构：School of Modern Information Industry Guangzhou College of Commerce Guangzhou511363 China School of Computer Science and Technology Dongguan University of Technology Dongguan523808 China Department of Mechanical and Mechatronics Engineering University of Auckland New Zealand School of Computer Science Shaanxi Normal University Xi’an710062 China Shenzhen Institute for Advanced Study University of Electronic Science and Technology of China Shenzhen518110 China i4AI Ltd LondonWCIN3AX United Kingdom 

出 版 物：《SSRN》 

年 卷 期：2024年

核心收录：

主　　题：Optimization 

摘      要：Numerical comparison on benchmark problems is often necessary in evaluating optimization algorithms with or without theoretical analysis. An implicit assumption is that the adopted set of benchmark problems is representative. However, to our knowledge, there are few results about how to evaluate the representativeness of a test suite, partly due to the difficulty of this issue. In this paper, we first define three different levels of representativeness, and open up a window for addressing step by step the issue of representativeness-measuring. Then we turn to address the type-III representativeness-measuring problem, and provide a methodology and metric for this problem. To illustrate how to use the proposed metric, the representativeness-measuring problem of benchmark problems for single-objective optimization is *** analysis covers as many as 1141 single-objective unconstrained continuous benchmark problems, primarily focusing on existing benchmark problems. Based on the defined the representativeness metric, some classical features and calculations are used to assess the representativeness of the benchmark problems. Assessment results show that most of benchmark problems of high representativeness are non-separable problems from the CEC and BBOB test suites. We select the top $5\%$ of most representative problems to build a new test suite, providing a more representative and rigorous reference in verifying the overall performance of the optimization algorithms. © 2024, The Authors. All rights reserved.