How experimental algorithmics can benefit from Mayo's extensions to Neyman-Pearson theory of testing

作     者：Bartz-Beielstein, Thomas 

作者机构：Cologne Univ Appl Sci Fac Comp Sci & Engn Sci D-51643 Gummersbach Germany 

出 版 物：《SYNTHESE》 

年 卷 期：2008年第163卷第3期

页      面：385-396页

核心收录：

学科分类：01[哲学] 0101[哲学-哲学] 0712[理学-科学技术史(分学科，可授理学、工学、农学、医学学位)] 

主　　题：new experimentalism experimental algorithmics optimization theory of testing Mayo's learning model significance 

摘      要：Although theoretical results for several algorithms in many application domains were presented during the last decades, not all algorithms can be analyzed fully theoretically. Experimentation is necessary. The analysis of algorithms should follow the same principles and standards of other empirical sciences. This article focuses on stochastic search algorithms, such as evolutionary algorithms or particle swarm optimization. Stochastic search algorithms tackle hard real-world optimization problems, e.g., problems from chemical engineering, airfoil optimization, or bio-informatics, where classical methods from mathematical optimization fail. Nowadays statistical tools that are able to cope with problems like small sample sizes, non-normal distributions, noisy results, etc. are developed for the analysis of algorithms. Although there are adequate tools to discuss the statistical significance of experimental data, statistical significance is not scientifically meaningful per se. It is necessary to bridge the gap between the statistical significance of an experimental result and its scientific meaning. We will propose some ideas on how to accomplish this task based on Mayo s learning model (NPT*).