检索结果-内蒙古大学图书馆

Mastering uncertainty in performance estimations of configurable software systems

EMPIRICAL software ENGINEERING 2023年第2期28卷 1-40页

作者： Dorn, Johannes Apel, Sven Siegmund, Norbert Univ Leipzig D-04109 Leipzig Germany Saarland Univ D-60041 Saarbrucken Germany Saarland Informat Campus D-60041 Saarbrucken Germany

Understanding the influence of configuration options on the performance of a software system is key for finding optimal system configurations, system understanding, and performance debugging. In the literature, a number of performance-influence modeling approaches have been proposed, which model a configuration option's influence and a configuration's performance as a scalar value. However, these point estimates falsely imply a certainty regarding an option's influence that neglects several sources of uncertainty within the assessment process, such as (1) measurement bias, choices of model representation and learning process, and incomplete data. This leads to the situation that different approaches and even different learning runs assign different scalar performance values to options and interactions among them. The true influence is uncertain, though. There is no way to quantify this uncertainty with state-of-the-art performance modeling *** propose a novel approach, P4, which is based on probabilistic programming, that explicitly models uncertainty for option influences and consequently provides a confidence interval for each prediction alongside a scalar. This way, we can explain, for the first time, why predictions may be erroneous and which option's influence may be unreliable. An evaluation on 13 real-world subject systems shows that P4's accuracy is in line with the state of the art while providing reliable confidence intervals, in addition to scalar predictions. We qualitatively explain how uncertain influences of individual options and interactions cause inaccurate predictions.

关键词： Probabilistic programming Performance-influence modeling configurable software systems Bayesian inference P4

来源：评论

学校读者我要写书评

暂无评论

Challenges and Insights from Optimizing configurable software systems 19

Challenges and Insights from Optimizing Configurable Softwar...

引用

13th International Workshop on Variability Modelling of software-Intensive systems (VAMOS)

作者： Siegmund, Norbert Bauhaus Univ Weimar Weimar Germany

ISBN: (纸本)9781450366489

Configuring a software system to optimize non-functional properties is a hard task. There are dozens to thousands of configuration options that can affect performance, energy consumption, and other attributes of the resulting program. Even worse, options may interact, such that their combined presence (or absence) has an influence on a non-functional property. In this talk, I report on our experiences with learning different performance models based on a multitude of sampling techniques. The goal is to raise awareness of the distinct challenges in this domain: constraints among options, the exponential search space, and suitable sampling and learning techniques. I show a variety of approaches including their strengths and weaknesses and close the talk with new challenges relevant for our community: changing environments and reproducibility.

关键词： configurable software systems Performance Sampling Learning Prediction

来源：评论

学校读者我要写书评

暂无评论

Mastering Uncertainty in Performance Estimations of configurable software systems 35

Mastering Uncertainty in Performance Estimations of Configur...

引用

35th IEEE/ACM International Conference on Automated software Engineering (ASE)

作者： Dorn, Johannes Apel, Sven Siegmund, Norbert Univ Leipzig Leipzig Germany Saarland Univ Saarbrucken Germany

ISBN: (纸本)9781450367684

Understanding the influence of configuration options on performance is key for finding optimal system configurations, system understanding, and performance debugging. In prior research, a number of performance-influence modeling approaches have been proposed, which model a configuration option's influence and a configuration's performance as a scalar value. However, these point estimates falsely imply a certainty regarding an option's influence that neglects several sources of uncertainty within the assessment process, such as (1) measurement bias, (2) model representation and learning process, and (3) incomplete data. This leads to the situation that different approaches and even different learning runs assign different scalar performance values to options and interactions among them. The true influence is uncertain, though. There is no way to quantify this uncertainty with state-of-the-art performance modeling approaches. We propose a novel approach, P4, based on probabilistic programming that explicitly models uncertainty for option influences and consequently provides a confidence interval for each prediction of a configuration's performance alongside a scalar. This way, we can explain, for the first time, why predictions may cause errors and which option's influences may be unreliable. An evaluation on 12 real-world subject systems shows that P4's accuracy is in line with the state of the art while providing reliable confidence intervals, in addition to scalar predictions.

关键词： Probabilistic Programming Performance-Influence Modeling configurable software systems P4

来源：评论

学校读者我要写书评

暂无评论

Performance evolution of configurable software systems: an empirical study

引用

EMPIRICAL software ENGINEERING 2023年第6期28卷 152-152页

作者： Kaltenecker, Christian Muehlbauer, Stefan Grebhahn, Alexander Siegmund, Norbert Apel, Sven Saarland Univ Saarland Informat Campus Saarbrucken Germany Univ Leipzig Leipzig Germany Adesso SE Berlin Germany

As a software system evolves, its performance can improve or degrade over time. Performance evolution is especially delicate in configurable software systems, where performance degradation may manifest only for specific configurations, making it especially hard to spot and fix. Problem. Prior work concentrated mainly on performance-bug detection and root-cause analysis of a single version of a system. The big picture of how performance co-evolves with a system and what role configurability plays is largely unclear. Approach. In an empirical study, we investigate the relation between configurability and performance evolution. Specifically, we analyze a total of 190 releases of 12 configurable real-world systems and examine the extent to which performance changes are specific to particular configurations and whether few or many configuration options cause performance changes. We triangulate our findings by analyzing change logs and commit messages of the respective projects to pin down causes of performance changes. Results. We found that almost every release of every subject system exhibits performance changes in some of their configurations. Notably, the majority of performance changes affects only a subset of the configuration space, and most performance changes are triggered by multiple options (up to 6). In a deeper analysis, we found that a considerable number of releases mention performance changes in the change log and commits: performance changes are reported in 45% 69\% of the releases in the change log and the commit messages, respectively, but only a fraction report the involved configuration options.

关键词： configurable software systems Performance Evolution Performance changes

来源：评论

学校读者我要写书评

暂无评论

A Study of Performance Testing in configurable software systems

引用

Journal of software Engineering and Applications 2021年第9期14卷 474-492页

作者： Xue Han University of Southern Indiana Evansville Indiana

Customizing applications through program configuration options has been proved by many open-source and commercial projects as one of the best practices in software engineering. However, traditional performance testing is not in synch with this industrial practice. Traditional performance testing techniques consider program inputs as the only external factor. It ignores the performance influence of configuration options. This study aims to stimulate research interest in performance testing in the context of configurable software systems by answering three research questions. That is, why it is necessary to conduct research in performance testing, what are the state-of-the-art techniques, and how do we conduct performance testing research in configurable software systems. In this study, we examine the unique characteristics and challenges of performance testing research in configurable software systems. We review and discuss research topics on the performance bug study, performance anti-patterns, program analysis, and performance testing. We share the research findings from the empirical study and outline the opening opportunities for new and advanced researchers to contribute to the research community.

关键词： configurable software systems Performance Testing software Configuration Performance Bug Study

来源：评论

学校读者我要写书评

暂无评论

Mastering uncertainty in performance estimations of configurable software systems 20

Mastering uncertainty in performance estimations of configur...

引用

Proceedings of the 35th IEEE/ACM International Conference on Automated software Engineering

作者： Johannes Dorn Sven Apel Norbert Siegmund Leipzig University Saarland University

ISBN: (纸本)9781450367684

关键词： probabilistic programming performance-influence modeling configurable software systems P4

来源：评论

学校读者我要写书评

暂无评论

Dividable Configuration Performance Learning

引用

IEEE TRANSACTIONS ON software ENGINEERING 2025年第1期51卷 106-134页

作者： Gong, Jingzhi Chen, Tao Bahsoon, Rami Univ Elect Sci & Technol China Dept Comp Sci & Engn Chengdu 610056 Peoples R China Loughborough Univ Dept Comp Sci Loughborough LE11 3TU England Univ Birmingham Sch Comp Sci Birmingham B15 2TT England

Machine/deep learning models have been widely adopted to predict the configuration performance of software systems. However, a crucial yet unaddressed challenge is how to cater for the sparsity inherited from the configuration landscape: the influence of configuration options (features) and the distribution of data samples are highly sparse. In this paper, we propose a model-agnostic and sparsity-robust framework for predicting configuration performance, dubbed DaL , based on the new paradigm of dividable learning that builds a model via "divide-and-learn". To handle sample sparsity, the samples from the configuration landscape are divided into distant divisions, for each of which we build a sparse local model, e.g., regularized Hierarchical Interaction Neural Network, to deal with the feature sparsity. A newly given configuration would then be assigned to the right model of division for the final prediction. Further, DaL adaptively determines the optimal number of divisions required for a system and sample size without any extra training or profiling. Experiment results from 12 real-world systems and five sets of training data reveal that, compared with the state-of-the-art approaches, DaL performs no worse than the best counterpart on 44 out of 60 cases (within which 31 cases are significantly better) with up to 1.61x improvement on accuracy;requires fewer samples to reach the same/better accuracy;and producing acceptable training overhead. In particular, the mechanism that adapted the parameter d can reach the optimal value for 76.43% of the individual runs. The result also confirms that the paradigm of dividable learning is more suitable than other similar paradigms such as ensemble learning for predicting configuration performance. Practically, DaL considerably improves different global models when using them as the underlying local models, which further strengthens its flexibility. To promote open science, all the data, code, and supplementary materials of thi

关键词： Accuracy Data models Training software systems Predictive models Adaptation models Tuning Training data software measurement Runtime performance engineering configurable software systems configuration learning performance modeling performance prediction software configuration

来源：评论

学校读者我要写书评

暂无评论

Fault tolerant software systems using software configurations for cloud computing

引用

JOURNAL OF CLOUD COMPUTING-ADVANCES systems AND APPLICATIONS 2018年第1期7卷 1-17页

作者： Chinnaiah, Mylara Reddy Niranjan, Nalini REVA Univ Sch Comp & Informat Technol Rukmini Knowledge Pk Bangalore 560064 Karnataka India Nitte Meenakshi Inst Technol Dept Comp Sci & Engn Bangalore 560064 Karnataka India

Customizable software systems consist of a large number of different, critical, non-critical and interdependent configurations. Reliability and performance of configurable system depend on successful completion of communication or interactions among its configurations. Most of the time users of configurable systems very often use critical configurations than non-critical configurations. Failure of critical configurations will have severe impact on system reliability and performance. We can overcome this problem by identifying critical configurations that play a vital role, then provide a suitable fault tolerant candidate to each critical configuration. In this article we have proposed an algorithm that identifies optimal fault tolerant candidate for every critical configuration of a software system. We have also proposed two schemes to classify configurations into critical and non-critical configurations based on: 1) Frequency of configuration interactions (IFrFT), 2) Characteristics and frequency of interactions (ChIFrFT). These schemes have played very important role in achieving reliability and fault tolerance of a software system in a cost effective manner. The percentage of successful interactions of IFrFT and ChIFrFT are 25 and 40% higher than that of the NoFT scheme. In NoFT scheme none of the configurations are supported by fault tolerance candidates. Performance of IFrFT, ChIFrFT, and NoFT schemes are tested using a file structure system.

关键词： configurable software systems Fault tolerance Reliability Configurations interactions

来源：评论

学校读者我要写书评

暂无评论

IoPV: On Inconsistent Option Performance Variations 2023

IoPV: On Inconsistent Option Performance Variations

引用

31st ACM Joint Meeting of the European software Engineering Conference / Symposium on the Foundations-of-software-Engineering (ESEC/FSE)

作者： Chen, Jinfu Ding, Zishuo Tang, Yiming Sayagh, Mohammed Li, Heng Adams, Bram Shang, Weiyi Wuhan Univ Wuhan Peoples R China Univ Waterloo Waterloo ON Canada Rochester Inst Technol Rochester MN USA ETS Montreal PQ Canada Polytech Montreal Montreal PQ Canada Queens Univ Kingston ON Canada

ISBN: (纸本)9798400703270

Maintaining a good performance of a software system is a primordial task when evolving a software system. The performance regression issues are among the dominant problems that large software systems face. In addition, these large systems tend to be highly configurable, which allows users to change the behaviour of these systems by simply altering the values of certain configuration options. However, such flexibility comes with a cost. Such software systems suffer throughout their evolution from what we refer to as "Inconsistent Option Performance Variation" (IoPV). An IoPV indicates, for a given commit, that the performance regression or improvement of different values of the same configuration option is inconsistent compared to the prior commit. For instance, a new change might not suffer from any performance regression under the default configuration (i.e., when all the options are set to their default values), while altering one option's value manifests a regression, which we refer to as a hidden regression as it is not manifested under the default configuration. Similarly, when developers improve the performance of their systems, performance regression might be manifested under a subset of the existing configurations. Unfortunately, such hidden regressions are harmful as they can go unseen to the production environment. In this paper, we first quantify how prevalent (in)consistent performance regression or improvement is among the values of an option. In particular, we study over 803 Hadoop and 502 Cassandra commits, for which we execute a total of 4,902 and 4,197 tests, respectively, amounting to 12,536 machine hours of testing. We observe that IoPV is a common problem that is difficult to manually predict. 69% and 93% of the Hadoop and Cassandra commits have at least one configuration that hides a performance regression. Worse, most of the commits have different options or tests leading to IoPV and hiding performance regressions. Therefore, we propose a predi

关键词： software performance Performance variation configurable software systems

来源：评论

学校读者我要写书评

暂无评论

Comprehensibility of Heterogeneous Configuration Knowledge: A User Study 17

Comprehensibility of Heterogeneous Configuration Knowledge: ...

引用

31st Brazilian Symposium on software Engineering (SBES)

作者： Cirilo, Elder Cafeo, Bruno Nunes, Ingrid Garcia, Alessandro Kulesza, Uira Lucena, Carlos Univ Fed Sao Joao del Rei Sao Joao Del Rei MG Brazil Univ Fed Mato Grosso do Sul Campo Grande MS Brazil Univ Fed Rio Grande do Sul Porto Alegre RS Brazil TU Dortmund Dortmund Germany Pontif Catholic Univ Rio de Janeiro Rio De Janeiro Brazil Univ Fed Rio Grande do Norte Natal RN Brazil

ISBN: (纸本)9781450353267

An essential part of configurable software systems is the configuration knowledge (CK), which is used by developers to derive customized products. Proper comprehension of the CK is key to achieve correct product customization. However, each configurable system is often built with heterogeneous software technologies, such as diverse types of frameworks. These technologies impose the use of additional concepts on the definition of the CK, thereby making it harder to comprehend. This paper presents a domain specific modeling technique aimed at improving the comprehensibility of the CK in framework-based configurable systems. We also present the results of a user study that compares domain-specific modeling with two other state-of-the-art CK techniques, namely annotation-based and general-purpose modeling. We analyzed the impact of these techniques on the CK comprehensibility across three configurable systems. The results strongly suggest that: (i) domain-specific models improve the CK comprehensibility;(ii) the use of general purpose models imposes significant obstacles to the CK comprehension;and (iii) participants who worked with domain-specific models rated the comprehension tasks as easier than those that worked without these models.

关键词： configurable software systems Configuration Knowledge

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：