检索结果-内蒙古大学图书馆

Investigating faults missed by test suites achieving high code coverage

JOURNAL OF SYSTEMS AND SOFTWARE 2018年 144卷 106-120页

作者： Schwartz, Amanda Puckett, Daniel Meng, Ying Gay, Gregory Univ South Carolina Upstate Spartanburg SC 29303 USA Univ South Carolina Columbia SC USA

code coverage criteria are commonly used to determine the adequacy of a test suite. However, studies investigating code coverage and fault-finding capabilities have mixed results. Some studies have shown that creating test suites to satisfy coverage criteria has a positive effect on finding faults, while other studies do not. In order to improve the fault-finding capabilities of test suites, it is essential to understand what is causing these mixed results. In this study, we investigated one possible source of variation in the results observed: fault type. Specifically, we studied 45 different types of faults and evaluated how effectively human-created test suites with high coverage percentages were able to detect each type of fault. Our results showed, with statistical significance, there were specific types of faults found less frequently than others. However, improvements in the formulation and selection of test oracles could overcome these weaknesses. Based on our results and the types of faults that were missed, we suggest focusing on the strength of test oracles along with code coverage to improve the effectiveness of test suites.

关键词： code coverage Automated testing Software testing Test suite effectiveness

来源：评论

学校读者我要写书评

暂无评论

DRC code coverage Test A Novel QA methodology

DRC Code Coverage Test A Novel QA methodology

引用

International Conference on IC Design and Technology (ICICDT)

作者： Lee, Yongseok Park, Jaehong Elsayed, Mohamed A. Gadallah, Maged E. Alimam, Mohamed Samsung Elect Co Ltd Design Enablement Dept Hwa Seong Keonggi Do South Korea Samsung Elect Co Ltd ASIC & IP Dept Hwa Seong Keonggi Do South Korea Mentor Cairo Egypt Mentor Tokyo Japan

ISBN: (纸本)9781538625491

With continuous development in the IC advanced technologies, physical verification is becoming increasingly more complex in the semiconductor manufacturing process. Currently, different regression patterns-based techniques are widely adopted in the design rules checking (DRC) verification. As complexity of design rules increases, limitations of those traditional techniques emerge such as focusing only on the rule final output, missing some of the underlying defects in the DRC code, requiring many QA cycles for the rule deck to be released, and lacking well-defined measure of the DRC coverage. This paper presents a new methodology in verifying the design rules checks, complementing the traditional Regression technique, through performing DRC rule deck line-by-line comparison with the regression layout patterns. This offers in-depth and quantitative analysis of all the DRC components, captures the possible defects in very early stages of the QA process, and reduces the time consumed in verification significantly compared to using just the traditional QA techniques.

关键词： code coverage patterns-based QA Regression Design rule checking physical verification TCL

来源：评论

学校读者我要写书评

暂无评论

code coverage and Postrelease Defects: A Large-Scale Study on Open Source Projects

引用

IEEE TRANSACTIONS ON RELIABILITY 2017年第4期66卷 1213-1228页

作者： Kochhar, Pavneet Singh Lo, David Lawall, Julia Nagappan, Nachiappan Singapore Management Univ Sch Informat Syst Singapore 188065 Singapore INRIA F-75012 Paris France Microsft Res Redmond WA 98052 USA

Testing is a pivotal activity in ensuring the quality of software. code coverage is a common metric used as a yardstick to measure the efficacy and adequacy of testing. However, does higher coverage actually lead to a decline in postrelease bugs? Do files that have higher test coverage actually have fewer bug reports? The direct relationship between code coverage and actual bug reports has not yet been analyzed via a comprehensive empirical study on real bugs. Past studies only involve a few software systems or artificially injected bugs (mutants). In this empirical study, we examine these questions in the context of open-source software projects based on their actual reported bugs. We analyze 100 large open-source Java projects and measure the code coverage of the test cases that come along with these projects. We collect real bugs logged in the issue tracking system after the release of the software and analyze the correlations between code coverage and these bugs. We also collect other metrics such as cyclomatic complexity and lines of code, which are used to normalize the number of bugs and coverage to correlate with other metrics as well as use thesemetrics in regression analysis. Our results showthat coverage has an insignificant correlation with the number of bugs that are found after the release of the software at the project level, and no such correlation at the file level.

关键词： code coverage empirical study open-source postrelease defects software testing

来源：评论

学校读者我要写书评

暂无评论

The Influence of code coverage Metrics on Automated Testing Efficiency in Android 18

The Influence of Code Coverage Metrics on Automated Testing ...

引用

ACM SIGSAC Conference on Computer and Communications Security (CCS)

作者： Dashevskyi, Stanislav Gadyatskaya, Olga Pilgun, Aleksandr Zhauniarovich, Yury Univ Luxembourg SnT Luxembourg Luxembourg HBKU Qatar Comp Res Inst Doha Qatar

ISBN: (纸本)9781450356930

code coverage is an important metric that is used by automated Android testing and security analysis tools to guide the exploration of applications and to assess efficacy. Yet, there are many different variants of this metric and there is no agreement within the Android community on which are the best to work with. In this paper, we report on our preliminary study using the state-of-the-art automated test design tool Sapienz. Our results suggest a viable hypothesis that combining different granularities of code coverage metrics can be beneficial for achieving better results in automated testing of Android applications.

关键词： android app automated testing code coverage dynamic analysis fault detection

来源：评论

学校读者我要写书评

暂无评论

A Novel Assertions-Based code coverage Automatic CAD Tool 17

A Novel Assertions-Based Code Coverage Automatic CAD Tool

引用

IEEE EUROCON -17th International Conference on Smart Technologies

作者： El-Kharashy, Hatem Khamis, Mostafa Salah, Amr Korany, Mohammed Alexandria Univ Fac Engn Alexandria Egypt Mentor Graph Corp Cairo Egypt

ISBN: (纸本)9781509038435

Justified and complete verification for hardware designs is a critical step as industrial hardware designs becomes more and more complex. Moreover, one of the most important aspects of verification is coverage which monitors during simulation which design specification, features and also RTL code area is being covered. Following how important coverage is, most hardware verification tools provide different metrics for measuring coverage. code coverage one of the most used metrics in hardware and even software simulation is responsible for checking which blocks of code is being exercised. However, as tools becoming more and more complex and provide large amount of features sometimes it becomes harder for quality assurance in tool testing to monitor and track the code coverage unit functionality in their tool. Also, some tools may not provide a mean to measure code coverage at all. This paper presents a novel method based on SystemVerilog assertions (SVA) to measure code coverage for hardware designs, justified and tested with a novel tool that can help QA engineers in debugging their code coverage units. Additionally, this tool can act as a standalone tool for measuring code coverage if the simulator does not naturally support it.

关键词： Assertion Based verification code coverage Concurrent assertions Immediate assertions Unit testing PERL

来源：评论

学校读者我要写书评

暂无评论

An Effective Android code coverage Tool 18

An Effective Android Code Coverage Tool

引用

Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security

作者： Aleksandr Pilgun Olga Gadyatskaya Stanislav Dashevskyi Yury Zhauniarovich Artsiom Kushniarou University of Luxembourg Esch-sur-Alzette Luxembourg Qatar Computing Research Institute Doha Qatar

ISBN: (纸本)9781450356930

The deluge of Android apps from third-party developers calls for sophisticated security testing and analysis techniques to inspect suspicious apps without accessing their source code. code coverage is an important metric used in these techniques to evaluate their effectiveness, and even as a fitness function to help achieving better results in evolutionary and fuzzy approaches. Yet, so far there are no reliable tools for measuring fine-grained bytecode coverage of Android apps. In this work we present ACVTool that instruments Android apps and measures the smali code coverage at the level of classes, methods, and instructions. Tool repository: https://***/pilgun/acvtool

关键词： android instrumentation smali automated testing black box code coverage dynamic analysis

来源：评论

学校读者我要写书评

暂无评论

Estimation of the Total Number of Software Failures from Test Data and code coverage 28

Estimation of the Total Number of Software Failures from Tes...

引用

28th IEEE International Symposium on Software Reliability Engineering

作者： Stieber, Harald A. Hu, Linghuan Wong, W. Eric Nuremberg Inst Technol Georg Simon Ohm Nurnberg Germany Univ Texas Dallas Dept Comp Sci Dallas TX USA

ISBN: (纸本)9781538623879

Total number of failures of a software system can help practitioners to have a better understanding of the software quality. In this paper, we propose a model to predict the total number of software failures in a software system by analyzing the failure data from testing using models based on Zipf's law together with the information on code coverage. Failure data and code coverage are combined in a Bayesian way. The methodology is applied to real world failure data to validate its predictability. The predictive accuracy of our model is also evaluated with different methods. The results of our experiment show that our proposed model can provide a very good estimation of the total number of failures. The estimation is stable from a very early point on.

关键词： software testing software reliability models code coverage Bayesian parameter estimation confidence interval Zipf's law

来源：评论

学校读者我要写书评

暂无评论

Evaluating an Automatic Text-based Test Case Selection using a Non-Instrumented code coverage Analysis 2

Evaluating an Automatic Text-based Test Case Selection using...

引用

2nd Brazilian Symposium on Systematic and Automated Software Testing (SAST)

作者： Magalhaes, Claudio Andrade, Joao Perrusi, Lucas Mota, Alexandre Ctr Informat Recife PE Brazil

ISBN: (纸本)9781450353021

During development, systems may be tested several times. In general, a system evolves from change requests, aiming at improving its behavior in terms of new features as well as fi xing failures. Thus, selecting the best test plan in terms of the closeness between test cases and the changed code and its dependencies is pursued by industry and academia. In this paper we measure the coverage achieved by an automatic test case selection based on information retrieval that relates change requests and test cases. But instead of using off-the-shelf coverage tools, like JaCoCo, we propose a way of obtaining code coverage of Android apk's without instrumentation. This was a basic requirement of our industrial partner. We performed some experiments on this industrial partner and promising results were obtained.

关键词： Information Retrieval Test case selection and prioritization code coverage

来源：评论

学校读者我要写书评

暂无评论

Traffic diversity and code coverage: a preliminary analysis

引用

INTERNATIONAL JOURNAL OF COMMUNICATION SYSTEMS 2016年第3期29卷 428-440页

作者： Lin, Ying-Dar Lai, Yuan-Cheng Lu, Chun-Nan Hung, Jui-Tsun Shao, Chun-Pin Natl Chiao Tung Univ Dept Comp Sci Hsinchu Taiwan Natl Taiwan Univ Sci & Technol Dept Informat Management Taipei Taiwan

It is generally assumed that using more diverse traffic to test network devices could achieve larger code coverage. However, how to describe the diversity of traffic traces and the relationship between the traffic diversity and code coverage is still an issue. In this paper, the traffic diversity is defined using the number of packets and the size of the subnets involved, and traces having various diversity are used to evaluate the corresponding code coverage for the programs in a network device. Experiment results show that more number of packets or larger size of network segments can generate larger diversity indices and thus larger code coverage. For Snort, as the number of packets increases from 1 to 10,000,000, representative diversity index and the code coverage can increase from 0 to 0.95 on the basis of Simpson's index and from 19.1% to 32.2%, respectively. As the size of network segments increases, representative diversity index and the code coverage can increase from 0.41 to 0.82 and from 28.2% to 32.2%, respectively. Similar results can be obtained in the case of Linux kernel. If the mappings among different diversity indices and the corresponding code coverage can be built beforehand, the quality of the tests can improved. Copyright (c) 2014 John Wiley & Sons, Ltd.

关键词： traffic diversity code coverage diversity index network test

来源：评论

学校读者我要写书评

暂无评论

The Impact of Fault Type on the Relationship Between code coverage and Fault Detection 11

The Impact of Fault Type on the Relationship Between Code Co...

引用

11th IEEE/ACM International Workshop in Automation of Software Test (AST)

作者： Schwartz, Amanda Hetzel, Michael Univ South Carolina Upstate Spartanburg SC 29303 USA

ISBN: (纸本)9781450341516

Structural coverage criteria are commonly used to determine the adequacy of a test suite. However, studies investigating structural coverage and fault-finding capabilities have mixed results. Some studies have shown generating test suites to satisfy structural coverage criteria has a positive effect on finding faults, while other studies show the opposite. These mixed results indicate there are factors not yet known that affect the ability of test suites satisfying structural coverage criteria to find faults. In order to improve the fault-finding capabilities of test suites, it is essential to understand what factors are causing this variance. Unfortunately very little work has been done to investigate the variance observed in the relationship between structural coverage criteria and fault-finding capabilities. In this paper, we investigate one possible source of variation in the results observed: fault type. We provide an empirical study which narrows down the focus of the relationship between structural coverage and fault-finding capabilities by focusing on object-oriented bugs. Specifically, we investigated 26 different types of object-oriented faults and evaluated how effectively test suites with high coverage percentages were able to detect each type of fault. We found that a test suite's ability to find faults varied significantly according to the type of fault (ranging from a rate of 0% to 87.5% mutants detected per fault type). We also found there are particular types of faults that were consistently found less frequently across all object programs.

关键词： Software testing code coverage test suite effectiveness

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：