code coverage criteria are widely used in object-oriented (OO) domains as test quality indicators. However, these criteria are based on the procedural point of view, and therefore do not address the specific features ...
详细信息
code coverage criteria are widely used in object-oriented (OO) domains as test quality indicators. However, these criteria are based on the procedural point of view, and therefore do not address the specific features of OO programs. In this article, we extend the code coverage criteria and introduce a new set of criterion, called "object coveragecriteria," which cope with OO features like object instantiation, inheritance, polymorphism, and dynamic binding. Unlike previous criteria, the new criteria regard the actual type of the object under test and some inherited codes from the parent/ancestor classes that represent the object's states and behaviors. The new criteria have been implemented in a prototype tool called OCov4J for the Java language. Using this tool and conducting an empirical study on 270 classes (with about 50 k lines of code without blank lines and comments) from several large and widely used open source projects, we have found a considerable positive correlation between the object coverage level (defined via the new proposed criteria) and the number of detected specific OO failures. Not only do the proposed criteria provide ease of use, high automation, and low execution cost, but also they can effectively be applied to real-world OO programs.
This is the RCR report of the artifact for the article "Fine-grained coverage-based Fuzzing." This report contains scripts and pre-build binary programs to reproduce the results presented in themain article....
详细信息
This is the RCR report of the artifact for the article "Fine-grained coverage-based Fuzzing." This report contains scripts and pre-build binary programs to reproduce the results presented in themain article. The artifact is released on Zenodo with DOI: 10.5281/zenodo.7275184. We claim the artifact to be available, functional, and reusable. The technology skills needed to review the artifact are knowing how to use Linux/Unix terminal and a basic understanding of Docker.
Fuzzing is a popular software testing method that discovers bugs by massively feeding target applications with automatically generated inputs. Many state-of-the-art fuzzers use branch coverage as a feedback metric to ...
详细信息
Fuzzing is a popular software testing method that discovers bugs by massively feeding target applications with automatically generated inputs. Many state-of-the-art fuzzers use branch coverage as a feedback metric to guide the fuzzing process. The fuzzer retains inputs for further mutation only if branch coverage is increased. However, branch coverage only provides a shallow sampling of program behaviors and hence may discard interesting inputs to mutate. This work aims to take advantage of the large body of research in defining finer-grained codecoverage metrics (such as control-flow, data-flow, or mutation coverage) and to evaluate how fuzzing performance is impacted when using these metrics to select interesting inputs for mutation. We propose to make branch coverage-based fuzzers support most fine-grained coverage metrics out of the box (i.e., without changing fuzzer internals). We achieve this by making the test objectives defined by thesemetrics (such as conditions to activate or mutants to kill) explicit as new branches in the target program. Fuzzing such a modified target is then equivalent to fuzzing the original target, but the fuzzer will also retain inputs covering the additional metric objectives for mutation. In addition, all the fuzzer mechanisms to penetrate hard-to-cover branches will help in covering the additional metric objectives. We use this approach to evaluate the impact of supporting two fine-grained coverage metrics (multiple condition coverage and weak mutation) over the performance of two state-of-the-art fuzzers (AFL++ and QSYM) with the standard LAVA-M and MAGMA benchmarks. This evaluation suggests that our mechanism for runtime fuzzer guidance, where the fuzzed code is instrumented with additional branches, is effective and could be leveraged to encode guidance from human users or static analyzers. Our results also show that the impact of fine-grained metrics over fuzzing performance is hard to predict before fuzzing and most of the t
code coverage criteria define test objectives and provide actionable stopping conditions for creating test cases. They are also widely accepted as indicators of a test suite's fault detection effectiveness. Severa...
详细信息
ISBN:
(纸本)9798350333350
code coverage criteria define test objectives and provide actionable stopping conditions for creating test cases. They are also widely accepted as indicators of a test suite's fault detection effectiveness. Several factors influence the relationship between codecoverage and test suite effectiveness. However, previous empirical studies investigating this relationship tend to fail to control some of these factors, resulting in contradictory results. Through a systematic review of 417 previous studies investigating the effectiveness of test suites from various venues, such as journals, conferences, workshops, and book chapters, we have identified several factors that can impact experimental studies' results and even threaten their validity. Some of these factors are well known, such as the test suite size (the number of test cases in the test suite), while some others are relatively unknown, such as the variety of execution traces (how different are test cases within a test suite in terms of structural coverage level). The list of factors we describe should be of interest to researchers and practitioners alike.
暂无评论