Deep-Learning-based Fault Localisation (DLFL) leverages deep neural networks to learn the relationship between statement behaviour and program failures, showing promising results. However, since DLFL uses program fail...
详细信息
Deep-Learning-based Fault Localisation (DLFL) leverages deep neural networks to learn the relationship between statement behaviour and program failures, showing promising results. However, since DLFL uses program failures as labels to conduct supervised learning, a labelled dataset is a requisite of applying DLFL. A failure is detected by comparing program output with a test oracle which is the standard answer for the given input. The problem is, test oracles are often difficult, or even impossible to acquire in real life, and that has severely restricted the application of DLFL since we have only unlabelled datasets in most cases. Thus, MetaFL: Metamorphic Fault Localisation Using Weakly Supervised Deep Learning is proposed, to provide a weakly supervised learning solution for DLFL. Instead of using test oracles, MetaFL uses metamorphic relations to prescribe expected behaviour of a program, and defines labels of metamorphic testing groups by verifying integrity in each group of test cases. Hence, a coarse-grained labelled dataset can be built from the originally unlabelled one, with which DLFL can work now, utilising a weakly supervised learning paradigm. The experiments show that MetaFL yields a performance comparable to plain DLFL under ideal condition (i.e. the labels of datasets are available). MetaFL successfully extends the methodology of DLFL from supervised learning to weakly supervised learning, and a fully labelled dataset is no longer mandatory for applying DLFL.
As agile software development and extreme programing have become increasingly popular, continuous integration (CI) has become a widely used collaborative work method. However, it is common to make changes frequently t...
详细信息
As agile software development and extreme programing have become increasingly popular, continuous integration (CI) has become a widely used collaborative work method. However, it is common to make changes frequently to a project during CI. If existing testing methods are applied to CI directly, it will be difficult to make testing resources focus on changes generated by CI, which results in insufficient testing for changes. To solve this problem, we propose a fuzz testing method for CI. First, differential analysis is performed to determine the change points generated during CI, change points are added to the taint source set, and static analysis is conducted to calculate the distances between each basic block and the taint sources. Then, the project under test is instrumented according to the distances. During fuzz testing, testing resources are allocated based on seed coverage to test the change points effectively. Using the proposed methods, we implement CIDFuzz as a prototype tool, and experiments are conducted on four open-source projects that use CI. Experimental results show that, compared with AFL and AFLGo, CIDFuzz can reduce the time costs of covering change points up to 39.59% and 41.64%, respectively. Also, CIDFuzz can reduce the time costs of reproducing vulnerabilities up to 34.78% and 25.55%.
Software testing is a complex and costly stage during the software development lifecycle. Nowadays, there is a wide variety of solutions to reduce testing costs and improve test quality. Focussing on test case generat...
详细信息
Software testing is a complex and costly stage during the software development lifecycle. Nowadays, there is a wide variety of solutions to reduce testing costs and improve test quality. Focussing on test case generation, Dynamic Symbolic Execution (DSE) is used to generate tests with good structural coverage. Regarding test suite evaluation, Mutation testing (MT) assesses the detection capability of the test cases by introducing minor localised changes that resemble real faults. DSE is however known to produce tests that do not have good mutation detection capabilities: in this paper, the authors set out to solve this by combining DSE and MT into a new family of approaches that the authors call Mutation-Inspired Symbolic Execution (MISE). First, this known result on a set of open source programs is confirmed: DSE by itself is not good at killing mutants, detecting only 59.9% out of all mutants. The authors show that a direct combination of DSE and MT (naive MISE) can produce better results, detecting up to 16% more mutants depending on the programme, though at a high computational cost. To reduce these costs, the authors set out a roadmap for more efficient versions of MISE, gaining its advantages while avoiding a large part of its additional costs.
Real-world programs contain a multitude of concealed components whose formal semantics not available to verification engines, like third-party API calls, inline assembly and SIMD instructions, system calls, sampling a...
详细信息
ISBN:
(纸本)9781450399012
Real-world programs contain a multitude of concealed components whose formal semantics not available to verification engines, like third-party API calls, inline assembly and SIMD instructions, system calls, sampling assignments from probability distributions and library calls. Albeit the success of program verification, proving correctness of such "open" programs has remained a challenge. Currently, this problem is handled by manually "closing" the program-by providing handwritten mocks that attempt to capture the behavior of these concealed components. Most often, writing such mock code (stub) is not only arduous, but are often erroneous, thus raising serious questions on the whole endeavor. In light of this challenge, we came up with a novel technique, almost verification as an attempt to prove correctness for such "open" programs.
Despite being the most used approach in teaching software testing in Brazil, the traditional teaching method might not simulate, facilitate or motivate the learning of the testing process and its activities. As a resu...
详细信息
ISBN:
(纸本)9781450399999
Despite being the most used approach in teaching software testing in Brazil, the traditional teaching method might not simulate, facilitate or motivate the learning of the testing process and its activities. As a result, undergraduate students may not understand a basic testing process and its components, nor be motivated to learn about the content. One approach that can facilitate the teaching of the software testing process and promote student motivation is educational games. Although there are several educational games for teaching software testing, no game was foun on literature that addresses the entire testing process, with its roles, artifacts and activities. This work presents the development of an educational game to support and motivate the learning of a basic software testing process. The research method was Design Science Research. For the development of the game, several playtestings were carried out to verify if the new proposal was able to motivate and teach the proposed content. To evaluate the fulfillment of the research objectives, an experiment was performed with 16 participants, and it was possible to verify a gain of specific knowledge of the participants after the experience with the game. In addition, positive effects were observed on motivational aspects perceived by the players with the game for the learning of the themes involved.
Existing coverage criteria for Unified Modeling Language (UML)-based system design have used mostly structural constructs and sequential-behavioural coverage criteria. However, concurrent-behavioural coverage criteria...
详细信息
Existing coverage criteria for Unified Modeling Language (UML)-based system design have used mostly structural constructs and sequential-behavioural coverage criteria. However, concurrent-behavioural coverage criteria are not sufficiently explored. The basic path coverage criteria, in UML-based system design, do not distinguish between non-interleaving and interleaving execution paths in concurrent activity diagrams. Herein, the main contribution is to present a new set of coverage criteria, based on activity-interleaving paths for concurrent activity diagrams. For this, first, we modified the definitions of basic path and basic path coverage criterion, and then we defined the interleaving activity path, interleaving activity path coverage, and total activity path coverage criteria. It is observed that the proposed coverage criteria explore most of the concurrency behaviours to uncover concurrency errors. They help to uncover different types of concurrency errors like synchronization, data-race, and deadlock.
As software projects evolve and grow in size and complexity, so do their test suites. Test suite reduction (TSR) aims at reducing the size of a test suite by removing redundant and obsolete test cases based on a cover...
详细信息
As software projects evolve and grow in size and complexity, so do their test suites. Test suite reduction (TSR) aims at reducing the size of a test suite by removing redundant and obsolete test cases based on a coverage metric while preserving its fault detection capabilities. The contributions of this paper are twofold: (1) We examine a lesser-known coverage criterion, that is, checked coverage. Checked coverage not only investigates if a part of the code was executed but also if it was checked by a test oracle. In an empirical evaluation, we performed TSR based on different reduction algorithms, coverage metrics, and open-source Java projects with our own TSR tool to determine the most effective and efficient combination of metric and method. (2) Given the results of the first evaluation, we further investigate the potential of parameter optimization in regard to a genetic reduction algorithm. In particular, we focus on finding a general setting for the parameters crossover rate and mutation rate such that test suites can be reduced in a reasonable time while maintaining a high fault detection power. In this paper, we explore (1) the potential of checked coverage as a coverage metric as well as (2) parameter tuning for a genetic algorithm in the context of test suite reduction. In two empirical evaluations, we assess how different coverage metrics, algorithm combinations, and parameter values perform on open-source Java projects and our own Java reduction framework;based on the results we determine the most effective and efficient combination of metric and method. image
Testability transformation (TT) is a source-to-source programme transformation that aims to improve the ability of a given test generation method to generate test data for the original programme. Herein, the correctne...
详细信息
Testability transformation (TT) is a source-to-source programme transformation that aims to improve the ability of a given test generation method to generate test data for the original programme. Herein, the correctness of testability transformations is shown. Translation validation is the process of proving that the transformed programme is a correct translation of the source programme being compiled. It is widely used to verify the correctness of various compiler optimizations and transformations during scheduling. The value propagation based equivalence checking (VP) method is an efficient translation validation approach proposed to verify the correctness of various compiler optimization applied during scheduling in high-level synthesis. VP-based translation validation of testability transformations is proposed. In particular, it is identified that the existing VP method fails to show the equivalence for some of the TTs. A dynamic cutpoint selection scheme and an enhancement to the VP method to overcome these limitations are shown. The enhanced VP method, called VP_TT, successfully shows the equivalence for the TTs where the VP method fails. Experimental results confirm the usefulness of VP_TT in the verification of testability transformations.
This article discusses a recently developed test suite for checking timing-based vulnerabilities in processor caches, which has revealed the insecurity of today's processor caches. The susceptibility of caches to ...
详细信息
This article discusses a recently developed test suite for checking timing-based vulnerabilities in processor caches, which has revealed the insecurity of today's processor caches. The susceptibility of caches to these vulnerabilities calls for more research on secure processor caches.
We summarize the open challenges and opportunities for fuzzing and symbolic execution as they emerged in discussions among researchers and practitioners in a Shonan Meeting and that were validated in a subsequent survey.
We summarize the open challenges and opportunities for fuzzing and symbolic execution as they emerged in discussions among researchers and practitioners in a Shonan Meeting and that were validated in a subsequent survey.
暂无评论