We present LgDb 2.0. The second generation of LgDb, an innovative framework for kernel code coverage, profiling and simulation. LgDb is built on top of Lguest and allows running an inspected kernel on a virtual enviro...
详细信息
ISBN:
(纸本)9781479907175
We present LgDb 2.0. The second generation of LgDb, an innovative framework for kernel code coverage, profiling and simulation. LgDb is built on top of Lguest and allows running an inspected kernel on a virtual environment instead of modifying the running kernel or using an extra target machine. LgDb 2.0 is using the Lguest hypervisor and the KGDB kernel debugger to debug and instrument kernel code. Unlike the standard approaches, LgDb enlist the hypervisor to achieve a better debugging environment for kernel development. LgDb strives to provide a generic environment for running performance evaluation and checking decision coverage for any inspected kernel. LgDb 2.0 improves over the original LgDb by using a simulated serial port and inspecting the tested code using KGDB. By using KGDB we eliminate the need for code injections making, profiling and code coverage testing easier.
Generating test cases from the specification can be done early in the software development life cycle. A test suite systematically generated from the software specification can be comprehensive in ensuring the intende...
详细信息
ISBN:
(纸本)9781424459124
Generating test cases from the specification can be done early in the software development life cycle. A test suite systematically generated from the software specification can be comprehensive in ensuring the intended functions to be exercised. Such a test suite is also instrumental in facilitating early discovery of software defects. On the other hand, an adequate level of code coverage is required to inspire confidence of the testing and quality assurance process. To examine the effectiveness of a specification-based test suite in covering program code, we conducted an exploratory case study which involved many different versions of programs that implemented the same specification. On the basis of the case study, this paper identifies the types of path in the programs, suggests enhancements to the existing test generation methods, and discusses several issues on how the code coverage can be improved.
This paper addresses the overall problem of estimating the quality of a test suite when testing is performed at a platform-independent level, using executable UML models. The problem is that the test suite is often re...
详细信息
ISBN:
(纸本)9789897582325
This paper addresses the overall problem of estimating the quality of a test suite when testing is performed at a platform-independent level, using executable UML models. The problem is that the test suite is often required to fulfill structural code coverage criteria. In the avionics domain it is usually required that the tests achieve 100% coverage according to logic-based coverage criteria. Such criteria are less effective when applied to executable UML models than when they are applied to code because the action code found in such models contains conditions in navigation and loops that are not explicit and therefore not captured by logic-based coverage criteria. We present two new coverage criteria for executable UML models, and we use an industrial application from the avionics domain to show that these two criteria should be combined with a logic-based criterion when testing the executable UML model. As long as the coverage is less than 100% at the model level, there is no point in running the tests at the code level since all functionality of the model is not yet tested, and this is necessary to achieve 100% coverage at the code level.
Context: In software development and maintenance, a software system may frequently be updated to meet rapidly changing user requirements. New test cases will be designed to ensure the correctness of new or modified fu...
详细信息
Context: In software development and maintenance, a software system may frequently be updated to meet rapidly changing user requirements. New test cases will be designed to ensure the correctness of new or modified functions, thus gradually increasing the test suite's size. Test suite reduction techniques aim to decrease the cost of regression testing by removing the redundant test cases from the test suite and then obtaining a representative set of test cases that still yield a high level of code coverage. Objective: Most of the existing reduction algorithms focus on decreasing the test suite's size. Yet, the differences in execution costs among test cases are usually significant and it may take a lot of execution time to run a test suite consisting of a few long-running test cases. This paper presents and empirically evaluates cost-aware algorithms that can produce the representative sets with lower execution costs. Method: We first use a cost-aware test case metric, called Irreplaceability, and its enhanced version, called Elrreplaceability, to evaluate the possibility that each test case can be replaced by others during test suite reduction. Furthermore, we construct a cost-aware framework that incorporates the concept of test irreplaceability into some well-known test suite reduction algorithms. Results: The effectiveness of the cost-aware framework is evaluated via the subject programs and test suites collected from the Software-artifact Infrastructure Repository - frequently chosen benchmarks for experimentally evaluating test suite reduction methods. The empirical results reveal that the presented algorithms produce representative sets that normally incur a low cost to yield a high level of test coverage. Conclusion: The presented techniques indeed enhance the capability of the traditional reduction algorithms to reduce the execution cost of a test suite. Especially for the additional Greedy algorithm, the presented techniques decrease the costs of the repre
Software bloat is code that is packaged in an application but is actually not necessary to run the application. The presence of software bloat is an issue for security, performance, and for maintenance. In this articl...
详细信息
Software bloat is code that is packaged in an application but is actually not necessary to run the application. The presence of software bloat is an issue for security, performance, and for maintenance. In this article, we introduce a novel technique for debloating, which we call coverage-based debloating. We implement the technique for one single language: Java bytecode. We leverage a combination of state-of-the-art Java bytecode coverage tools to precisely capture what parts of a project and its dependencies are used when running with a specific workload. Then, we automatically remove the parts that are not covered, in order to generate a debloated version of the project. We succeed to debloat 211 library versions from a dataset of 94 unique opensource Java libraries. The debloated versions are syntactically correct and preserve their original behaviour according to the workload. Our results indicate that 68.3 % of the libraries' bytecode and 20.3 % of their total dependencies can be removed through coverage-based debloating. For the first time in the literature on software debloating, we assess the utility of debloated libraries with respect to client applications that reuse them. We select 988 client projects that either have a direct reference to the debloated library in their source code or which test suite covers at least one class of the libraries that we debloat. Our results show that 81.5 % of the clients, with at least one test that uses the library, successfully compile and pass their test suite when the original library is replaced by its debloated version.
The goal of regression testing is to ensure that the behaviour of existing code, believed correct by previous testing, is not altered by new program changes. This paper argues that the primary focus of regression test...
详细信息
The goal of regression testing is to ensure that the behaviour of existing code, believed correct by previous testing, is not altered by new program changes. This paper argues that the primary focus of regression testing should be on code associated with (1) earlier bug fixes and (2) particular application scenarios considered to be important by the developer or tester. Existing coverage criteria do not enable such focus, for example, 100% branch coverage does not guarantee that a given bug fix is exercised or a given application scenario is tested. Therefore, there is a need for a new and complementary coverage criterion in which the user can definea test requirement characterizing a given behaviour to be covered as opposed to choosing from a pool of pre-defined and generic program elements. This paper proposes this new methodology and calls it UCov, a user-defined coverage criterion wherein a test requirement is an execution pattern of program elements, and possibly predicates, that a test case must satisfy. The proposed criterion is not meant to replace existing criteria, but to complement them as it focuses the testing on important code patterns that could go untested otherwise. UCov supports test case intent verification. For example, following a bug fix, the testing team may augment the regression suite with the test case that revealed the bug. However, this test case might become obsolete due to code modifications not related to the bug. But if a test requirement characterizing the bug was defined by the user, UCov would determine that test case intent verification failed. The UCov methodology was implemented for the Java platform, was successfully applied onto 10 real-life case studies and was shown to have advantages over JUnit. The implementation comprises the following tools: (1) TRSpec: allows the user to easily specify complex test requirements;(2) TRCheck: checks whether user-defined test requirements were satisfied, that is, supports test case intent
What is a test case for? Sometimes, to expose a fault. Tests can also exercise code, use memory or time, or produce desired output. Given a desired effect, a test case can be seen as a cause, and its components divide...
详细信息
What is a test case for? Sometimes, to expose a fault. Tests can also exercise code, use memory or time, or produce desired output. Given a desired effect, a test case can be seen as a cause, and its components divided into essential (required for effect) and accidental. Delta debugging is used for removing accidents from failing test cases, producing smaller test cases that are easier to understand. This paper extends delta debugging by simplifying test cases with respect to arbitrary effects, a generalization called cause reduction. Suites produced by cause reduction provide effective quick tests for real-world programs. For Mozilla's JavaScript engine, the reduced suite is possibly more effective for finding faults. The effectiveness of reduction-based suites persists through changes to the software, improving coverage by over 500 branches for versions up to 4months later. Cause reduction has other applications, including improving seeded symbolic execution, where using reduced tests can often double the number of additional branches explored. Copyright (c) 2015 John Wiley & Sons, Ltd.
In the domain of software testing, the generation of test cases is a critical process for detecting system errors and bugs. However, automated test case generation for smart contracts often encounters challenges relat...
详细信息
In the domain of software testing, the generation of test cases is a critical process for detecting system errors and bugs. However, automated test case generation for smart contracts often encounters challenges related to automation, vulnerability diversity, and coverage. This paper presents a novel method, the self-adaptive learning Genetic Algorithm (self-adaptive learning GA), designed to address these issues. Our research methodology incorporates several construction models, namely the Control Dependence Graph (CDG), Control Flow Graph (CFG), and Application Binary Interface (ABI). Initially, the ABI model provides essential information for generating and executing test cases. The CFG model subsequently visualizes potential execution paths through the functions of smart contracts. Ultimately, the CDG model identifies potential vulnerabilities in smart contracts. Using these models, our method enhances automatic test case generation in smart contracts by improving coverage and reducing execution time. We selected a variety of smart contracts from the Decentralized Finance (DeFi) ecosystem for data collection and comparative analysis. The experimental results show superior performance rates, with an average code coverage rate of 98.1%, a total of 3500 vulnerabilities detected, a vulnerability detection rate of 98.7%, a false positive rate of 1.3%, a recall of 98.2%, precision of 98.8%, a path uniqueness rate of 96.4%, false negative rate of 3.5%, an execution time of 25 s, and test case generation time of 16 s. In conclusion, our proposed approach demonstrates a significant improvement over existing methods for test case generation by providing a promising solution for the robustness of smart contracts and security enhancement in the DeFi ecosystem.
Mutation analysis assesses a test suite's adequacy by measuring its ability to detect small artificial faults, systematically seeded into the tested program. Mutation analysis is considered one of the strongest te...
详细信息
Mutation analysis assesses a test suite's adequacy by measuring its ability to detect small artificial faults, systematically seeded into the tested program. Mutation analysis is considered one of the strongest test-adequacy criteria. Mutation testing builds on top of mutation analysis and is a testing technique that uses mutants as test goals to create or improve a test suite. Mutation testing has long been considered intractable because the sheer number of mutants that can be created represents an insurmountable problem-both in terms of human and computational effort. This has hindered the adoption of mutation testing as an industry standard. For example, Google has a codebase of two billion lines of code and more than 150,000,000 tests are executed on a daily basis. The traditional approach to mutation testing does not scale to such an environment;even existing solutions to speed up mutation analysis are insufficient to make it computationally feasible at such a scale. To address these challenges, this paper presents a scalable approach to mutation testing based on the following main ideas: (1) mutation testing is done incrementally, mutating only changed code during code review, rather than the entire code base;(2) mutants are filtered, removing mutants that are likely to be irrelevant to developers, and limiting the number of mutants per line and per code review process;(3) mutants are selected based on the historical performance of mutation operators, further eliminating irrelevant mutants and improving mutant quality. This paper empirically validates the proposed approach by analyzing its effectiveness in a code-review-based setting, used by more than 24,000 developers on more than 1,000 projects. The results show that the proposed approach produces orders of magnitude fewer mutants and that context-based mutant filtering and selection improve mutant quality and actionability. Overall, the proposed approach represents a mutation testing framework that seamles
code-coverage-based test data adequacy criteria typically treat all coverable code elements (such as statements, basic blocks or outcomes of decisions) as equal. In practice, however, the probability that a test case ...
详细信息
code-coverage-based test data adequacy criteria typically treat all coverable code elements (such as statements, basic blocks or outcomes of decisions) as equal. In practice, however, the probability that a test case can expose a fault in a code element varies: some faults are more easily revealed than others. Thus, several researchers have suggested that if one could estimate the probability that a fault in a code element will cause a failure, one could use this estimate to determine the number of executions of a code element that are required to achieve a certain level of confidence in that element's correctness. This estimate, in turn, could be used to improve the fault-detection effectiveness of test suites and help testers distribute testing resources more effectively. This conjecture is intriguing;however, like many such conjectures it has never been directly examined empirically. If empirical evidence were to support this conjecture, it would motivate further research into methodologies for obtaining fault-exposure-potential estimates and incorporating them into test data adequacy criteria. This paper reports the results of experiments conducted to investigate the effects of incorporating an estimate of fault-exposure probability into the statement coverage test data adequacy criterion. The results of these experiments, however, ran contrary to the conjectures of previous researchers. Although incorporation of the estimates did produce statistically significant increases in the fault-detection effectiveness of test suites, these increases were quite small, suggesting that the approach might not be able to produce the gains hoped for and might not be worth the cost of its employment. Copyright (C) 2002 John Wiley Sons, Ltd.
暂无评论