Context: The importance of automotive software has been rapidly increasing because software controls many components of motor vehicles such as smart-key system, fire pressure monitoring system, and advanced driver ass...
详细信息
Context: The importance of automotive software has been rapidly increasing because software controls many components of motor vehicles such as smart-key system, fire pressure monitoring system, and advanced driver assistance system. Consequently, the automotive industry spends a large amount of human effort to test automotive software and is interested in automated testing techniques to ensure high-quality automotive software with reduced human effort. Objective: Applying automated test generation techniques to automotive software is technically challenging because of false alarms caused by imprecise test drivers/stubs and lack of tool supports for symbolic analysis of bit-fields and function pointers in C. To address such challenges, we have developed an automated testing framework MAESTRO. Method: MAESTRO automatically builds a test driver and stubs for a target task (i.e., a software unit consisting of target functions). Then, it generates test inputs to a target task with the test driver and stubs by applying concolic testing and fuzzing together in an adaptive way. In addition, MAESTRO transforms a target program that uses bitfields into a semantically equivalent one that does not use bit-fields. Also, MAESTRO supports symbolic function pointers by identifying the candidate functions of a symbolic function pointer through static analysis. Results: MAESTRO achieved 94.2% branch coverage and 82.3% MC/DC coverage on the four target modules (238 KLOC) developed by Hyundai Mobis. Furthermore, it significantly reduced the cost of coverage testing by reducing the manual effort for coverage testing by 58.8%. Conclusion: By applying automated testing techniques, MAESTRO can achieve high test coverage for automotive software with significantly reduced manual testing effort.
Test coverage analysis of exception handling features is a nontrivial task, since exception handling constructs introduce implicit control flow in programs. In this paper, we present a model-based technique for covera...
详细信息
ISBN:
(纸本)9781450362153
Test coverage analysis of exception handling features is a nontrivial task, since exception handling constructs introduce implicit control flow in programs. In this paper, we present a model-based technique for coverage analysis of exception handling features in object-oriented programs. Our technique is based on our previous work on coverage analysis. We first propose a program model, Augmented Call-based Object-Oriented System Dependence Graph (ACSD), and describe its construction. The ACSD represents the structural aspects of the exception handling mechanism, as well as intra-procedural and inter-procedural exceptional control and data flows. We propose several coverage measures specific to exception handling, and apply coverage-on-the-fly technique to the ACSD to determine those coverage measures. We also present the results of the experiments carried out using a prototype tool to demonstrate the efficacy of our ACSD model and coverage technique.
We introduce a new coverage measure, called the operational coverage, which is customized to the usage profile (count spectrum) of the entities to be covered. Operational coverage is proposed as an adequacy criterion ...
详细信息
ISBN:
(纸本)9781450341516
We introduce a new coverage measure, called the operational coverage, which is customized to the usage profile (count spectrum) of the entities to be covered. Operational coverage is proposed as an adequacy criterion for operational profile based testing, i.e., to assess the thoroughness of a black box test suite derived from the operational profile. To validate the approach we study the correlation between operational coverage of branches, statements, and functions, and the probability that the next test input will not fail. On the three subjects considered, we observed a moderate correlation in all cases (except a low correlation for function coverage for one subject), and consistently better results than traditional coverage measure.
Software is getting complicated due to the changing needs and flourishing development of software industry. To better improve software quality, we have to find the major reasons which cause the program crash. However,...
详细信息
ISBN:
(纸本)9781509055692
Software is getting complicated due to the changing needs and flourishing development of software industry. To better improve software quality, we have to find the major reasons which cause the program crash. However, debugging by software developer is not an efficient method, especially in large software. Many automated tools are developed to enhance the fault localization efficiency and reduce the maintenance cost. Most researches focus on improving the software testing process, and the primary triage method is based on the stack-trace hash (e.g., smartfuzz, basic fuzzing framework and Failure Observation engine), and is unchanged for a long time. Therefore, we propose a new triage method based on binary block coverage. Our triage method is designed by analyzing the binary level coverage results, on every time the input causes the program crash. For the same crash input, we also use traditional stack-trace hash method to contrast the flaws with our method. Our experiment results reveal that our proposed method based on code coverage exhibits better triages in terms of the number of unique bugs identified and correct classifications of faults.
In android ecosystem, the Apps marketplace vendor faces huge number of Apps with irregular quality. Besides bug finding, coverage index is neglected for the current Android testing services. However it is also a chall...
详细信息
ISBN:
(纸本)9781467365635
In android ecosystem, the Apps marketplace vendor faces huge number of Apps with irregular quality. Besides bug finding, coverage index is neglected for the current Android testing services. However it is also a challenge to measure the testingcoverage without source code. In this paper, we provide a systematic approach to measure the App testingcoverage for black-box testing and implement CovDroid, a black-box coverage system for android. Furthermore, we use a App with different test cases to prove our concept that coverage index can improve the App and measure test cases quality for App market or testing vendors.
Continuous integration (CI) is a programming practice that reduces the risk of project failure by integrating code changes multiple times a day. This has always been important to the Smalltalk community, so custom int...
详细信息
ISBN:
(纸本)9781450345248
Continuous integration (CI) is a programming practice that reduces the risk of project failure by integrating code changes multiple times a day. This has always been important to the Smalltalk community, so custom integration infrastructures are operated that allow CI testing for Smalltalk projects shared in Monticello repositories or traditional changesets. In the last few years, the open hosting platform GitHub has become more and more popular for Smalltalk projects. Unfortunately, there was no convenient way to enable CI testing for those projects. We present smalltalkCI, a continuous integration framework for Smalltalk. It aims to provide a uniform way to load and test Smalltalk projects written in different Smalltalk dialects. smalltalkCI runs on Linux, macOS, and on Windows and can be used locally as well as on a remote server. In addition, it is compatible with Travis CI and AppVeyor, which allows developers to easily set up free CI testing for their GitHub projects without having to run a custom integration infrastructure.
The Glass Box Test (GBT), also known as White Box Test or Structural Test, shows which parts of the program under test have, or have not, been executed. Many GBT tools are available for almost any programming language...
详细信息
ISBN:
(纸本)9783662448571
The Glass Box Test (GBT), also known as White Box Test or Structural Test, shows which parts of the program under test have, or have not, been executed. Many GBT tools are available for almost any programming language. Industry standards for safety-critical software require a very high or even complete coverage. At first glance, the GBT seems to be a well-established and mature testing technique that is based on standardized metrics. But on closer inspection, there are several serious shortcomings of the underlying models and metrics which lead to very imprecise, inconsistent coverage results of the various GBT tools. In this paper, a new and precise model for the GBT is presented. This model is used as a reference for the precise definition of all the popular coverage metrics that are around. The tool CodeCover which was developed in the University of Stuttgart is an implementation that strictly follows those definitions.
Software testing during the development process of embedded software is not only complex, but also the heart of quality control. Multi-core embedded software testing faces even more challenges. Major issues include: (...
详细信息
Software testing during the development process of embedded software is not only complex, but also the heart of quality control. Multi-core embedded software testing faces even more challenges. Major issues include: (1) how demanding efforts and repetitive tedious actions can be reduced;(2) how resource restraints of embedded system platform such as temporal and memory capacity can be tackled;(3) how embedded software parallelism degree can be controlled to empower multi-core CPU computing capacity;(4) how analysis is exercised to ensure sufficient coverage test of embedded software;(5) how to do data synchronization to address issues such as race conditions in the interrupt driven multi-core embedded system;(6) high level reliability testing to ensure customer satisfaction. To address these issues, this study develops an automatic testing environment for multi-core embedded software (ATEMES). Based on the automatic mechanism, the system can parse source code, instrument source code, generate testing programs for test case and test driver, support generating primitive, structure and object types of test input data, multi-round cross-testing, and visualize testing results. To both reduce test engineer's burden and enhance his efficiency when embedded software testing is in process, this system developed automatic testing functions including unit testing, coverage testing, multi-core performance monitoring. Moreover, ATEMES can perform automatic multi-round cross-testing benchmark testing on multi-core embedded platform for parallel programs adopting Intel TBB library to recommend optimized parallel parameters such as pipeline tokens. Using ATEMES on the ARM11 multi-core platform to conduct testing experiments, the results show that our constructed testing environment is effective, and can reduce burdens of test engineer, and can enhance efficiency of testing task. (C) 2011 Elsevier Inc. All rights reserved.
Technology of instrumentation is one of key technologies in the embedded software coverage testing. To reduce the impact of source program which from instrumented program implantated to the program to be tested, we pr...
详细信息
ISBN:
(纸本)9781457715846
Technology of instrumentation is one of key technologies in the embedded software coverage testing. To reduce the impact of source program which from instrumented program implantated to the program to be tested, we proposed a new technology of instrumentation which use the host environment as a test environment, A Sequence block as the basic unit of instrumentation. This method reduce the code expansion in some degree, and achieve an efficient Instrumentation. This technology have the versatility to coverage testing in embedded software Test. Tests for other languages, also have a certain reference value.
The complexity of the technologies involved in the Future Internet makes testing extremely challenging and demands for novel approaches and major advancement in the field. The overall aim of the FITTEST project is to ...
详细信息
ISBN:
(纸本)9780769543437
The complexity of the technologies involved in the Future Internet makes testing extremely challenging and demands for novel approaches and major advancement in the field. The overall aim of the FITTEST project is to address these testing challenges, by developing an integrated environment for automated testing, which can monitor the Future Internet application under test and adapt to the dynamic changes observed. Future Internet applications do not remain fixed after their release, services and components can be dynamically added by customers. Consequently, FITTEST testing will be continuous and post-release such that maintenance and quality assurance can cope with the changes in the intended use of an application after release. The testing environment will integrate, adapt and automate various techniques for continuous Future Internet testing (dynamic model inference, model-based testing, log-based diagnosis, oracle learning, combinatorial testing, concurrent testing, regression testing).
暂无评论