With continuous development in the IC advanced technologies, physical verification is becoming increasingly more complex in the semiconductor manufacturing process. Currently, different regression patterns-based techn...
详细信息
ISBN:
(纸本)9781538625491
With continuous development in the IC advanced technologies, physical verification is becoming increasingly more complex in the semiconductor manufacturing process. Currently, different regression patterns-based techniques are widely adopted in the design rules checking (DRC) verification. As complexity of design rules increases, limitations of those traditional techniques emerge such as focusing only on the rule final output, missing some of the underlying defects in the DRC code, requiring many QA cycles for the rule deck to be released, and lacking well-defined measure of the DRC coverage. This paper presents a new methodology in verifying the design rules checks, complementing the traditional Regression technique, through performing DRC rule deck line-by-line comparison with the regression layout patterns. This offers in-depth and quantitative analysis of all the DRC components, captures the possible defects in very early stages of the QA process, and reduces the time consumed in verification significantly compared to using just the traditional QA techniques.
code coverage is the primary metric used to assess the quality of test suites, and it is the foundation of many automated techniques ranging from fault localization to search-based optimization approaches. code covera...
详细信息
ISBN:
(纸本)9798400705885
code coverage is the primary metric used to assess the quality of test suites, and it is the foundation of many automated techniques ranging from fault localization to search-based optimization approaches. code coverage is measured by inserting probes into programs which keep track of executed code when running tests. While this can be easily done in many testing domains, it remains a challenging task for Android apps, mainly due to the nature of the Dalvik bytecode used for Android apps: First, the internal handling of registers inhibits common types of probes. To circumvent this problem, existing tools often rely on conversion of Dalvik bytecode to standard Java bytecode or source code, but during the conversion back to Dalvik bytecode errors and inconsistencies may occur. Furthermore, a strict limit of the number of methods and classes contained in a single archive of Dalvik bytecode (DEX file) requires spliting apps into multiple such DEX files (multidex approach), which is rarely supported by existing coverage instrumentation frameworks. This is not only a problem when trying to instrument regular multidex apps, but the coverage instrumentation itself increases the number of methods, potentially requiring a multidex solution even for apps that would otherwise fit in a single DEX file. In this paper we present WallMauer, a new code coverage tool that overcomes these limitations: It supports multidex, and avoids inconsistencies by rigorously instrumenting Dalvik bytecode directly. WallMauer solely requires an APK file as input and as such it can be easily integrated into any existing testing environment. Using a set of 1000 open source apps from the F-Droid repository we demonstrate that WallMauer is extremely robust, successfully instrumenting more than 99% of apps, more than any other state-of-the-art instrumentation framework.
Total number of failures of a software system can help practitioners to have a better understanding of the software quality. In this paper, we propose a model to predict the total number of software failures in a soft...
详细信息
ISBN:
(纸本)9781538623879
Total number of failures of a software system can help practitioners to have a better understanding of the software quality. In this paper, we propose a model to predict the total number of software failures in a software system by analyzing the failure data from testing using models based on Zipf's law together with the information on code coverage. Failure data and code coverage are combined in a Bayesian way. The methodology is applied to real world failure data to validate its predictability. The predictive accuracy of our model is also evaluated with different methods. The results of our experiment show that our proposed model can provide a very good estimation of the total number of failures. The estimation is stable from a very early point on.
Continuous integration (CI) is a software engineering practice that advocates the frequent integration of software through an automated build process. Existing research has explored the benefits of CI, such as detecti...
详细信息
ISBN:
(纸本)9798350311846
Continuous integration (CI) is a software engineering practice that advocates the frequent integration of software through an automated build process. Existing research has explored the benefits of CI, such as detecting errors earlier in the software life-cycle. Although CI heavily focuses on automated tests, it is still unclear whether CI is associated with better code coverage, which could be a major benefit of using CI. To investigate whether CI is associated with an improvement in code coverage, our work compares 30 projects that adopted CI (CI projects) and 30 projects that have never adopted CI (NOCI projects). In total, we studied 1,440 versions from different projects to analyze trends in code coverage related to CI. While evaluating trends of code coverage within CI and NOCI projects, we observe more projects with rising trends of code coverage in CI projects (50%) than NOCI projects (10%). Moreover, the maintaining trends are different, as CI projects tend to stabilize at a higher code coverage rate than NOCI projects. Investigating the CI projects alone, the statistical evidences indicate that the adoption of CI is associated with the increase in code coverage. The findings of this study, therefore, reveal a positive association between CI and a higher code coverage rate.
Typical *** applications extensively rely on packages hosted in the npm registry. As such packages may be used by thousands of other packages or applications, it is important to assess their code coverage. Moreover, i...
详细信息
ISBN:
(纸本)9781665435673
Typical *** applications extensively rely on packages hosted in the npm registry. As such packages may be used by thousands of other packages or applications, it is important to assess their code coverage. Moreover, increasing code coverage may help detect previously unknown issues. In this paper, we introduce TESA, a new tool that automatically assembles a test suite for any package in the npm registry. The test suite includes 1) tests written for the target package and usually hosted in its development repository, and 2) tests selected from dependent packages. The former tests allow assessing the code coverage of the target package, while the latter ones can increase code coverage by exploiting third-party tests that also exercise code in the target package. We use TESA to assess the code coverage of 500 popular npm packages. Then, we demonstrate that TESA can significantly increase code coverage by including tests from dependent packages. Finally, we show that the test suites assembled by TESA increase the effectiveness of existing dynamic program analyses to identify performance issues that are not detectable when only executing the developer's tests.
In order to solve the problems of traditional Fuzzing technique for software vulnerability detection, a novel method based on code coverage and test cost is proposed. Firstly, static analysis is applied to calculate t...
详细信息
ISBN:
(纸本)9781467386609
In order to solve the problems of traditional Fuzzing technique for software vulnerability detection, a novel method based on code coverage and test cost is proposed. Firstly, static analysis is applied to calculate the code coverage information, including basic block coverage and new block coverage. In addition, test path diversity information is introduced to elevate path coverage, which is achieved based on the sequence alignment algorithm. Secondly, test cost is analyzed respectively from running time and loop structure. The loop structure is simplified using finite expansion manner. Thirdly, the genetic algorithm fitness function is constructed based on the code coverage and test cost to guide the test case generation. Experiments on realistic binary software show that the method could obtain higher vulnerability detection accuracy and efficiency than the traditional Fuzzing technique.
code coverage measurement is an important element in white-box testing, both in industrial practice and academic research. Other related areas are highly dependent on code coverage as well, including test case generat...
详细信息
ISBN:
(纸本)9781509018550
code coverage measurement is an important element in white-box testing, both in industrial practice and academic research. Other related areas are highly dependent on code coverage as well, including test case generation, test prioritization, fault localization, and others. Inaccuracies of a code coverage tool sometimes do not matter that much but in certain situations they can lead to serious confusion. For Java, the prevalent approach to code coverage measurement is to use bytecode instrumentation due to its various benefits over source code instrumentation. However, if the results are to be mapped back to source code this may lead to inaccuracies due to the differences between the two program representations. In this paper, we systematically investigate the amount of differences in the results of these two Java code coverage approaches, enumerate the possible reasons and discuss the implications on various applications. For this purpose, we relied on two widely used tools to represent the two approaches and a set of benchmark programs from the open source domain.
Assessing the overall quality (adequacy for a particular purpose) of existing test suites is a complex task. Their code coverage is a simple yet powerful attribute for this purpose, so the additional benefits of mutat...
详细信息
ISBN:
(纸本)9781509036745
Assessing the overall quality (adequacy for a particular purpose) of existing test suites is a complex task. Their code coverage is a simple yet powerful attribute for this purpose, so the additional benefits of mutation analysis may not always justify the comparably much higher costs and complexity of the computation. Mutation testing methods and tools slowly start to reach a maturity level at which their use in everyday industrial practice becomes possible, yet it is still not completely clear in which situations they provide additional insights into various quality attributes of the test suites. This paper reports on an experiment conducted on four open source systems' test suites to compare them from the viewpoints of code coverage, mutation score and test suite reducibility (the amount test adequacy is degraded in a reduced test suite). The purpose of the comparison is to find out when the different attributes provide additional insights with respect to defect density, a separately computed attribute for the estimation of real faults. We demonstrate that in some situations code coverage might be a sufficient indicator of the expected defect density, but mutation and reducibility are better in most of the cases.
During development, systems may be tested several times. In general, a system evolves from change requests, aiming at improving its behavior in terms of new features as well as fi xing failures. Thus, selecting the be...
详细信息
ISBN:
(纸本)9781450353021
During development, systems may be tested several times. In general, a system evolves from change requests, aiming at improving its behavior in terms of new features as well as fi xing failures. Thus, selecting the best test plan in terms of the closeness between test cases and the changed code and its dependencies is pursued by industry and academia. In this paper we measure the coverage achieved by an automatic test case selection based on information retrieval that relates change requests and test cases. But instead of using off-the-shelf coverage tools, like JaCoCo, we propose a way of obtaining code coverage of Android apk's without instrumentation. This was a basic requirement of our industrial partner. We performed some experiments on this industrial partner and promising results were obtained.
The incompleteness of 3rd-party app testing is an accepted fact in Software Engineering. This issue makes it impossible to verify the app functionality and to confirm its safety to the end-user. To solve this problem,...
详细信息
ISBN:
(纸本)9781728195537
The incompleteness of 3rd-party app testing is an accepted fact in Software Engineering. This issue makes it impossible to verify the app functionality and to confirm its safety to the end-user. To solve this problem, enterprises developed strict policies. A company, willing to use modern apps, may perform an expensive security analysis, rely on trust or forbid the app. These strategies may lead companies to high direct and indirect spending with no guarantee of safety. In this work, we present a novel approach, called Dynamic Binary Shrinking, that allows a user to review app functionality and leave only tested code. The shrunk app produces 100% instruction coverage on observed behaviors and in this way guarantees the absence of unexplored, and therefore, potentially malicious code. On our running examples, we demonstrate that apps use less than 20% of the codebase. We developed an approach and the ACVCut tool to shrink Android apps towards the executed code.
暂无评论