While the relation between code coverage measures and fault detection is actively studied, only few works have investigated the correlation between measures of coverage and of reliability. In this work, we introduce a...
详细信息
While the relation between code coverage measures and fault detection is actively studied, only few works have investigated the correlation between measures of coverage and of reliability. In this work, we introduce a novel approach to measuring code coverage, called the operational coverage, that takes into account how much the program's entities are exercised so to reflect the profile of usage into the measure of coverage. operational coverage is proposed as (i) an adequacy criterion, i.e., to assess the thoroughness of a black box test suite derived from the operational profile, and as (ii) a selection criterion, i.e., to select test cases for operational profile-based testing. Our empirical evaluation showed that operational coverage is better correlated than traditional coverage with the probability that the next test case derived according to the user's profile will not fail. This result suggests that our approach could provide a good stopping rule for operational profile-based testing. With respect to test case selection, our investigations revealed that operational coverage outperformed the traditional one in terms of test suite size and fault detection capability when we look at the average results.
coverage metrics try to quantify how well a software artifact is tested. High coverage numbers instill confidence in the software and might even be necessary to obtain certification. Unfortunately, achieving high cove...
详细信息
ISBN:
(纸本)9781538636817
coverage metrics try to quantify how well a software artifact is tested. High coverage numbers instill confidence in the software and might even be necessary to obtain certification. Unfortunately, achieving high coverage numbers does not imply high quality of the test suite. One shortcoming is that coverage metrics do not measure how well test suites cover systems in production. We look at coverage from an operational perspective. We evaluate test suite quality by comparing runs executed during testing with runs executed in production. Branch frequency spectra are employed to capture the behavior during runtime. Differences in the branch frequency spectra between field executions and testing runs indicate test suite deficiencies. This post-release test suite quality assurance mechanism can be used to (1) build confidence by pooling coverage information from many execution sites and (2) guide test suite augmentation in order to prepare the test suite for the next release cycle.
We introduce covrel, an adaptive software testing approach based on the combined use of operational profile and coverage spectrum, with the ultimate goal of improving the delivered reliability of the program under tes...
详细信息
ISBN:
(纸本)9781538638682
We introduce covrel, an adaptive software testing approach based on the combined use of operational profile and coverage spectrum, with the ultimate goal of improving the delivered reliability of the program under test. operational profile-based testing is a black-box technique that selects test cases having the largest impact on failure probability in operation;as such, it is considered well suited when reliability is a major concern. Program spectrum is a characterization of a program's behavior in terms of the code entities (e.g., branches, statements, functions) that are covered as the program executes. The driving idea of covrel is to complement operational profile information with white-box coverage measures based on count spectra, so as to dynamically select the most effective test cases for reliability improvement. In particular, we bias operational profile-based test selection towards those entities covered less frequently. We assess the approach by experiments with 18 versions from 4 subjects commonly used in software testing research, comparing results with traditional operational and coverage testing. Results show that exploiting operational and coverage data in a combined adaptive way actually pays in terms of reliability improvement, with covrel overcoming conventional operational testing in more than 80% of the cases.
We introduce a new coverage measure, called the operational coverage, which is customized to the usage profile (count spectrum) of the entities to be covered. operational coverage is proposed as an adequacy criterion ...
详细信息
ISBN:
(纸本)9781450341516
We introduce a new coverage measure, called the operational coverage, which is customized to the usage profile (count spectrum) of the entities to be covered. operational coverage is proposed as an adequacy criterion for operational profile based testing, i.e., to assess the thoroughness of a black box test suite derived from the operational profile. To validate the approach we study the correlation between operational coverage of branches, statements, and functions, and the probability that the next test input will not fail. On the three subjects considered, we observed a moderate correlation in all cases (except a low correlation for function coverage for one subject), and consistently better results than traditional coverage measure.
Software vulnerabilities trend over time has been proposed by various researchers and academicians in recent years. But none of them have considered operational coverage function in vulnerability discovery modeling. I...
详细信息
Software vulnerabilities trend over time has been proposed by various researchers and academicians in recent years. But none of them have considered operational coverage function in vulnerability discovery modeling. In this research paper, we have proposed a generalized statistical model that determines the relationship between operational coverage function and the number of expected vulnerabilities. During the operational phase, possible vulnerable sites are covered and vulnerabilities present at a particular site are discovered with some probability. We have assumed that the proposed model follows the nonhomogeneous Poisson process properties;thus, different distributions are used to formulate the model. The numerical illustration shows that the proposed model performs better and has the good fitness to the Google Chrome data. The second focus of this research paper is to evaluate the total cost incurred by the developer after software release and to identify the optimal vulnerability disclosure time through multiobjective utility function. The proposed vulnerability discovery helps in optimization. The optimal time problem depends on the combined effect of cost, risk, and effort.
Background: Mass distribution of insecticide-treated nets (ITNs) is a cost-effective way to achieve universal coverage, but maintaining this coverage is more difficult. In addition to commonly used indicators, evaluat...
详细信息
Background: Mass distribution of insecticide-treated nets (ITNs) is a cost-effective way to achieve universal coverage, but maintaining this coverage is more difficult. In addition to commonly used indicators, evaluation of universal coverage should include coverage of effective nets and changes in coverage over time. Methods: Longitudinal and cross-sectional household ITN surveys were carried out from 2010 to 2013 in six locations representing a variety of settings across western Kenya. Five indicators were used to evaluate the current status of universal coverage: 1) ITN ownership -proportion of households that own at least one ITN, 2) access index ratio of the number of family members over the number of ITNs owned by that household, 3) operational coverage proportion of the at-risk population potentially covered by ITNs, assuming one ITN for every two people, 4) effective coverage -population coverage of effective ITNs, and 5) usage -proportion of the population that used ITNs the previous night. Results: ITN ownership and operational coverage increased substantially from 2010 to 2013, but this increase was mostly due to the 2011 mass distribution campaign. In 2013, household ITN ownership was on average 84.4% (95% CI [78.4, 90.5]) across the six study areas, and operational coverage was 83.2% (95% CI [72.5, 93.8]). The ITN access rate was 59.1% (95% CI [56.6, 61.7]), and 40.8% (95% CI [38.3, 43.4]) of the people at risk needed more nets to achieve universal coverage. About 88.5% (95% CI [86.1, 90.9]) of the ITNs were below three years old and 16.5% (95% CI [12.1, 20.9]) of the ITNs had hole(s). The estimated effective long-lasting insecticide-treated net (LLIN) coverage was 70.5% (95 CI [58.7, 82.3]). Approximately 18.4% (95% CI [15.5, 21.4]) of the ITNs were shared by more than three persons, and the population ITN usage rate was about 75-87%. The reason for not using ITNs was almost exclusively "net not available". Conclusion: Current methods of delivering ITN
暂无评论