The performance of a system is determined by its characteristics as well as by the composition of the load being processed. Hence, its quantitative description is a fundamental part of all performance evaluation studi...
详细信息
The performance of a system is determined by its characteristics as well as by the composition of the load being processed. Hence, its quantitative description is a fundamental part of all performance evaluation studies. Several methodologies for the construction of workload models, which are functions of the objective of the study, of the architecture of the system to be analyzed, and of the techniques adopted, are presented. A survey of a few applications of these methodologies to various types of systems (i.e., batch, interactive, data base, network-based, parallel, supercomputer) is given.
Currently, eXtensible Access Control Markup Language (XACML) has becoming the standard for implementing access control policies and consequently more attention is dedicated to testing the correctness of XACML policies...
详细信息
Currently, eXtensible Access Control Markup Language (XACML) has becoming the standard for implementing access control policies and consequently more attention is dedicated to testing the correctness of XACML policies. In particular, coverage measures can be adopted for assessing test strategy effectiveness in exercising the policy elements. This study introduces a set of XACML coverage criteria and describes the access control infrastructure, based on a monitor engine, enabling the coverage criterion selection and the on-line tracing of the testing activity. Examples of infrastructure usage and of assessment of different test strategies are provided.
Component-based system (CBS) development usually involves adapting best-matched components and writing integration code to handle the mismatches between system-to-be needs and available components. CBS presents unique...
详细信息
Component-based system (CBS) development usually involves adapting best-matched components and writing integration code to handle the mismatches between system-to-be needs and available components. CBS presents unique maintenance challenges as a system analyst focuses on integrating existing components to build a software application. The lack of detailed individual component specification has been a key area of concern in CBS development because of its impact on the maintenance phase of a CBS development life cycle. Recent research suggests that the framework for integrated test (Fit) helps to better comprehend systems. In this work, the author presents a study to investigate the usefulness of acceptance test cases (Fit tables) information, when used as a functional specification, during the maintenance process. The mean correct implementation percentage with and without Fit tables were 86.19 and 83.52, respectively. Similarly, the mean times for the Fit tables and without Fit tables were 46.5 and 45.73, respectively. The results show that acceptance test cases help improve the functional correctness of changes with a little more effort during the maintenance process of a CBS.
The three most often used benchmarks are characterized in detail and users are warned about a number of pitfalls. Two of them, Whetstone and Drystone, are synthetic benchmarks: they were written solely for benchmarkin...
详细信息
The three most often used benchmarks are characterized in detail and users are warned about a number of pitfalls. Two of them, Whetstone and Drystone, are synthetic benchmarks: they were written solely for benchmarking purposes and perform no useful computation. Linpack was distilled out of a real, purposeful program that is now used as a benchmark. Some other benchmarks, namely Livermore Fortran Kernels, Stanford Small programs Benchmark Set, EDN benchmarks, Sieve of Eratosthenes, Rhealstone and SPEC benchmarks, are briefly considered. Non-CPU influences in benchmark performance are discussed
This paper addresses the problem of constructing test data sets from formal specifications. Starting from a notion of an ideal exhaustive test data set, which is derived from the notion of satisfaction of the formal s...
详细信息
This paper addresses the problem of constructing test data sets from formal specifications. Starting from a notion of an ideal exhaustive test data set, which is derived from the notion of satisfaction of the formal specification, we show how to select by refinements a practicable test set, i.e. computable, while not rejecting correct programs (unbiased) and accepting only correct programs (valid), when assuming certain hypotheses. The hypotheses play an important role;they formalise common test practices and they express the gap between the success of the test and correctness;the size of the test set depends on the strength of the hypotheses. The paper shows an application of this theory for algebraic specifications and presents the actual procedures used to mechanically product such test sets, using Horn clause logic. These procedures are embedded in an interactive system, which, given some general hypothesis schemes and an algebraic specification, produces a test set and the corresponding hypotheses.
An unduly large number of test cases are required for effective testing of programmes containing complex decision statements. In this context, modified condition/decision coverage (MC/DC) testing has been acknowledged...
详细信息
An unduly large number of test cases are required for effective testing of programmes containing complex decision statements. In this context, modified condition/decision coverage (MC/DC) testing has been acknowledged to provide effective testing using a test suite whose size is linear in the number of the clauses present in a predicate. MC/DC testing is well-accepted and is mandated by several testing standards. T-way testing is another prominent testing technique that helps to limit the combinatorial explosion of black-box test cases. Its application to white-box testing promises to provide effective testing with a comparatively small number of test cases. The authors empirically investigate the effectiveness of MC/DC testing vis-a-vis white-box pairwise, 3-way and 4-way testing.
This paper presents a description of, and experience in using, AIDA, an automated instrumentation system to perform data flow analysis for Pascal programs. AIDA is capable of detecting not only data flow anomalies, bu...
详细信息
This paper presents a description of, and experience in using, AIDA, an automated instrumentation system to perform data flow analysis for Pascal programs. AIDA is capable of detecting not only data flow anomalies, but also certain kinds of errors. It is a useful software testing and development tool for Pascal programs. The associated implementation problems and their solutions are presented.
Internet of things (IoT) systems are becoming ubiquitous and assuring their quality is fundamental. Unfortunately, a few proposals for testing these complex, and often safety-critical, systems are present in the liter...
详细信息
Internet of things (IoT) systems are becoming ubiquitous and assuring their quality is fundamental. Unfortunately, a few proposals for testing these complex, and often safety-critical, systems are present in the literature. The authors propose an approach for acceptance testing of IoT systems adopting graphical user interfaces as a principal way of interaction. Acceptance testing is a type of black box testing based on test scenarios, i.e. sequences of steps/actions performed by the user or the system. In their approach, test scenarios are derived from a state machine that expresses the behaviour of the system under test, and test cases are derived from them by specifying the actual data and assertions and made executable by implementing the corresponding test scripts. As a case study, they selected a mobile health IoT system for diabetes management composed of local sensors/actuators, smartphones, and a remote cloud-based system. The effectiveness of the approach has been evaluated by measuring the capability of two test suites implemented using different localisation strategies (visual and structure-based) in detecting mutants of the original m-health system. Results show the effectiveness of the test suites implemented by following the proposed approach since 93% of the generated mutants have been detected.
Continuous experimentation is an up-and-coming technique for requirements engineering and testing, particularly for web-based systems. On the basis of a practitioner survey, this article gives an overview of challenge...
详细信息
Continuous experimentation is an up-and-coming technique for requirements engineering and testing, particularly for web-based systems. On the basis of a practitioner survey, this article gives an overview of challenges, implementation techniques, and current research in the field. This article is part of a theme issue on release engineering.
A feature model is a compact representation of all the products of a software product line. Automated analysis of feature models is rapidly gaining importance: new operations of analysis have been proposed, new tools ...
详细信息
A feature model is a compact representation of all the products of a software product line. Automated analysis of feature models is rapidly gaining importance: new operations of analysis have been proposed, new tools have been developed to support those operations and different logical paradigms and algorithms have been proposed to perform them. Implementing operations is a complex task that easily leads to errors in analysis solutions. In this context, the lack of specific testing mechanisms is becoming a major obstacle hindering the development of tools and affecting their quality and reliability. In this article, the authors present FaMa test suite, a set of implementation-independent test cases to validate the functionality of feature model analysis tools. This is an efficient and handy mechanism to assist in the development of tools, detecting faults and improving their quality. In order to show the effectiveness of their proposal, the authors evaluated the suite using mutation testing as well as real faults and tools. Their results are promising and directly applicable in the testing of analysis solutions. The authors intend this work to be a first step towards the development of a widely accepted test suite to support functional testing in the community of automated analysis of feature models.
暂无评论