The n-version machine learning (ML) system is an architecture approach to enhance the reliability of ML system outputs by exploiting ML model diversity and input data diversity. While existing studies theoretically sh...
详细信息
ISBN:
(纸本)9798350315943
The n-version machine learning (ML) system is an architecture approach to enhance the reliability of ML system outputs by exploiting ML model diversity and input data diversity. While existing studies theoretically show the relation between diversity metrics and system reliability, there is a shortage of empirical studies validating reliability models with diversity parameters in real datasets. In this paper, focusing on traffic sign recognition tasks, we empirically analyze the impact of diversity parameter estimations for predicting the reliability of three-version traffic sign classifier systems. Using five real-world traffic sign datasets, we confirm that the three-version architecture effectively enhances system reliability by applying diverse models and diversified input images. Then, we estimate the diversity parameters and apply them to variants of reliability prediction models. The prediction residuals between the observed reliability and the predicted reliability are mostly less than 0.017 across all data sets, which is half of the residual achieved by the conventional prediction model, except for the architecture of a single model with triple input. As the estimated values of diversity parameters tend to be stable with a relatively small number of samples, we consider that the reliability prediction models using diversity parameters are useful in the early-stage design of ML systems.
Machine learning (ML) is widely used in intelligent software systems. However, the uncertain outputs from ML models can lead to undesirable consequences in safety-critical applications. To improve system reliability, ...
详细信息
ISBN:
(纸本)9798350325454
Machine learning (ML) is widely used in intelligent software systems. However, the uncertain outputs from ML models can lead to undesirable consequences in safety-critical applications. To improve system reliability, we propose n-version ML architectures combining multiple inputs with multiple ML models to decide the system output by voting. The reliability of n-version ML systems can be characterized by two diversity measures;input diversity and model diversity. In this study, we consider Bayesiannetworks (Bns) for modeling the reliability of n-version ML systems outputs through multiple dependent diversity parameters. We present a preliminary Bns reliability model for a three-version ML system. Finally, we discuss the potential extension of the approach and issues for modeling large-scale systems.
Machine learning (ML) is extensively employed in AI-powered systems including safety-critical applications such as autonomous vehicles. The outputs from ML models are sensitive to real-world input data and error-prone...
详细信息
Machine learning (ML) is extensively employed in AI-powered systems including safety-critical applications such as autonomous vehicles. The outputs from ML models are sensitive to real-world input data and error-prone, thereby improving the reliability of ML systems' outputs has become a critical challenge in ML system design. In this paper, we introduce n-version ML architectures to enhance the ML system reliability and propose Bayesiannetworks (Bns) models to evaluate the reliability of system outputs targeting three-version ML systems. The proposed Bn reliability models allow us to formulate five distinct types of three-version ML architectures that are composed of diverse models and diverse input data sources. To validate the Bn reliability models with real samples from ML systems, we conduct empirical studies on traffic sign recognition tasks and evaluate prediction performance. As a result, we find the prediction residuals between the observed reliability and the predicted reliability by the Bn reliability models are less than 0.015 across all data sets, which is much better than the prediction performance by the baseline model. In addition, in comparison to the previous reliability models without exploiting Bns, the proposed models exhibit an advantage in reliability prediction, except for the triple model with single input architecture.
Machine learning (ML) models have been widely applied to real-world systems. However, outputs of ML models are generally uncertain and sensitive to real input data, which is a big challenge in designing highly reliabl...
详细信息
ISBN:
(数字)9781665421416
ISBN:
(纸本)9781665421416
Machine learning (ML) models have been widely applied to real-world systems. However, outputs of ML models are generally uncertain and sensitive to real input data, which is a big challenge in designing highly reliable ML-based software systems. Our study aims to improve the ML system reliability through a software architecture approach inspired by n-version programmingn-version ML architectures considered in our study combine multiple input data sets with multiple versions of ML models to determine the final system output by consensus. In this paper, we focus on three-version ML architectures and propose the reliability models for analyzing the system reliability by using diversity metrics for ML models and input data sets. The proposed model allows us to compare the reliability of a triple-model with triple-input (TMTI) architecture with other variants of three-version and two-version architectures. Through the numerical analysis of the proposed models, we find that i) the reliability of TMTI architecture is higher than other three-version architectures, but interestingly ii) it is generally lower than the reliability of double model with double input system (DMDI). Furthermore, we also find that a larger variance of model diversities negatively impacts the TMTI reliability, while a larger variance of input diversity has opposed impacts.
As one of the enabling technologies for cyber-physical systems and Internet of Things systems, the cloud computing provides cost-effective resources in an on-demand manner. This merit lends the cloud to running critic...
详细信息
As one of the enabling technologies for cyber-physical systems and Internet of Things systems, the cloud computing provides cost-effective resources in an on-demand manner. This merit lends the cloud to running critical services that need redundancy to achieve high reliability. This paper models a cloud service using the nversionprogramming (nVP) redundancy technique that creates and runs multiple task solver versions (TSVs) in parallel to perform a requested service and decides the output using the threshold voting. A malicious attacker may get an unauthorized access to a user's data when the user's and attacker's virtual machines co-reside in the same cloud server. To reduce the chance of the co-residence attack success and users' expense, an individual TSV cancellation policy is implemented, which removes a TSV's virtual machine from its host server immediately once this TSV completes the task execution. A probabilistic method is proposed to evaluate the task reliability and data security under the considered cloud service model. Constrained optimization problems are further formulated and solved, which find the optimal number of TSVs maximizing the task reliability subject to providing a desired level of data security. Examples are presented to demonstrate interactions and impacts of different parameters on the task reliability and data security, as well as on the optimization solutions.
With the development of cloud services, cloud servers must provide a safe and reliable cloud environment. To defend co-resident attack launched by malicious cloud users who co-resident with normal users on the same ph...
详细信息
Research and development teams have become increasingly focused on developing highly reliable software for safety-critical systems. It is a major challenge for real-time control systems to achieve high reliability sof...
详细信息
Research and development teams have become increasingly focused on developing highly reliable software for safety-critical systems. It is a major challenge for real-time control systems to achieve high reliability software to meet safety standards. A reliability evaluation focuses primarily on analytical and modeling techniques for fault prediction. In safety-critical systems like nuclear plant controls, aircraft controls and railroad signalization systems, n-version programming (nVP) is an effective technique for raising software's reliability, particularly in areas with high-risk ratios because small errors can result in hazardous incidents. It allows the software to be fault-tolerant, aiding it to produce accurate results even when the software has faults. We present an analytical method for assessing the reliability of n-version software systems. Analysis of the system's reliability and other performance metrics is provided with closed-form expressions. As an additional extension, we conduct numerical analyses of two cases, the 2VP system and 3VP system, in which suitable parameters are used. We conduct numerical simulations using MATLAB to generate the analytical results and compare the analytical results by using numerical results and neuro-fuzzy results using fuzzy interference systems.
Software requirement specifications have been observed to largely impact the dependability and the cost of software systems in software development and certification phases. Inappropriate specification of software req...
详细信息
ISBN:
(纸本)9781665426039
Software requirement specifications have been observed to largely impact the dependability and the cost of software systems in software development and certification phases. Inappropriate specification of software requirements can cause software developers' erroneous mental representations, thus leading to defects that propagate into subsequent development phases. Understanding the human error mechanisms of software requirement representation is significant for reducing the defects originated from requirements. This paper proposes a theory on the human error mechanism of software requirement, and derived two new criteria to avoid requirement specification triggering the human errors of developers. The criteria were validated by an experiment. Results show that: 1) once a requirement specification contained the error-prone scenarios of the two proposed criteria, developers indeed committed corresponding errors;2) violating the proposed criteria tended to cause common defects, which are the defects introduced by two or more developers in the same way.
Although graph-databases have been assuming an increasing relevance in applications that exhibit strong dependability requirements, including tolerance to malicious faults, few works have addressed Byzantine fault tol...
详细信息
Although graph-databases have been assuming an increasing relevance in applications that exhibit strong dependability requirements, including tolerance to malicious faults, few works have addressed Byzantine fault tolerance in this particular context, and previous attempts suffer from lack of flexibility and poor performance. This article describes and evaluates Fireplug, a flexible architecture to build robust geo-replicated graph databases. Fireplug can be configured to tolerate from crash to Byzantine faults, both within and across different datacenters. Furthermore, Fireplug is robust to bugs in existing graph database implementations, as it allows to combine multiple graph database instances in a cohesive manner. Thus, Fireplug can support many different deployments, according to the performance/robustness trade-offs imposed by the target application. Our evaluation shows that Fireplug is able implement Byzantine fault tolerance without penalty when compared to the built-in replication mechanism of neo4j, which only supports crash faults. Additionally, performance optimizations introduced by Fireplug improve the overall performance by up to 900 percent in geo-replicated scenarios.
The concept of functional safety gains importance with the increasing number of hazardous accidents in the railway industry. In literature, some hardware and software architectures are proposed for the functional safe...
详细信息
The concept of functional safety gains importance with the increasing number of hazardous accidents in the railway industry. In literature, some hardware and software architectures are proposed for the functional safety. As n-version programming is getting popular as preferred software architecture in railway industry, the effect of various hardware implementations of n-version programming on the functional safety remains unclear. In this study, two different hardware setups will be evaluated for n-version programming. After the effect of these hardware setups on the functional safety is analyzed, the effects on the hardware usage and overall response time will be tested on a sample train station.
暂无评论