检索结果-内蒙古大学图书馆

44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)

作者： Braun, Claus Halder, Sebastian Wunderlich, Hans-Joachim Univ Stuttgart Inst Comp Architecture & Comp Engn D-70569 Stuttgart Germany

ISBN: (纸本)9781479922338

Graphics processing units (GPUs) enable large-scale scientific applications and simulations on the desktop. To allow scientific computing on GPUs with high performance and reliability requirements, the application of software-based fault tolerance is attractive. algorithm-based fault tolerance (ABFT) protects important scientific operations like matrix multiplications. However, the application to floating-point operations necessitates the runtime classification of errors into inevitable rounding errors, allowed compute errors in the magnitude of such rounding errors, and into critical errors that are larger than those and not tolerable. Hence, an ABFT scheme needs suitable rounding error bounds to detect errors reliably. The determination of such error bounds is a highly challenging task, especially since it has to be integrated tightly into the algorithm and executed autonomously with low performance overhead. In this work, A-ABFT for matrix multiplications on GPUs is introduced, which is a new, parallel ABFT scheme that determines rounding error bounds autonomously at runtime with low performance overhead and high error coverage.

关键词： algorithm-based fault tolerance Rounding Error Estimation GPU Matrix Multiplication

来源：评论

学校读者我要写书评

暂无评论

ATTNChecker: Highly-Optimized fault Tolerant Attention for Large Language Model Training 25

ATTNChecker: Highly-Optimized Fault Tolerant Attention for L...

引用

30th Symposium on Principles and Practice of Parallel Programming

作者： Liang, Yuhang Li, Xinyi Ren, Jie Li, Ang Fang, Bo Chen, Jieyang Univ Oregon Eugene OR 97403 USA Pacific Northwest Natl Lab Richland WA 99352 USA Coll William & Mary Williamsburg VA USA

ISBN: (纸本)9798400714436

Large Language Models (LLMs) have demonstrated remarkable performance in various natural language processing tasks. However, the training of these models is computationally intensive and susceptible to faults, particularly in the attention mechanism, which is a critical component of transformer-based LLMs. In this paper, we investigate the impact of faults on LLM training, focusing on INF, NaN, and near-INF values in the computation results with systematic fault injection experiments. We observe the propagation patterns of these errors, which can trigger non-trainable states in the model and disrupt training, forcing the procedure to load from checkpoints. To mitigate the impact of these faults, we propose ATTNChecker, the first algorithm-based fault tolerance (ABFT) technique tailored for the attention mechanism in LLMs. ATTNChecker is designed based on fault propagation patterns of LLM and incorporates performance optimization to adapt to both system reliability and model vulnerability while providing lightweight protection for fast LLM training. Evaluations on four LLMs show that ATTNChecker incurs on average 7% overhead on training while detecting and correcting all extreme errors. Compared with the state-of-the-art checkpoint/restore approach, ATTNChecker reduces recovery overhead by up to 49x.

关键词： algorithm-based fault tolerance Attention Mechanism Large Language Models Matrix Multiplication

来源：评论

学校读者我要写书评

暂无评论

Block-checksum-based fault tolerance for Matrix Multiplication on Large-Scale Parallel Systems 20

Block-checksum-based Fault Tolerance for Matrix Multiplicati...

引用

20th IEEE International Conference on High Performance Computing and Communications (HPCC) / 16th IEEE International Conference on Smart City (SmartCity) / 4th IEEE International Conference on Data Science and Systems (DSS)

作者： Zhu, Yanchao Liu, Yi Li, Mingzhen Qian, Depei Beihang Univ Sino German Joint Software Inst Beijing Peoples R China

ISBN: (纸本)9781538666142

With the scaling up of high performance computers, resilience has become a big challenge. Among various kinds of software-based fault-tolerant approaches, the algorithm-based fault tolerance (ABFT) has some attractive characteristics in the era of exa-scale systems, such as high efficiency and light-weight. In particular, considering that many engineering and scientific applications rely on some fundamental algorithms, it is possible to provide algorithm-based fault-tolerant mechanisms in low level and make it application-independent. Previous fault-tolerant mechanisms for matrix computation use row and column checksums, which cannot be directly used in large-scale parallel systems. This paper proposes an algorithm-based fault tolerant approach for matrix multiplication on large-scale parallel systems. The mechanism uses block-checksum which not only meets the requirement of matrix computations on large-scale parallel systems but also reduces the overhead of fault-tolerance compared to traditional schemes based on row and column checksums. In addition, this paper gives method for choosing the size of blocks to achieve balance between accuracy and efficiency. The complexity analysis and examples demonstrate effectiveness and feasibility of our approach.

关键词： algorithm-based fault tolerance large-scale parallel system matrix multiplication matrix computation

来源：评论

学校读者我要写书评

暂无评论

Analysis and randomized design of algorithm-based fault tolerant multiprocessor systems under an extended model

引用

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 1997年第7期8卷 757-768页

作者： Yajnik, S Jha, NK PRINCETON UNIV DEPT ELECT ENGNPRINCETONNJ 08544

Reliability of compute-intensive applications can be improved by introducing fault tolerance into the system. algorithm-based fault tolerance (ABFT) is a low-cost scheme which provides the required fault tolerance to the system through system level encoding. In this paper, we propose randomized construction techniques, under an extended model, for the design of ABFT systems with the required fault tolerance capability. The model considers failures in the processors performing the checking operations.

关键词： algorithm-based fault tolerance concurrent error detection concurrent fault location randomized algorithms fault diagnosis transient faults

来源：评论

学校读者我要写书评

暂无评论

MODIFYING REAL CONVOLUTIONAL-CODES FOR PROTECTING DIGITAL FILTERING SYSTEMS

引用

IEEE TRANSACTIONS ON INFORMATION THEORY 1993年第2期39卷 553-564页

作者： REDINBO, GR ZAGAR, B GRAZ TECH UNIV INST ALLEGEMEINE ELEKTROTECH & ELEKT MESSTECHA-8010 GRAZAUSTRIA

Digital filters when implemented with very dense high-speed electronic devices are susceptible to both temporary and permanent failures, not easily protected by conventional fault-tolerant computer design principles. A new method is presented for protecting the overall realization against both hard and soft errors at the data sample level using the error-detecting properties of real convolutional codes. The normal filter system is surrounded with parallel parity channels defined by a real systematic convolutional code. Erroneous behavior is detected by comparing externally the calculated and regenerated parity samples. A rate k/n real convolutional code produces (n - k) parity samples for every k input samples causing the parity channels to operate at a rate decimated by k. Significant complexity reductions are possible by modifying the code structure, without loss of error protection, yielding simplified parity channels with finite impulse response (FIR) structures with computational rates decimated by k. The code modification procedure determines k nonzero scaling coefficients for weighting respective rows of the convolutional code's generator matrix. A matrix equation involving the code's original parity values and the denominator polynomial of the digital filter's transfer function is formed. Row manipulations separate this equation into two parts, a set of homogeneous equations constraining the modifying scaling coefficients and another set defining the implementation of the code parity values. An annihilator subspace in the dual space related to a condensed matrix defines the family of acceptable scaling coefficients when the code parameter k is less then the number of parity values, (n - k), times the number of poles in the filter. The code modification process has been automated in a computer algorithm. The effects of parity filter quantizations are analyzed and a bound on the mean-square error in the parity comparisons is given.

关键词： REAL CONVOLUTIONAL CODES fault-TOLERANT PROCESSING algorithm-based fault tolerance PROTECTING DIGITAL FILTERING ANNIHILATOR SUBSPACES

来源：评论

学校读者我要写书评

暂无评论

Exploiting data representation for fault tolerance

引用

JOURNAL OF COMPUTATIONAL SCIENCE 2016年 14卷 51-60页

作者： Elliott, J. Hoemmen, M. Mueller, F. North Carolina State Univ Dept Comp Sci Raleigh NC 27695 USA Sandia Natl Labs Ctr Res Comp POB 5800 Albuquerque NM 87185 USA

Incorrect computer hardware behavior may corrupt intermediate computations in numerical algorithms, possibly resulting in incorrect answers. Prior work models misbehaving hardware by randomly flipping bits in memory. We start by accepting this premise, and present an analytic model for the error introduced by a bit flip in an IEEE 754 floating-point number. We then relate this finding to the linear algebra concepts of normalization and matrix equilibration. In particular, we present a case study illustrating that normalizing both vector inputs of a dot product minimizes the probability of a single bit flip causing a large error in the dot product's result. Furthermore, the absolute error is either less than one or very large, which allows detection of large errors. Then, we apply this to the GMRES iterative solver. We count all possible errors that can be introduced through faults in arithmetic in the computationally intensive orthogonalization phase of GMRES, and show that when the matrix is equilibrated, the absolute error is bounded above by one. (C) 2016 Elsevier B.V. All rights reserved.

关键词： algorithm-based fault tolerance Resilient algorithms Numerical methods

来源：评论

学校读者我要写书评

暂无评论

DIAGNOSABILITY AND DIAGNOSIS OF algorithm-based fault-TOLERANT SYSTEMS

引用

IEEE TRANSACTIONS ON COMPUTERS 1993年第8期42卷 924-937页

作者： VINNAKOTA, B JHA, NK PRINCETON UNIV DEPT ELECT ENGN PRINCETON NJ 08544 USA

Parallel processing architectures are now in common use for signal processing and other computation-intensive applications. These applications are characterized by high throughput and long processing periods. Such characteristics decrease the reliability of high-performance architectures. The erroneous data produced by faulty processors could have damaging consequences, particularly in critical real-time applications. It is therefore desirable that any erroneous data produced by the system be detected and located as quickly as possible. algorithm-based fault tolerance (ABFT) is a low-cost system-level concurrent error detection and fault location scheme. We apply methods used in the analysis of multiprocessor systems employing system-level diagnosis to the analysis of ABFT systems. A new algorithm to analyze an ABFT system for its fault diagnosability is developed using these methods. based on this work, a fault diagnosis algorithm is developed for ABFT systems. No such algorithm has been presented previously.

关键词： algorithm-based fault tolerance CHECKSUM ENCODING CONCURRENT ERROR DETECTION CONCURRENT fault DIAGNOSIS fault DIAGNOSABILITY

来源：评论

学校读者我要写书评

暂无评论

Rollback-Free Recovery for a High Performance Dense Linear Solver With Reduced Memory Footprint

引用

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2024年第7期35卷 1307-1319页

作者： Loreti, Daniela Artioli, Marcello Ciampolini, Anna Univ Bologna Dept Comp Sci & Engn I-40126 Bologna Italy Italian Natl Agcy New Technol Energy & Sustainable I-40129 Bologna Italy

The scale of nowadays High Performance Computing (HPC) systems is the key element that determines the achievement of impressive performance, as well as the reason for their relatively limited reliability. Over the last decade, specific areas of the High Performance Computing (HPC) research field have addressed the issue at different levels, by enriching the infrastructure, the platforms, or the algorithms with fault tolerance features. In this work, we focus on the rather-pervasive task of computing the solution of a dense, unstructured linear system and we propose an algorithm-based technique to obtain fault tolerance to multiple anywhere-located faults during the parallel computation. We particularly study the ways to boost the performance of the rollback-free recovery, and we provide an extensive evaluation of our technique w.r.t. to other state-of-the-art algorithm-based methods.

关键词： fault tolerant systems fault tolerance Linear systems Circuit faults Vectors Program processors Task analysis Rollback-free recovery algorithm-based fault tolerance high performance computing linear systems solver

来源：评论

学校读者我要写书评

暂无评论

A backward/forward recovery approach for the preconditioned conjugate gradient method

引用

JOURNAL OF COMPUTATIONAL SCIENCE 2016年第Part3期17卷 522-534页

作者： Fasi, Massimiliano Langou, Julien Robert, Yves Ucar, Bora Univ Manchester Manchester M13 9PL Lancs England Univ Colorado Denver Denver CO USA ENS Lyon Lyon France Univ Tennessee Knoxville TN USA Univ Lyon INRIA CNRS LIPUMR5668ENS LyonUCBL Lyon France

Several recent papers have introduced a periodic verification mechanism to detect silent errors in iterative solvers. Chen (2013, pp. 167-176) has shown how to combine such a verification mechanism (a stability test checking the orthogonality of two vectors and recomputing the residual) with checkpointing: the idea is to verify every d iterations, and to checkpoint every c x d iterations. When a silent error is detected by the verification mechanism, one can rollback to and re-execute from the last checkpoint. In this paper, we also propose to combine checkpointing and verification, but we use algorithm-based fault tolerance (ABFT) rather than stability tests. ABFT can be used for error detection, but also for error detection and correction, allowing a forward recovery (and no rollback nor re-execution) when a single error is detected. We introduce an abstract performance model to compute the performance of all schemes, and we instantiate it using the preconditioned conjugate gradient algorithm. Finally, we validate our new approach through a set of simulations. (C) 2016 Elsevier B.V. All rights reserved.

关键词： fault-tolerance Silent errors algorithm-based fault tolerance Checkpointing Sparse matrix-vector multiplication Preconditioned conjugate gradient method

来源：评论

学校读者我要写书评

暂无评论

Tests and tolerances for high-performance software-implemented fault detection

引用

IEEE TRANSACTIONS ON COMPUTERS 2003年第5期52卷 579-591页

作者： Turmon, M Granat, R Katz, DS Lou, JZ Jet Prop Lab Data Understanding Syst Grp Pasadena CA 91109 USA Jet Prop Lab Parallel Applicat Technol Grp Pasadena CA 91109 USA

We describe and test a software approach to fault detection in common numerical algorithms. Such result checking or algorithm-based fault tolerance (ABFT) methods may be used, for example, to overcome single-event upsets in computational hardware or to detect errors in complex, high-efficiency implementations of the algorithms. Following earlier work, we use checksum methods to validate results returned by a numerical subroutine operating subject to unpredictable errors in data. We consider common matrix and Fourier algorithms which return results satisfying a necessary condition having a linear form;the checksum tests compliance with this condition. We discuss the theory and practice of setting numerical tolerances to separate errors caused by a fault from those inherent in finite-precision floating-point calculations. We concentrate on comprehensively defining and evaluating tests having various accuracy/computational burden tradeoffs, and we emphasize average-case algorithm behavior rather than using worst-case upper bounds on error.

关键词： algorithm-based fault tolerance result checking error analysis aerospace parallel numerical algorithms

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：