检索结果-内蒙古大学图书馆

Structured precision skipping: Accelerating convolutional neural networks with budget-aware dynamic precision selection

引用

JOURNAL OF SYSTEMS architecture 2022年第0期124卷 102403-102403页

作者： Huang, Kai Chen, Siang Li, Bowen Claesen, Luc Yao, Hao Chen, Junjian Jiang, Xiaowen Liu, Zhili Xiong, Dongliang Zhejiang Univ Inst VLSI Design Hangzhou 310030 Peoples R China Univ Hasselt Engn Technol Elect ICT Dept B-3590 Diepenbeek Belgium CSG Digital Grid Res Inst Guangzhou 510670 Peoples R China Sec chip Hangzhou 310030 Peoples R China

Despite the remarkable advancement in various intelligence tasks achieved by convolutional Neural Networks, the massive computation and storage consumption limit applications on resource-constrained devices. Existing works explore to reduce computation cost by leveraging the input-dependent redundancy at runtime. The irregular dynamic sparsity distribution, however, limits the real speedup for dynamic models deployed in traditional neural network accelerators. To solve this problem, we propose an algorithm-architecture co-design, named structured precision skipping (SPS), to exploit the dynamic precision redundancy in statically quantized models. SPS computes most neurons in a lower precision and only a small portion of important neurons in a higher precision to preserve performance. Specifically, we first propose the structured dynamic block to exploit the dynamic sparsity in a structured manner. Based on the block, we then apply a budget-aware training method by inducing a budget regularization to learn the precision skipping under a target resource constraint. Finally, we present an architecture design based on the bit-serial architecture with support for SPS models, where only a predict controller module with small overhead is introduced. Extensive evaluation results demonstrate that SPS can achieve up to 1.5x speedup and 1.4x energy saving on various models and datasets with marginal accuracy loss.

关键词： convolutional neural networks algorithm-architecture co-design Model compression and acceleration Dynamic quantization

来源：评论

学校读者我要写书评

暂无评论

FLNA: An Energy-Efficient Point Cloud Feature Learning Accelerator with Dataflow Decoupling 60

FLNA: An Energy-Efficient Point Cloud Feature Learning Accel...

引用

60th ACM/IEEE design Automation conference (DAC)

作者： Lyu, Dongxu Li, Zhenyu Chen, Yuzhou Xu, Ningyi He, Guanghui Shanghai Jiao Tong Univ Sch Elect Informat & Elect Engn Shanghai Peoples R China Shanghai Jiao Tong Univ AI Inst MoE Key Lab Artificial Intelligence Shanghai Peoples R China Huixi Technol Chongqing Peoples R China

ISBN: (纸本)9798350323481

Grid-based feature learning network plays a key role in recent point-cloud based 3D perception. However, high point sparsity and special operators lead to large memory footprint and long processing latency, posing great challenges to hardware acceleration. We propose FLNA, a novel feature learning accelerator with algorithm-architecture co-design. At algorithm level, the dataflow-decoupled graph is adopted to reduce 86% computation by exploiting inherent sparsity and concat redundancy. At hardware design level, we customize a pipelined architecture with block-wise processing, and introduce transposed SRAM strategy to save 82.1% access power. Implemented on a 40nm technology, FLNA achieves 13.4 - 43.3x speedup over RTX 2080Ti GPU. It rivals the state-of-the-art accelerator by 1.21x energy-efficiency improvement with 50.8% latency reduction.

关键词： Point Cloud Feature Learning Accelerator algorithm-architecture co-design Sparsity Exploitation

来源：评论

学校读者我要写书评

暂无评论

coopMC: algorithm-architecture co-Optimization for Markov Chain Monte Carlo Accelerators 28

CoopMC: Algorithm-Architecture Co-Optimization for Markov Ch...

引用

28th Annual IEEE International Symposium on High-Performance computer architecture (HPCA)

作者： Chai, Yuji Ko, Glenn G. Ting, Wei-Te Mark Bailey, Luke Brooks, David Wei, Gu-Yeon Harvard Univ Cambridge MA 02138 USA Stochastic Inc Atlanta GA 30318 USA

ISBN: (纸本)9781665420273

Bayesian machine learning is useful for applications that may make high-risk decisions with limited, noisy, or unlabeled data, as it provides great data efficiency and uncertainty estimation. Building on previous efforts, this work presents coopMC, an algorithm-architecture co-optimization for developing more efficient MCMC-based Bayesian inference accelerators. coopMC utilizes dynamic normalization (DyNorm), LUT-based exponential kernels (TableExp), and log-domain kernel fusion (LogFusion) to reduce computational precision and shrink ALU area by 7.5x without noticeable reduction in model performance. Also, a Tree-based Gibbs sampler (TreeSampler) improves hardware runtime from O(N) to O(log(N)), an 8.7x speedup, and yields 1.9x better area efficiency than the existing state-of-the-art Gibbs sampling architecture. These methods have been tested on 10 diverse workloads using 3 different types of Bayesian models, demonstrating applicability to many Bayesian algorithms. In an end-to-end case study, these optimizations achieve a 33% logic area reduction, 62% power reduction, and 1.53x speedup over previous state-of-the-art end-to-end MCMC accelerators.

关键词： algorithm-architecture co-design Hardware Accelerator Bayesian Machine Learning Markov Chain Monte Carlo

来源：评论

学校读者我要写书评

暂无评论

ARK: Fully Homomorphic Encryption Accelerator with Runtime Data Generation and Inter-Operation Key Reuse 55

ARK: Fully Homomorphic Encryption Accelerator with Runtime D...

引用

55th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)

作者： Kim, Jongmin Lee, Gwangho Kim, Sangpyo Sohn, Gina Rhu, Minsoo Kim, John Ahn, Jung Ho Seoul Natl Univ Seoul South Korea Korea Adv Inst Sci & Technol Daejeon South Korea

ISBN: (数字)9781665462723

ISBN: (纸本)9781665462723

Homomorphic Encryption (HE) is one of the most promising post-quantum cryptographic schemes that enable privacy-preserving computation on servers. However, noise accumulates as we perform operations on HE-encrypted data, restricting the number of possible operations. Fully HE (FHE) removes this restriction by introducing the bootstrapping operation, which refreshes the data;however, FHE schemes are highly memory-bound. Bootstrapping, in particular, requires loading GBs of evaluation keys and plaintexts from off-chip memory, which makes FHE acceleration fundamentally bottlenecked by the off-chip memory bandwidth. In this paper, we propose ARK, an Accelerator for FHE with Runtime data generation and inter-operation Key reuse. ARK enables practical FHE workloads with a novel algorithm-architecture co-design to accelerate bootstrapping. We first eliminate the off-chip memory bandwidth bottleneck through runtime data generation and inter-operation key reuse. This approach enables ARK to fully exploit on-chip memory by substantially reducing the size of the working set. On top of such algorithmic enhancements, we build ARK microarchitecture that minimizes on-chip data movement through an efficient, alternating data distribution policy based on the data access patterns and a streamlined dataflow organization of the tailored functional units-including base conversion, number-theoretic transform, and automorphism units. Overall, our codesign effectively handles the heavy computation and data movement overheads of FHE, drastically reducing the cost of HE operations, including bootstrapping.

关键词： fully homomorphic encryption domain-specific architecture algorithm-architecture co-design

来源：评论

学校读者我要写书评

暂无评论

DTQAtten: Leveraging Dynamic Token-based Quantization for Efficient Attention architecture

DTQAtten: Leveraging Dynamic Token-based Quantization for Ef...

引用

25th design, Automation and Test in Europe conference and Exhibition (DATE)

作者： Yang, Tao Li, Dongyue Song, Zhuoran Zhao, Yilong Liu, Fangxin Wang, Zongwu He, Zhezhi Jiang, Li Shanghai Jiao Tong Univ Shanghai Peoples R China Shanghai Qi Zhi Inst Shanghai Peoples R China Shanghai Jiao Tong Univ AI Inst MoE Key Lab Artificial Intelligence Shanghai Peoples R China

ISBN: (纸本)9783981926361

Models based on the attention mechanism, i.e. transformers, have shown extraordinary performance in Natural Language Processing (NLP) tasks. However, their memory footprint, inference latency, and power consumption are still prohibitive for efficient inference at edge devices, even at data centers. To tackle this issue, we present an algorithm-architecture co-design with dynamic and mixed-precision quantization, DTQAtten. We present empirically that the tolerance to the noise varies from token to token in attention-based NLP models. This finding leads us to quantize different tokens with mixed levels of bits. Thus, we design a compression framework that (i) dynamically quantizes tokens while they are forwarded in the models and (ii) jointly determines the ratio of each precision. Moreover, due to the dynamic mixed-precision tokens caused by our framework, previous matrix-multiplication accelerators (e.g. systolic array) cannot effectively exploit the benefit of the compressed attention computation. We thus design our accelerator with the variable-speed systolic array (VSSA) and propose an effective optimization strategy to alleviate the pipeline-stall problem in VSSA without hardware overhead. We conduct experiments with existing attention-based NLP models, including BERT and GPT-2 on various language tasks. Our results show that DTQAtten outperforms the previous neural network accelerator Eyeriss by 13.12x in terms of speedup and 3.8x in terms of energy-saving. compared with the state-of-the-art attention accelerator SpAtten, our DTQAtten achieves at least 2.65x speedup and 3.38x energy efficiency improvement.

关键词： transformers domain-specific accelerator dynamic quantization algorithm-architecture co-design

来源：评论

学校读者我要写书评

暂无评论

SpAtten: Efficient Sparse Attention architecture with Cascade Token and Head Pruning 27

SpAtten: Efficient Sparse Attention Architecture with Cascad...

引用

27th IEEE International Symposium on High-Performance computer architecture (HPCA)

作者： Wang, Hanrui Zhang, Zhekai Han, Song MIT EECS Cambridge MA 02139 USA

ISBN: (纸本)9781665422352

The attention mechanism is becoming increasingly popular in Natural Language Processing (NLP) applications, showing superior performance than convolutional and recurrent architectures. However, general-purpose platforms such as CPUs and GPUs are inefficient when performing attention inference due to complicated data movement and low arithmetic intensity. Moreover, existing NN accelerators mainly focus on optimizing convolutional or recurrent models, and cannot efficiently support attention. In this paper, we present SpAtten, an efficient algorithm-architecture co-design that leverages token sparsity, head sparsity, and quantization opportunities to reduce the attention computation and memory access. Inspired by the high redundancy of human languages, we propose the novel cascade token pruning to prune away unimportant tokens in the sentence. We also propose cascade head pruning to remove unessential heads. Cascade pruning is fundamentally different from weight pruning since there is no trainable weight in the attention mechanism, and the pruned tokens and heads are selected on the fly. To efficiently support them on hardware, we design a novel top-k engine to rank token and head importance scores with high throughput. Furthermore, we propose progressive quantization that first fetches MSBs only and performs the computation;if the confidence is low, it fetches LSBs and recomputes the attention outputs, trading computation for memory reduction. Extensive experiments on 30 benchmarks show that, on average, SpAtten reduces DRAM access by 10.0 x with no accuracy loss, and achieves 1.6x, 3.0x, 162x, 347x speedup, and 1.4x, 3.2 x, 1193x, 4059x energy savings over A(3) accelerator, MNNFast accelerator, TITAN Xp GPU, Xeon CPU, respectively.

关键词： Natural Language Processing Attention Domain-Specific Accelerator algorithm-architecture co-design Pruning Quantization

来源：评论

学校读者我要写书评

暂无评论

ARK: Fully Homomorphic Encryption Accelerator with Runtime Data Generation and Inter-Operation Key Reuse 22

ARK: Fully Homomorphic Encryption Accelerator with Runtime D...

引用

Proceedings of the 55th Annual IEEE/ACM International Symposium on Microarchitecture

作者： Jongmin Kim Gwangho Lee Sangpyo Kim Gina Sohn Minsoo Rhu John Kim Jung Ho Ahn Seoul National University Seoul South Korea KAIST Daejeon South Korea

ISBN: (纸本)9781665462723

Homomorphic Encryption (HE) is one of the most promising post-quantum cryptographic schemes that enable privacy-preserving computation on servers. However, noise accumulates as we perform operations on HE-encrypted data, restricting the number of possible operations. Fully HE (FHE) removes this restriction by introducing the bootstrapping operation, which refreshes the data; however, FHE schemes are highly memory-bound. Bootstrapping, in particular, requires loading GBs of evaluation keys and plaintexts from off-chip memory, which makes FHE acceleration fundamentally bottlenecked by the off-chip memory *** this paper, we propose ARK, an Accelerator for FHE with Runtime data generation and inter-operation Key reuse. ARK enables practical FHE workloads with a novel algorithm-architecture co-design to accelerate bootstrapping. We first eliminate the off-chip memory bandwidth bottleneck through runtime data generation and inter-operation key reuse. This approach enables ARK to fully exploit on-chip memory by substantially reducing the size of the working set. On top of such algorithmic enhancements, we build ARK microarchitecture that minimizes on-chip data movement through an efficient, alternating data distribution policy based on the data access patterns and a streamlined dataflow organization of the tailored functional units ---including base conversion, number-theoretic transform, and automorphism units. Overall, our co-design effectively handles the heavy computation and data movement overheads of FHE, drastically reducing the cost of HE operations, including bootstrapping.

关键词： fully homomorphic encryption domain-specific architecture algorithm-architecture co-design

来源：评论

学校读者我要写书评

暂无评论

DTQAtten: leveraging dynamic token-based quantization for efficient attention architecture 22

DTQAtten: leveraging <u>d</u>ynamic <u>t</u>oken-based quant...

引用

Proceedings of the 2022 conference & Exhibition on design, Automation & Test in Europe

作者： Tao Yang Dongyue Li Zhuoran Song Yilong Zhao Fangxin Liu Zongwu Wang Zhezhi He Li Jiang Shanghai Jiao Tong University Shanghai China Shanghai Jiao Tong University Shanghai China and Shanghai Qi Zhi Institute Shanghai China and Shanghai Jiao Tong University

ISBN: (纸本)9783981926361

Models based on the attention mechanism, i.e. transformers, have shown extraordinary performance in Natural Language Processing (NLP) tasks. However, their memory footprint, inference latency, and power consumption are still prohibitive for efficient inference at edge devices, even at data centers. To tackle this issue, we present an algorithm-architecture co-design with dynamic and mixed-precision quantization, DTQAtten. We present empirically that the tolerance to the noise varies from token to token in attention-based NLP models. This finding leads us to quantize different tokens with mixed levels of bits. Thus, we design a compression framework that (i) dynamically quantizes tokens while they are forwarded in the models and (ii) jointly determines the ratio of each precision. Moreover, due to the dynamic mixed-precision tokens caused by our framework, previous matrix-multiplication accelerators (e.g. systolic array) cannot effectively exploit the benefit of the compressed attention computation. We thus design our accelerator with the variable-speed systolic array (VSSA) and propose an effective optimization strategy to alleviate the pipeline-stall problem in VSSA without hardware overhead. We conduct experiments with existing attention-based NLP models, including BERT and GPT-2 on various language tasks. Our results show that DTQAtten outperforms the previous neural network accelerator Eyeriss by 13.12× in terms of speedup and 3.8× in terms of energy-saving. compared with the state-of-the-art attention accelerator SpAtten, our DTQAtten achieves at least 2.65× speedup and 3.38× energy efficiency improvement.

关键词： dynamic quantization transformers algorithm-architecture co-design domain-specific accelerator

来源：评论

学校读者我要写书评

暂无评论

Towards closing the specification gap by integrating algorithm-level and system-level design

引用

design AUTOMATION FOR EMBEDDED SYSTEMS 2015年第4期19卷 389-419页

作者： Zhang, Jiaxing Schirner, Gunar Northeastern Univ Dept Elect & Comp Engn Boston MA 02115 USA

Modern multiprocessor system-on-chip (MPSoC) designs face challenges in tremendous complexity imposed by the wide range of functional and architectural requirements. design automation methodologies address some of the complexity through design abstractions from two different angles, namely functionality and architecture. algorithm-level design (ALD), such as Simulink, focuses on managing the functional complexity through algorithm modeling. Meanwhile, system-level design (SLD) addresses the platform complexity by exploring and synthesizing architecture models from specifications captured in a system-level design language (SLDL). However, these two design methodologies are inherently disjoint as they focus on different design domains at separate abstraction levels. As a result, transition from algorithm models to system-level explorations often requires re-authoring of the functional SLDL specification to expose hierarchy and parallelism. This thus forms the Specification Gap causing a loss of productivity and stalling the overall design cycle. This paper proposes to join ALD and SLD to close the specification gap through a Specification Synthesis approach. We introduce Algo2Spec, which synthesizes an SLDL specification out of an algorithm model in Simulink. Algo2Spec enables a rapid heterogeneous design Space Exploration while still tuning the algorithm according to functional needs. With Algo2Spec, system level design principles propagate up to higher abstraction levels and a new joint algorithm/architecture co-design flow is created. The joint flow seamlessly spans from algorithm modeling down to heterogeneous implementations crossing multiple abstractions. Our approach empowers designers to create, simulate, and explore models in a rapid design cycle. Utilizing the joint flow, we demonstrate opportunities for algorithm and architecture co-design on a set of real-world benchmark applications ranging from 57 to 5733 Simulink blocks. The automatic synthesis avoids t

关键词： Specification synthesis Simulink MPSoC design methodology design space exploration algorithm-architecture co-design

来源：评论

学校读者我要写书评

暂无评论

FPGA architecture for 2D Discrete Fourier Transform Based on 2D Decomposition for Large-sized Data

引用

JOURNAL OF SIGNAL PROCESSING SYSTEMS FOR SIGNAL IMAGE AND VIDEO TECHNOLOGY 2011年第1期64卷 109-122页

作者： Yu, Chi-Li Kim, Jung-Sub Deng, Lanping Kestur, Srinidhi Narayanan, Vijaykrishnan Chakrabarti, Chaitali Arizona State Univ Sch Elect Comp & Energy Engn Tempe AZ 85069 USA Penn State Univ Dept Comp Sci & Engn University Pk PA 16802 USA

Applications based on Discrete Fourier Transforms (DFT) are extensively used in several areas of signal and digital image processing. Of particular interest is the two-dimensional (2D) DFT which is more computation- and bandwidth-intensive than the one-dimensional (1D) DFT. Traditionally, a 2D DFT is computed using Row-column (RC) decomposition, where 1D DFTs are computed along the rows followed by 1D DFTs along the columns. Both application specific and reconfigurable hardware have utilized this scheme for high-performance implementations of 2D DFT. However, architectures based on RC decomposition are not efficient for large input size data due to memory bandwidth constraints. In this paper, we propose an efficient architecture to implement 2D DFT for large-sized input data based on a novel 2D decomposition algorithm. This architecture achieves very high throughput by exploiting the inherent parallelism due to the algorithm decomposition and by utilizing the row-wise burst access pattern of the external memory. A high throughput memory interface has been designed to enable maximum utilization of the memory bandwidth. In addition, an automatic system generator is provided for mapping this architecture onto a reconfigurable platform of Xilinx Virtex-5 devices. For a 2K x2K input size, the proposed architecture is 1.96 times faster than RC decomposition based implementation under the same memory constraints, and also outperforms other existing implementations.

关键词： Multi-dimensional signal processing DFT algorithm-architecture co-design

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：