检索结果-内蒙古大学图书馆

Accelerating CNN Training With Concurrent Execution of GPU and processing-in-memory

IEEE ACCESS 2024年 12卷 160190-160204页

作者： Choi, Jungwoo Lee, Hyuk-Jae Sohn, Kyomin Yu, Hak-Soo Rhee, Chae Eun Seoul Natl Univ Interuniv Semicond Res Ctr ISRC Dept Elect Engn & Comp Sci Seoul 08826 South Korea Samsung Elect Hwaseong Si 18448 South Korea Hanyang Univ Dept Elect Engn Seoul 04763 South Korea

Training of convolutional neural networks (CNN) consumes a lot of time and resources. While most previous works have focused on accelerating the convolutional (CONV) layer, the proportion of non-convolutional (non-CONV) layers, such as batch normalization, is gradually increasing during training. Non-CONV layers have low cache reuse and arithmetic intensity, thereby performance is limited by memory bandwidth. processing-in-memory (PIM) can utilize wide memory bandwidth, making it suitable for acceleration of non-CONV layers. Therefore, it makes sense to perform the computationally complex CONV layer on the host and handle the memory bottleneck challenges of the non-CONV layer on the PIM. Further improved performance can be expected if they run simultaneously. However, memory access conflicts between the host and PIM are the biggest factors hindering performance improvement. Prior studies proposed bank partitioning to alleviate memory conflicts, but it is not effective because CNN training involves significant data sharing between CONV and non-CONV layers. In this paper, we propose a memory scheduling and CNN training flow for the pipelined execution of CONV layers on the host and non-CONV layers on PIM. First, instead of applying bank partitioning, the host and PIM exclusively access memory for a certain period to avoid the movement of shared data between host memory and PIM memory. The conditions for switching the memory access authority between the host and PIM are set per layer, taking into account memory access characteristics and the number of queued memory requests. Second, in the training flow, CONV and non-CONV layers are pipelined in units of output feature map channels. Specifically, for the backward pass, the non-CONV tasks of the feature map gradient calculation phase and the weight gradient update phase are rearranged so that they can be easily performed within CONV layers. Experimental results show that the proposed pipelined execution achieves an aver

关键词： Training Graphics processing units Convolutional neural networks Pipelines Electric breakdown Bandwidth Batch normalization Switches Scheduling algorithms Random access memory processing-in-memory convolutional neural networks neural network training GPU

来源：评论

学校读者我要写书评

暂无评论

A Novel ReRAM-Based processing-in-memory Architecture for Graph Traversal

引用

ACM TRANSACTIONS ON STORAGE 2018年第1期14卷 9-9页

作者： Han, Lei Shen, Zhaoyan Liu, Duo Shao, Zili Huang, H. Howie Li, Tao Hong Kong Polytech Univ Dept Comp Mong ManWai Bldg Hong Kong Hong Kong Peoples R China Chongqing Univ Coll Comp Sci 174 Shazhengjie Chongqing Peoples R China George Washington Univ Dept Elect & Comp Engn 801 22nd St NW Washington DC USA Univ Florida Dept Elect & Comp Engn 339D Larsen Hall Gainesville FL USA

Graph algorithms such as graph traversal have been gaining ever-increasing importance in the era of big data. However, graph processing on traditional architectures issues many random and irregular memory accesses, leading to a huge number of data movements and the consumption of very large amounts of energy. To minimize the waste of memory bandwidth, we investigate utilizing processing-in-memory (PIM), combined with non-volatile metal-oxide resistive random access memory (ReRAM), to improve both computation and I/O performance. We propose a new ReRAM-based processing-in-memory architecture called RPBFS, in which graph data can be persistently stored and processed in place. We study the problem of graph traversal, and we design an efficient graph traversal algorithm in RPBFS. Benefiting from low data movement overhead and high bank-level parallel computation, RPBFS shows a significant performance improvement compared with both the CPU-based and the GPU-based BFS implementations. On a suite of real-world graphs, our architecture yields a speedup in graph traversal performance of up to 33.8x, and achieves a reduction in energy over conventional systems of up to 142.8x.

关键词： ReRAM BFS processing-in-memory architecture

来源：评论

学校读者我要写书评

暂无评论

A programmable shared-memory system for an array of processing-in-memory devices

引用

CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS 2019年第2期22卷 385-398页

作者： Lee, Sangkuen Sim, Hyogi Kim, Youngjae Vazhkudai, Sudharshan S. Oak Ridge Natl Lab 1 Bethel Valley Rd Oak Ridge TN USA Oak Ridge Natl Lab NCCS Technol Integrat TechInt Grp 1 Bethel Valley Rd Oak Ridge TN USA Sogang Univ Dept Comp Sci & Engn Off AS911 35 Baekbeomro Seoul 04107 South Korea

processing in memory (PIM), the concept of integrating processing directly with memory has been attracting a lot of attention, since PIM can assist in overcoming the throughput limitation caused by data movement between CPU and memory. The challenge, however, is that it requires the programmers to have a deep understanding of the PIM architecture to maximize the benefits such as data locality and parallel thread execution on multiple PIM devices. In this study, we present AnalyzeThat, a programmable shared-memory system for parallel data processing with PIM devices. Thematic to AnalyzeThat is a rich PIM-aware data structure (PADS), which is an encapsulation that integrally ties together the data, the analysis tasks and the runtime needed to interface with the PIM device array. The PADS abstraction provides (i) a sophisticated key-value data container that allows programmers to easily store data on multiple PIMs, (ii) a suite of parallel operations with which users can easily implement data analysis applications, and (iii) a runtime, hidden to programmers, which provides the mechanisms needed to overlay both the data and the tasks on the PIM device array in an intelligent fashion, based on PIM-specific information collected from the hardware. We have developed a PIM emulation framework called AnalyzeThat. Our experimental evaluation with representative data analytics applications suggests that the proposed system can significantly reduce the PIM programming effort without losing its technology benefits.

关键词： Programmable devices Storage systems processing-in-memory Big data processing

来源：评论

学校读者我要写书评

暂无评论

Benchmarking a New Paradigm: Experimental Analysis and Characterization of a Real processing-in-memory System

引用

IEEE ACCESS 2022年 10卷 52565-52608页

作者： Gomez-Luna, Juan El Hajj, Izzat Fernandez, Ivan Giannoula, Christina Oliveira, Geraldo F. Mutlu, Onur Swiss Fed Inst Technol Dept Informat Technol & Elect Engn CH-8092 Zurich Switzerland Amer Univ Beirut Dept Comp Sci Beirut 11072020 Lebanon Univ Malaga Dept Comp Architecture Malaga 29016 Spain Natl Tech Univ Athens Sch Elect & Comp Engn GR-10682 Athens Greece

Many modern workloads, such as neural networks, databases, and graph processing, are fundamentally memory-bound. For such workloads, the data movement between main memory and CPU cores imposes a significant overhead in terms of both latency and energy. A major reason is that this communication happens through a narrow bus with high latency and limited bandwidth, and the low data reuse in memory-bound workloads is insufficient to amortize the cost of main memory access. Fundamentally addressing this data movement bottleneck requires a paradigm where the memory system assumes an active role in computing by integrating processing capabilities. This paradigm is known as processing-in-memory (PIM). Recent research explores different forms of PIM architectures, motivated by the emergence of new 3D-stacked memory technologies that integrate memory with a logic layer where processing elements can be easily placed. Past works evaluate these architectures in simulation or, at best, with simplified hardware prototypes. In contrast, the UPMEM company has designed and manufactured the first publicly-available real-world PIM architecture. The UPMEM PIM architecture combines traditional DRAM memory arrays with general-purpose in- order cores, called DRAM processing Units (DPUs), integrated in the same chip. This paper provides the first comprehensive analysis of the first publicly-available real-world PIM architecture. We make two key contributions. First, we conduct an experimental characterization of the UPMEM-based PIM system using microbenchmarks to assess various architecture limits such as compute throughput and memory bandwidth, yielding new insights. Second, we present PrIM (processing-in-memory benchmarks), a benchmark suite of 16 workloads from different application domains (e.g., dense/sparse linear algebra, databases, data analytics, graph processing, neural networks, bioinformatics, image processing), which we identify as memory-bound. We evaluate the performance and

关键词： Computer architecture Benchmark testing Random access memory Graphics processing units Hardware Software Energy consumption processing-in-memory near-data processing memory systems data movement bottleneck DRAM benchmarking real-system characterization workload characterization

来源：评论

学校读者我要写书评

暂无评论

CPSAA: Accelerating Sparse Attention Using Crossbar-Based processing-in-memory Architecture

引用

IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS 2024年第6期43卷 1741-1754页

作者： Li, Huize Jin, Hai Zheng, Long Liao, Xiaofei Huang, Yu Liu, Cong Xu, Jiahong Duan, Zhuohui Chen, Dan Gui, Chuangyi Huazhong Univ Sci & Technol Natl Engn Res Ctr Big Data Technol & Syst Sch Comp Sci & Technol Serv Comp Technol & Syst LabCluster & Grid Comp L Wuhan 430074 Peoples R China

The attention-based neural network attracts great interest due to its excellent accuracy enhancement. However, the attention mechanism requires huge computational efforts to process unnecessary calculations, significantly limiting the system's performance. To reduce the unnecessary calculations, researchers propose sparse attention to convert some dense-dense matrices multiplication (DDMM) operations to sampled dense-dense matrix multiplication (SDDMM) and sparse matrix multiplication (SpMM) operations. However, current sparse attention solutions introduce massive off-chip random memory access since the sparse attention matrix is generally unstructured. We propose CPSAA, a novel crossbar-based processing-in-memory (PIM)-featured sparse attention accelerator to eliminate off-chip data transmissions. 1) We present a novel attention calculation mode to balance the crossbar writing and crossbar processing latency. 2) We design a novel PIM-based sparsity pruning architecture to eliminate the pruning phase's off-chip data transfers. 3) Finally, we present novel crossbar-based SDDMM and SpMM methods to process unstructured sparse attention matrices by coupling two types of crossbar arrays. Experimental results show that CPSAA has an average of 89.6x , 32.2x , 17.8x , 3.39x , and 3.84x performance improvement and 755.6x , 55.3x , 21.3x , 5.7x , and 4.9x energy-saving when compare with GPU, field programmable gate array, SANGER, ReBERT, and ReTransformer.

关键词： Sparse matrices Computer architecture Matrix converters Microprocessors Field programmable gate arrays Virtual machine monitors Hardware Attention mechanism domain-specific accelerator processing-in-memory resistive random access memory (ReRAM)

来源：评论

学校读者我要写书评

暂无评论

Making Better Use of processing-in-memory Through Potential-Based Task Offloading

引用

IEEE ACCESS 2020年 8卷 61631-61641页

作者： Kim, Byoung-Hak Rhee, Chae Eun Inha Univ Dept Informat & Commun Engn Incheon 22212 South Korea

There is an increasing demand for a novel computing structure for data-intensive applications such as artificial intelligence and virtual reality. The processing-in-memory (PIM) is a promising alternative to reduce the overhead caused by data movement. Many studies have been conducted on the utilization of the PIM taking advantage of the bandwidth increased by the through silicon via (TSV). One approach is to design an optimized PIM architecture for a specific application, the other is to find the tasks that will be more advantageous when offloading to PIM. The goal of this paper is to make the PIM, a newly introduced technology, be easily applied to various applications. The programmable GPU-based PIM is the target system. The essential but simple task offloading conditions are proposed to secure as many candidate tasks as possible when there is any potential benefit from the PIM. The PIM design options then are explored reflecting the characteristics of the candidate tasks actively. When determining offloading conditions, it is difficult to simultaneously consider three time-energy-power objectives. Thus, the problem is divided into two sub-problems. The first offloading condition is designed based on time-energy constraints, whereas the second offloading condition is modeled to satisfy time-power constraints. During the whole processes, the offloading conditions and the PIM design options are carefully configured in a complementary manner to reduce the tasks that are excluded from the offloading. In the simulation results, the suitability of the modeled two offloading conditions and the proposed PIM design are verified using various benchmarks and then, they are compared with previous works in terms of processing speed and energy.

关键词： High bandwidth memory near-data-processing processing-in-memory task offloading

来源：评论

学校读者我要写书评

暂无评论

RADAR: A Skew-Resistant and Hotness-Aware Ordered Index Design for processing-in-memory Systems

引用

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2024年第9期35卷 1598-1614页

作者： Hua, Yifan Zheng, Shengan Kong, Weihan Zhou, Cong Huang, Kaixin Ma, Ruoyan Huang, Linpeng Shanghai Jiao Tong Univ Sch Elect Informat & Elect Engn Shanghai 200240 Peoples R China Shanghai Jiao Tong Univ Sch Elect Informat & Elect Engn AI Inst MoEKey Lab Artificial Intelligence Shanghai 200240 Peoples R China

Pointer chasing becomes the performance bottleneck for today's in-memory indexes due to the memory wall. Emerging processing-in-memory (PIM) technologies are promising to mitigate this bottleneck, by enabling low-latency memory access and aggregated memory bandwidth scaling with the number of PIM modules. Prior PIM-based indexes adopt a fixed granularity to partition the key space and maintain static heights of skiplist nodes among PIM modules to accelerate index operations on skiplist, neglecting the changes in skewness and hotness of data access patterns during runtime. In this article, we present RADAR, an innovative PIM-friendly skiplist that dynamically partitions the key space among PIM modules to adapt to varying skewness. An offline learning-based model is employed to catch hotness changes to adjust the heights of skiplist nodes. In multiple datasets, RADAR achieves up to 198.2x performance improvement and consumes 47.4% less memory than state-of-the-art designs on real PIM hardware.

关键词： Indexes Parallel processing Bandwidth Random access memory Aggregates Throughput Spaceborne radar processing-in-memory ordered index pointer chasing load balance index partition

来源：评论

学校读者我要写书评

暂无评论

A Binary-Activation, Multi-Level Weight RNN and Training Algorithm for ADC-/DAC-Free and Noise-Resilient processing-in-memory Inference With eNVM

引用

IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTING 2023年第2期11卷 292-302页

作者： Ma, Siming Brooks, David Wei, Gu-Yeon Harvard Univ Sch Engn & Appl Sci Cambridge MA 02138 USA

We propose a new algorithm for training neural networks with binary activations and multi-level weights, which enables efficient processing-in-memory circuits with embedded nonvolatile memories (eNVM). Binary activations obviate costly DACs and ADCs. Multi-level weights leverage multi-level eNVM cells. Compared to existing algorithms, our method not only works for feed-forward networks (e.g., fully-connected and convolutional), but also achieves higher accuracy and noise resilience for recurrent networks. In particular, we present an RNN-based trigger-word detection PIM accelerator, with detailed hardware noise models and circuit co-design techniques, and validate our algorithm's high inference accuracy and robustness against a variety of real hardware non-idealities.

关键词： Embedded non-volatile memory in-memory-computing processing-in-memory recurrent neural networks trigger word detection

来源：评论

学校读者我要写书评

暂无评论

Parasitic-Aware Modeling and Neural Network Training Scheme for Energy-Efficient processing-in-memory With Resistive Crossbar Array

引用

IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS 2022年第2期12卷 436-444页

作者： Cao, Tiancheng Liu, Chen Gao, Yuan Goh, Wang Ling Nanyang Technol Univ Sch Elect & Elect Engn Singapore 639798 Singapore ASTAR Inst Microelect IME Singapore 138634 Singapore

This paper presents a parasitic-aware modeling method for fast and accurate simulation of processing-in-memory (PIM) neural network (NN) implemented in resistive memristor crossbar array. This work proposed an efficient and accurate crossbar line resistance estimation model named gamma-compact model, and the associated NN training scheme that takes the impact of line resistance into consideration with time complexity of O(mn) for a mxn resistive crossbar. The impact of the crossbar array parasitics to the vector-matrix multiplication (VMM) computation accuracy and multi-layer PIM NN inference accuracy are analyzed in detail. The advantage of the proposed model is demonstrated by a multilayer perceptron (MLP) of size 784 x 128 x 10 implemented with resistive random access memory (RRAM) crossbar arrays for MNIST hand-written digits classification. The proposed method reduced the VMM computation error by 186 and 17 times compared to uncompensated method and the state-of-art compensation method, respectively. Maximum 98.1% inference accuracy is achieved with only 0.17% degradation compared to the ideal model.

关键词： processing-in-memory resistive memristor crossbar interconnect line resistance neuromorphic system

来源：评论

学校读者我要写书评

暂无评论

In-depth Survey of processing-in-memory Architectures for Deep Neural Networks

引用

JOURNAL OF SEMICONDUCTOR TECHNOLOGY AND SCIENCE 2023年第5期23卷 322-339页

作者： Jang, Ji-Hoon Shin, Jin Park, Jun-Tae Hwang, In-Seong Kim, Hyun Seoul Natl Univ Sci & Technol Res Ctr Elect & Informat Technol Dept Elect & Informat Engn Seoul 01811 South Korea

processing-in-memory (PIM) is an emerging computing architecture that has gained significant attention in recent times. It aims to maximize data movement efficiency by moving away from the traditional von Neumann architecture. PIM is particularly well-suited for handling deep neural networks (DNNs) that require significant data movement between the processing unit and the memory device. As a result, there has been substantial research in this area. To optimally handle DNNs with diverse structures and inductive biases, such as convolutional neural networks, graph convolutional networks, recurrent neural networks, and transformers, within a PIM architecture, careful consideration should be given to how data mapping and data flow are processed in PIM. This paper aims to provide insight into these aspects by analyzing the characteristics of various DNNs and providing detailed explanations of how they have been implemented with PIM architectures using commercially available memory technologies like DRAM and next-generation memory technologies like ReRAM.

关键词： processing-in-memory deep learning next-generation memory near-memory computing deep neural network

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：