检索结果-内蒙古大学图书馆

CPSAA: Accelerating Sparse Attention Using Crossbar-Based processing-in-memory Architecture

IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS 2024年第6期43卷 1741-1754页

作者： Li, Huize Jin, Hai Zheng, Long Liao, Xiaofei Huang, Yu Liu, Cong Xu, Jiahong Duan, Zhuohui Chen, Dan Gui, Chuangyi Huazhong Univ Sci & Technol Natl Engn Res Ctr Big Data Technol & Syst Sch Comp Sci & Technol Serv Comp Technol & Syst LabCluster & Grid Comp L Wuhan 430074 Peoples R China

The attention-based neural network attracts great interest due to its excellent accuracy enhancement. However, the attention mechanism requires huge computational efforts to process unnecessary calculations, significantly limiting the system's performance. To reduce the unnecessary calculations, researchers propose sparse attention to convert some dense-dense matrices multiplication (DDMM) operations to sampled dense-dense matrix multiplication (SDDMM) and sparse matrix multiplication (SpMM) operations. However, current sparse attention solutions introduce massive off-chip random memory access since the sparse attention matrix is generally unstructured. We propose CPSAA, a novel crossbar-based processing-in-memory (PIM)-featured sparse attention accelerator to eliminate off-chip data transmissions. 1) We present a novel attention calculation mode to balance the crossbar writing and crossbar processing latency. 2) We design a novel PIM-based sparsity pruning architecture to eliminate the pruning phase's off-chip data transfers. 3) Finally, we present novel crossbar-based SDDMM and SpMM methods to process unstructured sparse attention matrices by coupling two types of crossbar arrays. Experimental results show that CPSAA has an average of 89.6x , 32.2x , 17.8x , 3.39x , and 3.84x performance improvement and 755.6x , 55.3x , 21.3x , 5.7x , and 4.9x energy-saving when compare with GPU, field programmable gate array, SANGER, ReBERT, and ReTransformer.

关键词： Sparse matrices Computer architecture Matrix converters Microprocessors Field programmable gate arrays Virtual machine monitors Hardware Attention mechanism domain-specific accelerator processing-in-memory resistive random access memory (ReRAM)

来源：评论

学校读者我要写书评

暂无评论

RADAR: A Skew-Resistant and Hotness-Aware Ordered Index Design for processing-in-memory Systems

引用

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2024年第9期35卷 1598-1614页

作者： Hua, Yifan Zheng, Shengan Kong, Weihan Zhou, Cong Huang, Kaixin Ma, Ruoyan Huang, Linpeng Shanghai Jiao Tong Univ Sch Elect Informat & Elect Engn Shanghai 200240 Peoples R China Shanghai Jiao Tong Univ Sch Elect Informat & Elect Engn AI Inst MoEKey Lab Artificial Intelligence Shanghai 200240 Peoples R China

Pointer chasing becomes the performance bottleneck for today's in-memory indexes due to the memory wall. Emerging processing-in-memory (PIM) technologies are promising to mitigate this bottleneck, by enabling low-latency memory access and aggregated memory bandwidth scaling with the number of PIM modules. Prior PIM-based indexes adopt a fixed granularity to partition the key space and maintain static heights of skiplist nodes among PIM modules to accelerate index operations on skiplist, neglecting the changes in skewness and hotness of data access patterns during runtime. In this article, we present RADAR, an innovative PIM-friendly skiplist that dynamically partitions the key space among PIM modules to adapt to varying skewness. An offline learning-based model is employed to catch hotness changes to adjust the heights of skiplist nodes. In multiple datasets, RADAR achieves up to 198.2x performance improvement and consumes 47.4% less memory than state-of-the-art designs on real PIM hardware.

关键词： Indexes Parallel processing Bandwidth Random access memory Aggregates Throughput Spaceborne radar processing-in-memory ordered index pointer chasing load balance index partition

来源：评论

学校读者我要写书评

暂无评论

NPC: A Non-Conflicting processing-in-memory Controller in DDR memory Systems

引用

IEEE TRANSACTIONS ON COMPUTERS 2025年第3期74卷 1025-1039页

作者： Lee, Seungyong Lee, Sanghyun Seo, Minseok Park, Chunmyung Shin, Woojae Lee, Hyuk-Jae Kim, Hyun Seoul Natl Univ Interuniv Semicond Res Ctr Dept Elect & Comp Engn Seoul 08826 South Korea SK Hynix Inc Icheon 17336 Gyeonggi Do South Korea Seoul Natl Univ Sci & Technol Dept Elect & Informat Engn Seoul 01811 South Korea Seoul Natl Univ Sci & Technol Res Ctr Elect & Informat Technol Seoul 01811 South Korea

processing-in-memory (PIM) has emerged as a promising solution to address the memory wall problem. Existing memory interfaces must support new PIM commands to utilize PIM, making the definition of PIM commands according to memory modes a major issue in the development of practical PIM products. For performance and OS-transparency, the memory controller is responsible for changing the memory mode, which requires modifying the controller and resolving conflicts with existing functionalities. Additionally, it must operate to minimize mode transition overhead, which can cause significant performance degradation. In this study, we present NPC, a memory controller designed for mode transition PIM that delivers PIM commands via the DDR interface. NPC issues PIM commands while transparently changing the memory mode with a dedicated scheduling policy that reduces the number of mode transitions with aggregative issuing. Moreover, existing functions, such as refresh, are optimized for PIM operation. We implement NPC in hardware and develop a PIM emulation system to validate it on FPGA platforms. Experimental results reveal that NPC is compatible with existing interfaces and functionality, and the proposed scheduling policy improves performance by 2.2x with balanced fairness, achieving up to 97% of the ideal performance. These findings have the potential to aid the application of PIM in real systems and contribute to the commercialization of mode transition PIM.

关键词： memory management Processor scheduling Random access memory Error correction codes Computers Calibration Microprocessors Micromechanical devices Hardware Degradation processing-in-memory memory controller DRAM scheduling policy

来源：评论

学校读者我要写书评

暂无评论

processing-in-memory Using Optically-Addressed Phase Change memory

Processing-in-Memory Using Optically-Addressed Phase Change ...

引用

IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED)

作者： Yang, Guowei Demirkiran, Cansu Kizilates, Zeynep Ece Ocampo, Carlos A. Rios Coskun, Ayse K. Joshi, Ajay Boston Univ Boston MA 02215 USA Univ Maryland College Pk MD 20742 USA

ISBN: (纸本)9798350311754

Today's Deep Neural Network (DNN) inference systems contain hundreds of billions of parameters, resulting in significant latency and energy overheads during inference due to frequent data transfers between compute andmemory units. processing-in-memory (PiM) has emerged as a viable solution to tackle this problem by avoiding the expensive data movement. PiM approaches based on electrical devices suffer from throughput and energy efficiency issues. In contrast, Optically-addressed Phase Change memory (OPCM) operates with light and achieves much higher throughput and energy efficiency compared to its electrical counterparts. This paper introduces a system-level design that takes the OPCM programming overhead into consideration, and identifies that the programming cost dominates the DNN inference on OPCM-based PiM architectures. We explore the design space of this system and identify themost energy-efficientOPCMarray size and batch size. We propose a novel thresholding and reordering technique on the weight blocks to further reduce the programming overhead. Combining these optimizations, our approach achieves up to 65.2 x higher throughput than existing photonic accelerators for practical DNN workloads.

关键词： optical computing phase change memory processing-in-memory deep neural networks

来源：评论

学校读者我要写书评

暂无评论

PyGim : An Efficient Graph Neural Network Library for Real processing-in-memory Architectures

引用

PROCEEDINGS OF THE ACM ON MEASUREMENT AND ANALYSIS OF COMPUTING SYSTEMS 2024年第3期8卷 1-36页

作者： Giannoula, Christina Yang, Peiming Fernandez, Ivan Yang, Jiacheng Durvasula, Sankeerth Li, Yu xin Sadrosadati, Mohammad Luna, Juan gomez Mutlu, Onur Pekhimenko, Gennady Univ Toronto Toronto ON Canada Swiss Fed Inst Technol Zurich Switzerland Vector Inst Toronto ON Canada CentML Toronto ON Canada Barcelona Supercomp Ctr Barcelona Spain Univ Politecn Cataluna Barcelona Spain NVIDIA Zurich Switzerland

Graph Neural Networks (GNNs) are emerging models to analyze graph-structure data. GNN execution involves both compute-intensive and memory-intensive kernels. The latter kernels dominate execution time, because they are significantly bottlenecked by data movement between memory and processors. processing-in-memory (PIM) systems can alleviate this data movement bottleneck by placing simple processors near or inside to memory arrays. This work investigates the potential of PIM systems to alleviate the data movement bottleneck in GNNs, and introduces PyGim, an efficient and easy-to-use GNN library for real PIM systems. We propose intelligent parallelization techniques for memory-intensive kernels of GNNs tailored for real PIM systems, and develop an easy-to-use Python API for them. PyGim employs a cooperative GNN execution, in which the compute- and memory-intensive kernels are executed in processor-centric and memory-centric computing systems, respectively, to fully exploit the hardware capabilities. PyGim integrates a lightweight autotuner to tune the parallelization strategy of the memory-intensive kernel of GNNs and enable high programming ease. We extensively evaluate PyGim on a real-world PIM system that has 16 PIM DIMMs with 1992 PIM cores connected to a Host CPU. In GNN inference, we demonstrate that it outperforms prior state-of-the-art PIM works by on average 4.38x (up to 7.20x), and state-of-the-art PyTorch running on Host by on average 3.04x (up to 3.44x). PyGim improves energy efficiency by 2.86x (up to 3.68x) and 1.55x (up to 1.75x) over prior PIM and PyTorch Host schemes, respectively. In memory-intensive kernel of GNNs, PyGim provides 11.6x higher resource utilization in PIM system than that of PyTorch library (optimized CUDA implementation) in GPU systems. Our work provides useful recommendations for software, system and hardware designers. PyGim is publicly and freely available at https://***/CMU-SAFARI/PyGim facilitate the widespread use of PIM

关键词： machine learning graph neural networks sparse matrix-matrix multi- plication library multicore processing-in-memory near-data processing memory systems data movement bottleneck DRAM benchmarking real-system characterization workload characterization

来源：评论

学校读者我要写书评

暂无评论

FeMPIM: A FeFET-Based Multifunctional processing-in-memory Cell

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS 2024年第4期71卷 2299-2303页

作者： Yan, Aibin Chen, Yu Gao, Zhongyu Ni, Tianming Huang, Zhengfeng Cui, Jie Girard, Patrick Wen, Xiaoqing Hefei Univ Technol Sch Microelect Hefei 230009 Peoples R China Anhui Univ Sch Comp Sci & Technol Hefei 230601 Peoples R China Anhui Polytech Univ Sch Integrated Circuits Wuhu 241000 Peoples R China Univ Montpellier Lab Informat Robot & Microelect Montpellier CNRS F-34095 Montpellier France Kyushu Inst Technol Grad Sch Comp Sci & Syst Engn Fukuoka 8208502 Japan

The Von-Neumann memory wall bottleneck that keeps expanding is mainly caused by the frequent data transfer between the main memory and the processor. The processing in-memory (PiM) capabilities of emerging nonvolatile devices have the potential to partially alleviate the memory wall problem. In this brief, we use the ferroelectric field-effect transistor (FeFET), one of the emerging nonvolatile devices, to design a multifunctional processing in-memory cell, namely FeMPIM. It can perform multiple logic operations in computing mode as well as content searching in ternary content-addressable memory (TCAM) mode. Simulation results demonstrate the multifunctional capability of the proposed FeMPIM as well as its moderate overhead when compared with the complementary metal-oxide-semiconductor (CMOS) based and the existing FeFET-based devices.

关键词： FeFETs Computer architecture Microprocessors Arrays Transistors Voltage Nonvolatile memory Ferroelectric field-effect transistor ternary content-addressable memory processing-in-memory

来源：评论

学校读者我要写书评

暂无评论

Pyramid: Accelerating LLM Inference With Cross-Level processing-in-memory

IEEE COMPUTER ARCHITECTURE LETTERS

引用

IEEE COMPUTER ARCHITECTURE LETTERS 2025年第1期24卷 121-124页

作者： Yan, Liang Lu, Xiaoyang Chen, Xiaoming Han, Yinhe Sun, Xian-He Chinese Acad Sci Inst Comp Technol Intelligent Comp Syst Beijing 100190 Peoples R China Univ Chinese Acad Sci Beijing 100190 Peoples R China IIT Dept Compute Sci Chicago IL 60616 USA

Integrating processing-in-memory (PIM) with GPUs accelerates large language model (LLM) inference, but existing GPU-PIM systems encounter several challenges. While GPUs excel in large general matrix-matrix multiplications (GEMM), they struggle with small-scale operations better suited for PIM, which currently cannot handle them independently. Additionally, the computational demands of activation operations exceed the capabilities of current PIM technologies, leading to excessive data movement between the GPU and memory. PIM's potential for general matrix-vector multiplications (GEMV) is also limited by insufficient support for fine-grained parallelism. To address these issues, we propose Pyramid, a novel GPU-PIM system that optimizes PIM for LLM inference by strategically allocating cross-level computational resources within PIM to meet diverse needs and leveraging the strengths of both technologies. Evaluation results demonstrate that Pyramid outperforms existing systems like NeuPIM, AiM, and AttAcc by factors of 2.31x, 1.91x, and 1.72x, respectively.

关键词： Graphics processing units Decoding Computational modeling Parallel processing Systolic arrays Computer architecture Table lookup Random access memory Interpolation Transformers Large language models processing-in-memory

来源：评论

学校读者我要写书评

暂无评论

Scalability Limitations of processing-in-memory using Real System Evaluations

引用

PROCEEDINGS OF THE ACM ON MEASUREMENT AND ANALYSIS OF COMPUTING SYSTEMS 2024年第1期8卷 1-28页

作者： Jonatan, Gilbert Cho, Haeyoon Son, Hyojun Wu, Xiangyu Livesay, Neal Mora, Evelio Shivdikar, Kaustubh Abellan, Jose L. Joshi, Ajay Kaeli, David Kim, John Korea Adv Inst Sci & Technol Daejeon South Korea Northeastern Univ Boston MA 02115 USA Univ Catolica Murcia Murcia Spain Univ Murcia Murcia Spain Boston Univ Boston MA 02215 USA

processing-in-memory (PIM), where the compute is moved closer to the memory or the data, has been widely explored to accelerate emerging workloads. Recently, different PIM-based systems have been announced by memory vendors to minimize data movement and improve performance as well as energy efficiency. One critical component of PIM is the large amount of compute parallelism provided across many PIM "nodes" or the compute units near the memory. In this work, we provide an extensive evaluation and analysis of real PIM systems based on UPMEM PIM. We show that while there are benefits of PIM, there are also scalability challenges and limitations as the number of PIM nodes increases. In particular, we show how collective communications that are commonly found in many kernels/workloads can be problematic for PIM systems. To evaluate the impact of collective communication in PIM architectures, we provide an in-depth analysis of two workloads on the UPMEM PIM system that utilize representative common collective communication patterns - AllReduce and All-to-All communication. Specifically, we evaluate 1) embedding tables that are commonly used in recommendation systems that require AllReduce and 2) the Number Theoretic Transform (NTT) kernel which is a critical component of Fully Homomorphic Encryption (FHE) that requires All-to-All communication. We analyze the performance benefits of these workloads and show how they can be efficiently mapped to the PIM architecture through alternative data partitioning. However, since each PIM compute unit can only access its local memory, when communication is necessary between PIM nodes (or remote data is needed), communication between the compute units must be done through the host CPU, thereby severely hampering application performance. To increase the scalability (or applicability) of PIM to future workloads, we make the case for how future PIM architectures need efficient communication or interconnection networks between the PIM nod

关键词： processing-in-memory interconnection networks collective communication

来源：评论

学校读者我要写书评

暂无评论

DDAM: Data Distribution-Aware Mapping of CNNs on processing-in-memory Systems

引用

ACM TRANSACTIONS ON DESIGN AUTOMATION OF ELECTRONIC SYSTEMS 2023年第3期28卷 36-36页

作者： Wang, Junpeng Du, Haitao Ding, Bo Xu, Qi Chen, Song Kang, Yi Univ Sci & Technol China Sch Microelect 96 JinZhai Rd Hefei 230026 Anhui Peoples R China Hefei Comprehens Natl Sci Ctr Inst Artificial Intelligence 5089 West Wangjiang Rd Hefei 230088 Anhui Peoples R China

Convolution neural networks (CNNs) are widely used algorithms in image processing, natural language processing and many other fields. The large amount of memory access of CNNs is one of the major concerns in CNN accelerator designs that influences the performance and energy-efficiency. With fast and low-cost memory access, processing-in-memory (PIM) system is a feasible solution to alleviate the memory concern of CNNs. However, the distributed manner of data storing in PIM systems is in conflict with the large amount of data reuse of CNN layers. Nodes of PIM systems may need to share their data with each other before processing a CNN layer, leading to extra communication overhead. In this article, we propose DDAM to map CNNs onto PIM systems with the communication overhead reduced. Firstly, A data transfer strategy is proposed to deal with the data sharing requirement among PIM nodes by formulating a Traveling-Salesman-Problem (TSP). To improve data locality, a dynamic programming algorithm is proposed to partition the CNN and allocate a number of nodes to each part. Finally, an integer linear programming (ILP)-based mapping algorithm is proposed to map the partitioned CNN onto the PIM system. Experimental results show that compared to the baselines, DDAM can get a higher throughput of 2.0x with the energy cost reduced by 37% on average.

关键词： Convolutional neural networks processing-in-memory

来源：评论

学校读者我要写书评

暂无评论

Plug N? PIM: An integration strategy for processing-in-memory accelerators

引用

INTEGRATION-THE VLSI JOURNAL 2023年 88卷 185-195页

作者： Santos, Paulo C. Forlin, Bruno E. Alves, Marco A. Z. Carro, Luigi Fed Univ Rio Grande Porto Alegre Rio Grande do S Brazil Univ Parana Curitiba Brazil

processing-in-memory (PIM) devices have reemerged as a promise to mitigate the memory-wall and the limitations of transferring massive amount of data from main memories to the host processors. Novel memory technologies and the advent of 3D-stacked integration have provided means to compute data in-memory, either by exploring the inherent analog capabilities or by tight-coupling logic and memory. However, allowing the effective use of a PIM device demands significant and costly modifications on the host processor to support instructions offloading, cache coherence, virtual memory management, and communication between different PIM instances. This paper tackles these challenges by presenting a set of solutions to couple host and PIM with no modifications at host side. Moreover, we highlight the limitations presented on modern host processors that may prevent full performance extraction of the PIM devices. This work presents Plug N' PIM, a set of strategies and procedures to seamlessly couple host general-purpose processors and PIM devices. We show that with our techniques one can exploit the benefits of a PIM device with seamless integration between host and PIM, bypassing possible limitations on the host side.

关键词： processing-in-memory Code offloading Cache coherence System integration

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：