检索结果-内蒙古大学图书馆

Allspark: Workload Orchestration for Visual Transformers on processing in-memory Systems

IEEE TRANSACTIONS ON COMPUTERS 2025年第2期74卷 427-441页

作者： Ge, Mengke Wang, Junpeng Chen, Binhan Zhong, Yingjian Du, Haitao Chen, Song Kang, Yi Hefei Comprehens Natl Sci Ctr Inst Artificial Intelligence Hefei 230088 Peoples R China Univ Sci & Technol China USTC Sch Microelect Hefei 230026 Peoples R China Anhui Univ Hefei 230039 Peoples R China

The advent of Transformers has revolutionized computer vision, offering a powerful alternative to convolutional neural networks (CNNs), especially with the local attention mechanism that excels at capturing local structures within the input and achieve state-of-the-art performance. processing in-memory (PIM) architecture offers extensive parallelism, low data movement costs, and scalable memory bandwidth, making it a promising solution to accelerate Transformer with memory-intensive operations. However, the crucial issue lies in efficiently deploying an entire model onto resource-limited PIM system while parallelizing each transformer block with potentially many computational branches based on local-attention mechanisms. We present Allspark, which focuses on workload orchestration for visual Transformers on PIM systems, aiming at minimizing inference latency. Firstly, to fully utilize the massive parallelism of PIM, Allspark employs a fine-grained partitioning scheme for computational branches, and formats a systematic layout and interleaved dataflow with maximized data locality and reduced data movement. Secondly, Allspark formulates the scheduling of the complete model on a resource-limited distributed PIM system as an integer linear programming (ILP) problem. Thirdly, as local-global data interactions exhibit complex yet regular dependencies, Allspark provides a two-stage placement method, which simplifies the challenging placement of computational branches on the PIM system into the structured layout and greedy-based binding, to minimize NoC communication costs. Extensive experiments on 3D-stacked DRAM-based PIM systems show that Allspark brings $1.2\times$1.2x $\sim$similar to $24.0\times$24.0x inference speedup for various visual Transformers over baselines. Compared to Nvidia V100 GPU, Allspark-enriched PIM system yields average speedups of $2.3\times$2.3x and energy savings of $20\times$20x $\sim$similar to $55\times$55x.

关键词： Transformers Visualization Parallel processing Computer architecture Computational modeling Bandwidth Costs memory management Layout System-on-chip processing in-memory scheduling visual transformer spatial architecture model parallelism

来源：评论

学校读者我要写书评

暂无评论

processing in-memory Architecture with On-Chip Transfer Learning Function for Compensating Characteristic Variation

Processing In-Memory Architecture with On-Chip Transfer Lear...

引用

12th IEEE International memory Workshop (IMW)

作者： Hayashikoshi, Masanori Suzuki, Junichi Watanabe, Yasuaki Furuya, Akiko Yabuta, Toshio Ido, Masamichi Kiyota, Yukiyoshi Kono, Takashi Renesas Elect Corp Shared R&D Core Technol Div Tokyo Japan

ISBN: (纸本)9781728163062

Along with more awareness of the need to endpoint data processing, the endpoint artificial intelligence (AI) technology with transfer learning function that enables not only inference but also additional learning at endpoint for adapting to the difference of surrounding environment will be of much importance in the future. In this work, Flash memory-based processing in-memory architecture (Flash-PIM) with transfer learning function and its learning method are proposed. By sharing the flash memory array for storing connection weight of synapse during inference and learning process, total area is reduced by 25.1 % and abnormality detection accuracy is 1.9 - 2.3 % better than conventional transfer learning due to eliminating the influence of variations in Flash memory cells. This approach is also effective for other non-volatile memory devices such as emerging memories.

关键词： nonvolatile memory processing in-memory artificial intelligence

来源：评论

学校读者我要写书评

暂无评论

NNPIM: A processing in-memory Architecture for Neural Network Acceleration

引用

IEEE TRANSACTIONS ON COMPUTERS 2019年第9期68卷 1325-1337页

作者： Gupta, Saransh Imani, Mohsen Kaur, Harveen Rosing, Tajana Simunic Univ Calif San Diego Dept Comp Sci & Engn La Jolla CA 92093 USA

Neural networks (NNs) have shown great ability to process emerging applications such as speech recognition, language recognition, image classification, video segmentation, and gaming. It is therefore important to make NNs efficient. Although attempts have been made to improve NNs' computation cost, the data movement between memory and processing cores is the main bottleneck for NNs' energy consumption and execution time. This makes the implementation of NNs significantly slower on traditional CPU/GPU cores. In this paper, we propose a novel processing in-memory architecture, called NNPIM, that significantly accelerates neural network's inference phase inside the memory. First, we design a crossbar memory architecture that supports fast addition, multiplication, and search operations inside the memory. Second, we introduce simple optimization techniques which significantly improves NNs' performance and reduces the overall energy consumption. We also map all NN functionalities using parallel in-memory components. To further improve the efficiency, our design supports weight sharing to reduce the number of computations in memory and consecutively speedup NNPIM computation. We compare the efficiency of our proposed NNPIM with GPU and the state-of-the-art PIM architectures. Our evaluation shows that our design can achieve 131.5x higher energy efficiency and is 48.2x faster as compared to NVIDIA GTX 1,080 GPU architecture. Compared to state-of-the-art neural network accelerators, NNPIM can achieve on an average 3.6x higher energy efficiency and is 4.6x faster, while providing the same classification accuracy.

关键词： Non-volatile memory processing in-memory neural networks

来源：评论

学校读者我要写书评

暂无评论

Digital-based processing in-memory: A Highly-Parallel Accelerator for Data Intensive Applications 19

Digital-based Processing In-Memory: A Highly-Parallel Accele...

引用

International Symposium on memory Systems (MEMSYS)

作者： Imani, Mohsen Gupta, Saransh Rosing, Tajana Univ Calif San Diego Dept Comp Sci & Engn La Jolla CA 92093 USA

ISBN: (纸本)9781450372060

Recently, processing in-memory (PIM) has been shown as a promising solution to address data movement issue in the current processors. However, today's PIM technologies are mostly analog-based, which involve both scalability and efficiency issues. In this paper, we propose a novel digital-based PIM which accelerates fundamental operations and diverse data analytic procedures using processing in-memory technology. Instead of sending a large amount of data to the processing cores for computation, our design performs a large part of computation tasks inside the memory;thus the application performance can be accelerated significantly by avoiding the memory access bottleneck. Digital-based PIM supports bit-wise operations between two selected bit-line of the memory block and then extends it to support row-parallel arithmetic operations.

关键词： processing in-memory Non-volatile memory Machine learning acceleration

来源：评论

学校读者我要写书评

暂无评论

DUAL: Acceleration of Clustering Algorithms using Digital-based processing in-memory 53

DUAL: Acceleration of Clustering Algorithms using Digital-ba...

引用

53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)

作者： Imani, Mohsen Pampana, Saikishan Gupta, Saransh Zhou, Minxuan Kim, Yeseong Rosing, Tajana UC Irvine Dept Comp Sci Irvine CA 92697 USA DGIST Dept Informat & Commun Engn Daegu South Korea Univ Calif San Diego Dept Comp Sci & Engn San Diego CA USA

ISBN: (数字)9781728173832

ISBN: (纸本)9781728173832

Today's applications generate a large amount of data that need to be processed by learning algorithms. In practice, the majority of the data are not associated with any labels. Unsupervised learning, i.e., clustering methods, are the most commonly used algorithms for data analysis. However, running clustering algorithms on traditional cores results in high energy consumption and slow processing speed due to a large amount of data movement between memory and processing units. In this paper, we propose DUAL, a Digital-based Unsupervised learning AcceLeration, which supports a wide range of popular algorithms on conventional crossbar memory. Instead of working with the original data, DUAL maps all data points into high-dimensional space, replacing complex clustering operations with memory-friendly operations. We accordingly design a PIM-based architecture that supports all essential operations in a highly parallel and scalable way. DUAL supports a wide range of essential operations and enables in-place computations, allowing data points to remain in memory. We have evaluated DUAL on several popular clustering algorithms for a wide range of large-scale datasets. Our evaluation shows that DUAL provides a comparable quality to existing clustering algorithms while using a binary representation and a simplified distance metric. DUAL also provides 58.8x speedup and 251.2x energy efficiency improvement as compared to the state-of-the-art solution running on GPU.

关键词： processing in-memory Unsupervised learning Hyperdimensional computing Algorithm-hardware co-design

来源：评论

学校读者我要写书评

暂无评论

Ultra-Efficient processing in-memory for Data Intensive Applications 17

Ultra-Efficient Processing In-Memory for Data Intensive Appl...

引用

54th ACM/EDAC/IEEE Design Automation Conference (DAC)

作者： Imani, Mohsen Gupta, Saransh Rosing, Tajana Univ Calif San Diego CSE Dept La Jolla CA 92093 USA

ISBN: (纸本)9781450349277

Recent years have witnessed a rapid growth in the domain of Internet of Things (IoT). This network of billions of devices generates and exchanges huge amount of data. The limited cache capacity and memory bandwidth make transferring and processing such data on traditional CPUs and GPUs highly inefficient, both in terms of energy consumption and delay. However, many IoT applications are statistical at heart and can accept a part of inaccuracy in their computation. This enables the designers to reduce complexity of processing by approximating the results for a desired accuracy. In this paper, we propose an ultra-efficient approximate processing in-memory architecture, called APIM, which exploits the analog characteristics of non-volatile memories to support addition and multiplication inside the crossbar memory, while storing the data. The proposed design eliminates the overhead involved in transferring data to processor by virtually bringing the processor inside memory. APIM dynamically configures the precision of computation for each application in order to tune the level of accuracy during runtime. Our experimental evaluation running six general OpenCL applications shows that the proposed design achieves up to 20 x performance improvement and provides 480 x improvement in energy-delay product, ensuring acceptable quality of service. In exact mode, it achieves 28 x energy savings and 4.8 x speed up compared to the state-of-the-art GPU cores.

关键词： processing in-memory Non-volatile memory Emerging computing

来源：评论

学校读者我要写书评

暂无评论

Power-Delay Area-Efficient processing-In-memory Based on Nanocrystalline Hafnia Ferroelectric Field-Effect Transistors

引用

ACS APPLIED MATERIALS & INTERFACES 2023年第1期15卷 1463-1474页

作者： Kim, Giuk Ko, Dong Han Kim, Taeho Lee, Sangho Jung, Minhyun Lee, Young Kyu Lim, Sehee Jo, Minyoung Eom, Taehyong Shin, Hunbeom Jeong, Yeongseok Jung, Seongook Jeon, Sanghun Korea Adv Inst Sci & Technol Sch Elect Engn Daejeon 34141 South Korea Yonsei Univ Sch Elect Engn Seoul 03722 South Korea

Ferroelectric field-effect transistors (FeFETs) have attracted enormous attention for low-power and high-density nonvolatile memory devices in processing-inmemory (PIM). However, their small memory window (MW) and limited endurance severely degrade the area efficiency and reliability of PIM devices. Herein, we overcome such challenges using key approaches covering from the material to the device and array architecture. High ferroelectricity was successfully demonstrated considering the thermodynamics and kinetics, even in a relatively thick (>= 30 nm) ferroelectric material that was unexplored so far. Moreover, we employed a metal-ferroelectric-metal-insulator-semiconductor architecture that enabled desirable voltage division between the ferroelectric and the metal-oxide-semiconductor FET, leading to a large MW (similar to 11 V), fast operation speed (<20 ns), and high endurance (similar to 10(11) cycles) characteristics. Subsequently, reliable and energy-efficient multiply-and-accumulation (MAC) operations were verified using a fabricated FeFET-PIM array. Furthermore, a system-level simulation demonstrated the high energy efficiency of the FeFET-PIM array, which was attributed to charge-domain computing. Finally, the proposed signed weight MAC computation achieved high accuracy on the CIFAR-10 dataset using the VGG-8 network.

关键词： processing in-memory energy efficiency area efficiency memory window endurance FeFETs

来源：评论

学校读者我要写书评

暂无评论

PhD Forum: Efficient Privacy-Preserving processing via memory-Centric Computing 43

PhD Forum: Efficient Privacy-Preserving Processing via Memor...

引用

43rd International Symposium on Reliable Distributed Systems

作者： Mwaisela, Mpoki Univ Neuchatel Neuchatel Switzerland

ISBN: (纸本)9798331530044;9798331530037

Privacy-preserving computation techniques like homomorphic encryption (HE) and secure multi-party computation (SMPC) enhance data security by enabling processing on encrypted data. However, the significant computational and CPU-DRAM data movement overhead resulting from the underlying cryptographic algorithms impedes the adoption of these techniques in practice. Existing approaches focus on improving computational overhead using specialized hardware like GPUs and FPGAs, but these methods still suffer from the same processor-DRAM overhead. Novel hardware technologies that support in-memory processing have the potential to address this problem. memory-centric computing, or processing-in-memory (PIM), brings computation closer to data by introducing low-power processors called data processing units (DPUs) into memory. Besides its in-memory computation capability, PIM provides extensive parallelism, resulting in significant performance improvement over state-of-the-art approaches. We propose a framework that uses recently available PIM hardware to achieve efficient privacy-preserving computation. Our design consists of a four-layer architecture: (1) an application layer that decouples privacy-preserving applications from the underlying protocols and hardware;(2) a protocol layer that implements existing secure computation protocols (HE and MPC);(3) a data orchestration layer that leverages data compression techniques to mitigate the data transfer overhead between DPUs and host memory;(4) a computation layer which implements DPU kernels on which secure computation algorithms are built.

关键词： privacy-preserving processing secure multi-party computation homomorphic encryption processing in-memory memory-centric computing

来源：评论

学校读者我要写书评

暂无评论

Evaluating the Potential of In-memory processing to Accelerate Homomorphic Encryption 43

Evaluating the Potential of In-Memory Processing to Accelera...

引用

43rd International Symposium on Reliable Distributed Systems

作者： Mwaisela, Mpoki Hari, Joel Yuhala, Peterson Menetrey, James Felber, Pascal Schiavoni, Valerio Univ Neuchatel Neuchatel Switzerland Univ Bern Bern Switzerland

ISBN: (纸本)9798331530044;9798331530037

The widespread adoption of cloud-based solutions introduces privacy and security concerns. Techniques such as homomorphic encryption (HE) mitigate this problem by allowing computation over encrypted data without the need for decryption. However, the high computational and memory overhead associated with the underlying cryptographic operations has hindered the practicality of HE-based solutions. While a significant amount of research has focused on reducing computational overhead by utilizing hardware accelerators like GPUs and FPGAs, there has been relatively little emphasis on addressing HE memory overhead. processing in-memory (PIM) presents a promising solution to this problem by bringing computation closer to data, thereby reducing the overhead resulting from processor-memory data movements. In this work, we evaluate the potential of a PIM architecture from UPMEM for accelerating HE operations. Firstly, we focus on PIM-based acceleration for polynomial operations, which underpin HE algorithms. Subsequently, we conduct a case study analysis by integrating PIM into two popular and open-source HE libraries, OpenFHE and HElib. Our study concludes with key findings and takeaways gained from the practical application of HE operations using PIM, providing valuable insights for those interested in adopting this technology.

关键词： homomorphic encryption processing in-memory privacy-preserving computation OpenFHE HElib

来源：评论

学校读者我要写书评

暂无评论

ORCHARD: Visual Object Recognition Accelerator Based on Approximate In-memory processing 17

ORCHARD: Visual Object Recognition Accelerator Based on Appr...

引用

IEEE/ACM 36th International Conference on Computer-Aided Design (ICCAD)

作者： Kim, Yeseong Imani, Mohsen Rosing, Tajana Univ Calif San Diego La Jolla CA 92093 USA

ISBN: (纸本)9781538630938

In recent years, machine learning for visual object recognition has been applied to various domains, e.g., autonomous vehicle, heath diagnose, and home automation. However, the recognition procedures still consume a lot of processing energy and incur a high cost of data movement for memory accesses. In this paper, we propose a novel hardware accelerator design, called ORCHARD, which processes the object recognition tasks inside memory. The proposed design accelerates both the image feature extraction and boosting-based learning algorithm, which are key subtasks of the state-of-the-art image recognition approaches. We optimize the recognition procedures by leveraging approximate computing and emerging non-volatile memory (NVM) technology. The NVM-based in-memory processing allows the proposed design to mitigate the CMOS-based computation overhead, highly improving the system efficiency. In our evaluation conducted on circuit-and device-level simulations, we show that ORCHARD successfully performs practical image recognition tasks, including text, face, pedestrian, and vehicle recognition with 0.3% of accuracy loss made by computation approximation. In addition, our design significantly improves the performance and energy efficiency by up to 376x and 1896x, respectively, compared to the existing processor-based implementation.

关键词： Adaboost object recognition processing in-memory non-volatile memory

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：