检索结果-内蒙古大学图书馆

Minimizing transformer inference overhead using controlling element on Shenwei AI accelerator

Frontiers of Information Technology and Electronic Engineering 2025年第4期26卷 605-622页

作者： Zhao, Yulong Wu, Chunzhi Wang, Yizhuo Zhang, Lufei Zhang, Yaguang Shen, Wenyuan Fan, Hao Fang, Hankang Qin, Yi Liu, Xin State Key Laboratory of Mathematical Engineering and Advanced Computing Wuxi214000 China School of Non-Commissioned Officer Space Engineering University Beijing100004 China National Supercomputing Center in Wuxi Wuxi214000 China Zhejiang Lab Hangzhou310000 China National Research Centre of Parallel Computer Engineering and Technology Beijing100081 China

Transformer models have become a cornerstone of various natural language processing (NLP) tasks. However, the substantial computational overhead during the inference remains a significant challenge, limiting their deployment in practical applications. In this study, we address this challenge by minimizing the inference overhead in transformer models using the controlling element on artificial intelligence (AI) accelerators. Our work is anchored by four key contributions. First, we conduct a comprehensive analysis of the overhead composition within the transformer inference process, identifying the primary bottlenecks. Second, we leverage the management processing element (MPE) of the Shenwei AI (SWAI) accelerator, implementing a three-tier scheduling framework that significantly reduces the number of host-device launches to approximately 1/10 000 of the original PyTorch-GPU setup. Third, we introduce a zero-copy memory management technique using segment-page fusion, which significantly reduces memory access latency and improves overall inference efficiency. Finally, we develop a fast model loading method that eliminates redundant computations during model verification and initialization, reducing the total loading time for large models from 22 128.31 ms to 1041.72 ms. Our contributions significantly enhance the optimization of transformer models, enabling more efficient and expedited inference processes on AI accelerators. © Zhejiang University Press 2025.

关键词： Transformer inference optimization Three-tier scheduling Zero-copy memory management Fast model loading

来源：评论

学校读者我要写书评

暂无评论

An online service-oriented performance profiling tool for cloud computing systems

引用

Frontiers of Computer Science 2013年第3期7卷 431-445页

作者： Haibo MI Huaimin WANG Yangfan ZHOU Michael Rung-Tsong LY~ Hua CAI Gang YIN National Lab for Parallel & Distributed Processing National University of Defense Technology Changsha 410073 China Shenzhen Research Institute The Chinese University of Hong Kong Shenzhen 518000 China Computing Platform Alibaba Cloud Computing Company Hangzhou 310000 China

The growing scale and complexity of component interactions in cloud computing systems post great challenges for operators to understand the characteristics of system performance. Profiling has long been proved to be an effective approach to performance analysis; however, existing approaches confront new challenges that emerge in cloud computing systems. First, the efficiency of the profiling becomes of critical concern; second, service-oriented profiling should be considered to support separation-of-concerns performance analysis. To address the above issues, in this paper, we present P-Tracer, an online performance profiling tool specifically tailored for cloud computing systems. P-Tracer constructs a specific search engine that proactively processes performance logs and generates a particular index for fast queries; second, for each service, P-Tracer retrieves a statistical insight of performance characteristics from multi-dimensions and provides operators with a suite of web-based interfaces to query the critical information. We evaluate P- Tracer in the aspects of tracing overheads, data preprocessing scalability and querying efficiency. Three real-world case studies that happened in Alibaba cloud computing platform demonstrate that P-Tracer can help operators understand soft-ware behaviors and localize the primary causes of performance anomalies effectively and efficiently.

关键词： cloud computing performance profiling performance anomaly visual analytics

来源：评论

学校读者我要写书评

暂无评论

Bridging the gap between HPC and big data frameworks 43rd

Bridging the gap between HPC and big data frameworks

引用

43rd International Conference on Very Large Data Bases, VLDB 2017

作者： Anderson, Michael Smith, Shaden Sundaram, Narayanan Capotă, Mihai Zhao, Zheguang Dulloor, Subramanya Satish, Nadathur Willke, Theodore L. Parallel Computing Lab United States University of Minnesota United States Brown University United States Infrastructure Research Lab Intel Corporation United States

Apache Spark is a popular framework for data analytics with attractive features such as fault tolerance and interoperability with the Hadoop ecosystem. Unfortunately, many analytics operations in Spark are an order of magnitude or more slower compared to native implementations written with high performance computing tools such as MPI. There is a need to bridge the performance gap while retaining the benefits of the Spark ecosystem such as availability, productivity, and fault tolerance. In this paper, we propose a system for integrating MPI with Spark and analyze the costs and benefits of doing so for four distributed graph and machine learning applications. We show that offloading computation to an MPI environment from within Spark provides 3.1-17.7 × speedups on the four sparse applications, including all of the overheads. This opens up an avenue to reuse existing MPI libraries in Spark with little effort. © 2017 VLDB Endowment.

关键词： Fault tolerance

来源：评论

学校读者我要写书评

暂无评论

CloudRAMSort: Fast and efficient large-scale distributed RAM sort on shared-nothing cluster 12

CloudRAMSort: Fast and efficient large-scale distributed RAM...

引用

2012 ACM SIGMOD International Conference on Management of Data, SIGMOD '12

作者： Kim, Changkyu Park, Jongsoo Satish, Nadathur Lee, Hongrae Dubey, Pradeep Chhugani, Jatin Parallel Computing Lab. Intel Labs. Santa Clara CA United States Structured Data Group Google Research Mountain View United States

ISBN: (纸本)9781450312479

Sorting is a fundamental kernel used in many database operations. The total memory available across cloud computers is now sufficient to store even hundreds of terabytes of data in-memory. Applications requiring high-speed data analysis typically use in-memory sorting. The two most important factors in designing a high-speed in-memory sorting system are the single-node sorting performance and inter-node communication. In this paper, we present CloudRAMSort, a fast and efficient system for large-scale distributed sorting on shared-nothing clusters. CloudRAMSort performs multi-node optimizations by carefully overlapping computation with inter-node communication. The system uses a dynamic multi-stage random sampling approach for improved load-balancing between nodes. CloudRAMSort maximizes per-node efficiency by exploiting modern architectural features such as multiple cores and SIMD (Single-Instruction Multiple Data) units. This holistic combination results in the highest performing sorting performance on distributed shared-nothing platforms. CloudRAMSort sorts 1 Terabyte (TB) of data in 4.6 seconds on a 256-node Xeon X5680 cluster called the Intel Endeavor system. CloudRAMSort also performs well on heavily skewed input distributions, sorting 1 TB of data generated using Zipf distribution in less than 5 seconds. We also provide a detailed analytical model that accurately projects (within avg. 7%) the performance of CloudRAMSort with varying tuple sizes and interconnect bandwidths. Our analytical model serves as a useful tool to analyze performance bottlenecks on current systems and project performance with future architectural advances. With architectural trends of increasing number of cores, bandwidth, SIMD width, cache-sizes, and interconnect bandwidth, we believe CloudRAMSort would be the system of choice for distributed sorting of large-scale in-memory data of current and future systems. © 2012 ACM.

关键词： Sorting

来源：评论

学校读者我要写书评

暂无评论

A Multi-Dimensional Assessment Model and Its Application in E-learning Courses of Computer Science 20

A Multi-Dimensional Assessment Model and Its Application in ...

引用

21st Annual Conference of the Special Interest Group in Information Technology Education, SIGITE 2020

作者： Luo, Jiwen Lu, Feng Wang, Tao Key Lab. of Parallel and Distributed Computing College of Computer National University of Defense Technology Changsha China National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China

ISBN: (纸本)9781450370455

Computer science is a practical discipline. It is always a great challenge to evaluate students' computer practice using computer-aided means for large scale students. We always need to address problems such as suspected plagiarism and deviation of the overall difficulty factor. In this paper, a multi-dimensional assessment model is designed for CS courses based on the detailed practice processing data in an E-learning system. The model comprehensively evaluates the students' learning process and results in three aspects of correctness, originality, and quality detection. Besides, the teacher can easily participate in the assessment according to their needs. The correctness is an essential requirement, and the originality is based on the clustering results of students' behaviors after clone detection to curb homework plagiarism. SonarQube is used to detect code quality and put forward higher requirements for codes. Manual participation intelligence has improved the flexibility and applicability of the model to a certain extent. We applied this model on the EduCoder online education platform and carried out a comprehensive analysis of 485 students in the parallel Programming Principles and Practice Class of Huazhong University of Science and Technology. Experiment results confirm the distinction, rationality, and fairness of the model in assessing student performance. It not only gives students a credible, comprehensive score in large-scale online practical programming courses but also gives teachers and students corresponding suggestions based on the evaluation results. Furthermore, the model can be extended to other online education platforms. © 2020 ACM.

关键词： Students

来源：评论

学校读者我要写书评

暂无评论

Tensaurus: A Versatile Accelerator for Mixed Sparse-Dense Tensor Computations

Tensaurus: A Versatile Accelerator for Mixed Sparse-Dense Te...

引用

IEEE Symposium on High-Performance Computer Architecture

作者： Nitish Srivastava Hanchen Jin Shaden Smith Hongbo Rong David Albonesi Zhiru Zhang Cornell University Microsoft AI & Research Intel Parallel Computing Lab

ISBN: (数字)9781728161495

ISBN: (纸本)9781728161501

Tensor factorizations are powerful tools in many machine learning and data analytics applications. Tensors are often sparse, which makes sparse tensor factorizations memory bound. In this work, we propose a hardware accelerator that can accelerate both dense and sparse tensor factorizations. We co-design the hardware and a sparse storage format, which allows accessing the sparse data in vectorized and streaming fashion and maximizes the utilization of the memory bandwidth. We extract a common computation pattern that is found in numerous matrix and tensor operations and implement it in the hardware. By designing the hardware based on this common compute pattern, we can not only accelerate tensor factorizations but also mixed sparse-dense matrix operations. We show significant speedup and energy benefit over the state-of-the-art CPU and GPU implementations of tensor factorizations and over CPU, GPU and accelerators for matrix operations.

关键词： Tensile stress Sparse matrices Hardware Kernel Acceleration Matrix decomposition Bandwidth

来源：评论

学校读者我要写书评

暂无评论

Multiprocessor scheduling using mean-field annealing 12th

引用

10 Workshops held in conjunction with 12th International parallel Symposium and 9th Symposium on parallel and Distributed Processing, IPPS/SPDP 1998

作者： Salleh, Shaharuddin Zomaya, Albert Y. Dept of Mathematics University of Technology Malaysia KB 791 Johor Bahru80990 Malaysia Parallel Computing Research Lab Dept of Electrical and Electronics Eng University of Western Australia PerthWA6907 Australia

ISBN: (纸本)3540643591

This paper presents our work on the static task scheduling model using the mean-field annealing (MFA) technique. Mean-field annealing is a technique of thermostatic annealing that takes the statistical properties of particles as its learning paradigm. It combines good features from the Hopfield neural network and simulated annealing, to overcome their weaknesses and improve on their performances. Our MFA model for task scheduling is derived from its prototype, namely, the graph partitioning problem. MFA is deterministic in nature and this gives the advantage o f faster convergence to the equilibrium temperature, compared to simulated annealing. Our experimental work verifies this finding on various network and task graph sizes. Our work also includes the simulation of the MFA model on several network topologies and parameters. © Springer-Verlag Berlin Heidelberg 1998.

关键词： Hopfield neural networks

来源：评论

学校读者我要写书评

暂无评论

A time- A nd cost-optimal algorithm for overlap graphs, with applications 25

A time- A nd cost-optimal algorithm for overlap graphs, with...

引用

25th International Conference on parallel Processing, ICPP 1996

作者： Olariu, S. Zomaya, A.Y. Department of Computer Science Old Dominion University Norfolk VA 23529-0162 United States Parallel Computing Research Lab Dept. of Electrical and Electronic Eng Univ. of Western Austialia PerthWA6007 Australia

ISBN: (纸本)081867623X

In varioui practical applications it is convenient to associate a cer I ain graph with a family Z of intervals. One such graph is the well-known overlap graphof Z, whose vertices are the iiitervals in I , with two intervals connected by and edge if imd only if they overlap but neither of them strictly contains the other. The first main contribution of this paper is to propose time- A nd cost-optimal algorithms for computiiig the connected components of an overlap graph. The iask of computing the connected components of an overlap graph arises in numerous applications including traffic control, robot arm manipulation, segmentation of range images, routing, automated surveillance systems, recognizing polygonal configurations, and code generation for parallel niachines. We begin by showing that any sequential algo i ithm that determines the connected components of the overlap graph of a family of n intervals must take Ω(n log n) time in the algebraic tree model. Next, we show that any parallel algorithm for this problem must take Ω(log n) time in the CREW model even if an infinite numbei of processors and meinoiy cells are available. We then go on to show that both the sequential and the parallel lowei bounds are tight by providing matching algorithms running in Θ(n log n) sequential time and Θ(log n) time using n processors in the CREW model, respectively. At the same time, if the endpoints of the intervals are given in sorted ordLi. our sequential algorithm runs in Θ(n)time, improving the best previously known result. It is interesting to note t l i i L t even if the endpoints are sorted, Ω(1og n) is a time lower hound for solving the problem in the CREW model, regaicllrss of the amount of resources available. As an application, we show that our algorithm affords us a time- A nd cost-optimal solution to a restricted version of the single row routing problem. The best previously known result for routing a set of nets without interstreet crossings run5 in O(1og n log l

关键词： Trees (mathematics)

来源：评论

学校读者我要写书评

暂无评论

Delayed Difference Scheme for Large Scale Scientific Simulations

引用

Physical Review Letters 2014年第21期113卷 218701-218701页

作者： Dheevatsa Mudigere Sunil D. Sherlekar Santosh Ansumali Parallel Computing Lab Intel Labs Bangalore 560103 India Engineering Mechanics Unit Jawaharlal Nehru Centre for Advanced Scientific Research Jakkur Bangalore 560064 India

We argue that the current heterogeneous computing environment mimics a complex nonlinear system which needs to borrow the concept of time-scale separation and the delayed difference approach from statistical mechanics and nonlinear dynamics. We show that by replacing the usual difference equations approach by a delayed difference equations approach, the sequential fraction of many scientific computing algorithms can be substantially reduced. We also provide a comprehensive theoretical analysis to establish that the error and stability of our scheme is of the same order as existing schemes for a large, well-characterized class of problems.

关键词： HETEROGENEOUS computing COMPUTER algorithms NONLINEAR dynamical systems STATISTICAL mechanics DIFFERENCE equations

来源：评论

学校读者我要写书评

暂无评论

Data structure and movement for lattice-based simulations

引用

Physical Review E 2013年第1期88卷 013314-013314页

作者： Aniruddha G. Shet Shahajhan H. Sorathiya Siddharth Krithivasan Anand M. Deshpande Bharat Kaul Sunil D. Sherlekar Santosh Ansumali Parallel Computing Lab Intel Labs Bangalore 560103 India Engineering Mechanics Unit Jawaharlal Nehru Centre for Advanced Scientific Research Jakkur Bangalore 560064 India

We show that for the lattice Boltzmann model, the existing paradigm in computer science for the choice of the data structure is suboptimal. In this paper we use the requirements of physical symmetry necessary for recovering hydrodynamics in the lattice Boltzmann description to propose a hybrid data layout for the method. This hybrid data structure, which we call a structure of an array of structures, is shown to be optimal for the lattice Boltzmann model. Finally, the possible advantages of establishing a connection between group theoretic symmetry requirements and the construction of the data structure is discussed in the broader context of grid-based methods.

关键词： DATA structures (Computer science) LATTICE theory SIMULATION methods & models LATTICE Boltzmann methods HYDRODYNAMICS COMPUTATIONAL grids (Computer systems)

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：