检索结果-内蒙古大学图书馆

Runtime support for type-safe dynamic Java classes 14th

14th European Conference on Object-Oriented Programming, ECOOP 2000

作者： Malabarba, Scott Pandey, Raju Gragg, Jeff Barr, Earl Barnes, J. Fritz Parallel and Distributed Computing Laboratory Computer Science Department University of California DavisCA95616 United States

ISBN: (纸本)3540676600

Modern software must evolve in response to changing condi- tions. In the most widely used programming environments, code is static and cannot change at runtime. This poses problems for applications that have limited down-time. More support is needed for dynamic evolution. In this paper we present an approach for supporting dynamic evolution of Java programs. In this approach, Java programs can evolve by changing their components, namely classes, during their execution. Changes in a class lead to changes in its instances, thereby allowing evolution of both code and state. The approach promotes compatibility with existing Java applications, and maintains the security and type safety controls imposed by Java's dynamic linking mechanism. Experimental analyses of our implementation indicate that the implementation imposes a moderate performance penalty relative to the unmodified virtual machine. © Springer-Verlag Berlin Heidelberg 2000.

关键词： Java programming language

来源：评论

学校读者我要写书评

暂无评论

Communication Analysis for Multidimensional parallel Training of Large-scale DNN Models 25

Communication Analysis for Multidimensional Parallel Trainin...

引用

25th IEEE International Conferences on High Performance computing and Communications, 9th International Conference on Data Science and Systems, 21st IEEE International Conference on Smart City and 9th IEEE International Conference on Dependability in Sensor, Cloud and Big Data Systems and Applications, HPCC/DSS/SmartCity/DependSys 2023

作者： Lai, Zhiquan Hao, Yanqi Li, Shengwei Li, Dongsheng College of Computer National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing Changsha China

ISBN: (纸本)9798350330014

Multidimensional parallel training has been widely applied to train large-scale deep learning models like GPT-3. The efficiency of parameter communication among training devices/processes is often the performance bottleneck of large model training. Analysis of parameter communication mode and traffic has important reference significance for the research of interconnection network design and computing task scheduling to improve the training performance. In this paper, we analyze the parametric communication modes in typical 3D parallel training (data parallelism, pipeline parallelism, and tensor parallelism), and model the traffic in different communication modes. Finally, taking GPT-3 as an example, we present the communication in its 3D parallel training. © 2023 IEEE.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

Funnel: An Efficient Sparse Attention Accelerator with Multi-Dataflow Fusion 22

Funnel: An Efficient Sparse Attention Accelerator with Multi...

引用

22nd IEEE International Symposium on parallel and Distributed Processing with Applications, ISPA 2024

作者： Ma, Shenghong Xu, Jinwei Jiang, Jingfei Wang, Yaohua Li, Dongsheng National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing College of Computer Changsha China

ISBN: (纸本)9798331509712

The self-attention mechanism is the core component of Transformer, which provides a powerful ability to understand the sequence context. However, the self-attention mechanism also suffers from a large amount of redundant computation. Model sparsification can effectively reduce computational load, but the irregularity of non-zeros introduced by sparsification significantly decreases hardware efficiency. This paper proposes Funnel, an accelerator that dynamically predicts sparse attention patterns and efficiently processes unstructured sparse data. Firstly, we adopt a fast quantization method based on lookup table to minimize the cost of sparse patterns prediction. Secondly, we propose Funnel computing Unit (FCU), a hardware architecture that efficiently handles sparse attention through multi-dataflow fusion. Sampled Dense-Dense Matrix Multiplication (SDDMM) and Sparse-Dense Matrix Multiplication (SpMM) are core components of sparse attention mechanism. FCU unifies the computation ways of matrix inner product and row-wise product to support SDDMM and SpMM at the same time, which greatly reduces the storage and movement overhead of intermediate results. Lastly, we devise a lightweight buffer and data tiling strategy tailored to the proposed accelerator, aimed at enhancing data reuse. Experiments demonstrate that our accelerator achieves 0.10-0.25 sparsity with small accuracy loss. When computing the self-attention layer, it attains hardware efficiency ranging from 60% to 85%. Compared to CPU and GPU, it achieves 5.60x and 8.20x speedup. Compared to the state-of-the-art attention accelerators A3, SpAtten, FTRANS, and Sanger, it achieves 7.37x, 4.52x, 9.58x, and 3.08x speedup. © 2024 IEEE.

关键词： FPGA Funnel computing unit Sparse attention Transformer

来源：评论

学校读者我要写书评

暂无评论

Speculative symbolic execution

Speculative symbolic execution

引用

2012 IEEE 23rd International Symposium on Software Reliability Engineering, ISSRE 2012

作者： Zhang, Yufeng Chen, Zhenbang Wang, Ji National Laboratory for Parallel and Distributed Processing Department of Computing Science National University of Defense Technology Changsha China

ISBN: (纸本)9780769548883

Symbolic execution is an effective path oriented and constraint based program analysis technique. Recently, there is a significant development in the research and application of symbolic execution. However, symbolic execution still suffers from the scalability problem in practice, especially when applied to large-scale or very complex programs. In this paper, we propose a new fashion of symbolic execution, named Speculative Symbolic Execution (SSE), to speed up symbolic execution by reducing the invocation times of constraint solver. In SSE, when encountering a branch statement, the search procedure may speculatively explore the branch without regard to the feasibility. Constraint solver is invoked only when the speculated branches are accumulated to a specified number. In addition, we present a key optimization technique that enhances SSE greatly. We have implemented SSE and the optimization technique on Symbolic Pathfinder (SPF). Experimental results on six programs show that, our method can reduce the invocation times of constraint solver by 20.7% to 48.7% (with an average of 29.9%), and save the search time from 23.6% to 43.6% (with an average of 30%). © 2012 IEEE.

关键词： Java programming language

来源：评论

学校读者我要写书评

暂无评论

Implementing real-time partitioned convolution algorithms on conventional operating systems

Implementing real-time partitioned convolution algorithms on...

引用

14th International Conference on Digital Audio Effects, DAFx 2011

作者： Battenberg, Eric Avižienis, Rimas Parallel Computing Laboratory Department of Electrical Engineering and Computer Science University of California Berkeley CA United States

ISBN: (纸本)9782954035109

We describe techniques for implementing real-time partitioned convolution algorithms on conventional operating systems using two different scheduling paradigms: time-distributed (cooperative) and multi-threaded (preemptive). We discuss the optimizations applied to both implementations and present measurements of their performance for a range of impulse response lengths on a recent high-end desktop machine. We find that while the time-distributed implementation is better suited for use as a plugin within a host audio application, the preemptive version was easier to implement and significantly outperforms the time-distributed version despite the overhead of frequent context switches.

关键词： Impulse response

来源：评论

学校读者我要写书评

暂无评论

Handoff of application sessions across time and space

Handoff of application sessions across time and space

引用

International Conference on Communications (ICC2001)

作者： Phan, T. Xu, K. Guy, R. Bagrodia, R. Parallel Computing Laboratory Computer Science Department University of California Los Angeles CA 90095 United States

Personal computing on mobile platforms such as laptops and personal digital assistants, rather than in a traditional desktop environment, is becoming increasingly more common. In this paper we address the issue of application session transfer for uninterrupted data access across this diverse range of platforms. This work is part of the iMASH project, a multi-year, multi-discipline collaborative effort focused on enabling mobile client platforms and incorporating them into existing legacy networked systems for use by medical practitioners. We have developed a tiered architecture that includes a middleware server layer positioned between existing application servers and multiple clients to make session transfer transparent to the user. Any client application executing our Middleware-Aware Remote Code library can save and restore its session by interacting with a middleware server. As a proof of concept, we have implemented the transfer of bookmarks, history, web cache, and user preferences with the Mozilla open source web browser. From this effort we have established baseline performance metrics and have found that the overhead is within reasonable bounds of just a few seconds of latency.

关键词： Mobile computing

来源：评论

学校读者我要写书评

暂无评论

Efficient Large Models Fine-tuning on Commodity Servers via Memory-balanced Pipeline parallelism 25

Efficient Large Models Fine-tuning on Commodity Servers via ...

引用

作者： Liu, Yujie Lai, Zhiquan Liu, Weijie Wang, Wei Li, Dongsheng College of Computer National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing Changsha China

ISBN: (纸本)9798350330014

Large models have achieved impressive performance in many downstream tasks. Using pipeline parallelism to fine-tune large models on commodity GPU servers is an important way to make the excellent performance of large models available to the general public. Previous solutions fail to achieve an efficient memory-balanced pipeline parallelism. In this poster, we introduce a memory load-balanced pipeline parallel solution. This solution balances memory consumption across stages on commodity GPU servers via NVLink bridges. It establishes a new pathway to offload data from GPU to CPU by using the PCIe link of adjacent GPUs connected by the NVLink bridge. Furthermore, our method orchestrates offload operations to minimize the offload latency during large model fine-tuning. Experiments demonstrate that our solution can balance the memory footprint among pipeline stages without sacrificing training performance. © 2023 IEEE.

关键词： Program processors

来源：评论

学校读者我要写书评

暂无评论

Highly Optimized Code Generation for Stencil Codes with Computation Reuse for GPUs

引用

Journal of Computer Science & Technology 2016年第6期31卷 1262-1274页

作者： Wen-Jing Ma Kan Gao Guo-Ping Long Laboratory of Parallel Software and Computing Science Institute of Software Chinese Academy of Sciences Beijing 100190 China State Key Laboratory of Computer Science Institute of Software Chinese Academy of Sciences Beijing 100190 China Information Center China Association for Science and Technology Beijing 100863 China

Computation reuse is known as an effective optimization technique. However, due to the complexity of modern GPU architectures, there is yet not enough understanding regarding the intriguing implications of the interplay of compu- ration reuse and hardware specifics on application performance. In this paper, we propose an automatic code generator for a class of stencil codes with inherent computation reuse on CPUs. For such applications, the proper reuse of intermediate results, combined with careful register and on-chip local memory usage, has profound implications on performance. Current state of the art does not address this problem in depth, partially due to the lack of a good program representation that can expose all potential computation reuse. In this paper, we leverage the computation overlap graph （COG）, a simple representation of data dependence and data reuse with ＂element view＂, to expose potential reuse opportunities. Using COG, we propose a portable code generation and tuning framework for GPUs. Compared with current state-of-the-art code generators, our experimental results show up to 56.7% performance improvement on modern GPUs such as NVIDIA C2050.

关键词： GPGPU OpenCL stencil code generation computation reuse

来源：评论

学校读者我要写书评

暂无评论

Impact of geometry aspect ratio on 10-nm gate-all-around silicon-germanium nanowire field effect transistors 14

Impact of geometry aspect ratio on 10-nm gate-all-around sil...

引用

2014 14th IEEE International Conference on Nanotechnology, IEEE-NANO 2014

作者： Chao, Pei-Jung Li, Yiming Parallel and Scientific Computing Laboratory Institute of Biomedical Engineering National Chaio Tung University Hsinchu300 Taiwan

ISBN: (纸本)9781479956227

In this paper, we study electrical characteristics of gate-all-around (GAA) silicon-germanium (SiGe) nanowire field effect transistors (NWFETS) with different aspect ratio (AR) of channel. Device characteristics: the subthreshold swing (SS), the drain induced barrier lowering (DIBL), and the ION/IOFF ratio are simulated by using three-dimensional quantum mechanically corrected device simulation. Electrical characteristics of 10-nm-gate GAA Si1-xGex NWFET devices are explored with respect to different thickness of SiGe and Ge's mole fraction. It is investigated that an ellipse-shaped channel with a small aspect ratio possesses better DC characteristics, compared with the one which has large AR due to its good gate controllability. © 2014 IEEE.

关键词： Aspect ratio

来源：评论

学校读者我要写书评

暂无评论

Rethinking the Distributed DNN Training Cluster Design from the Cost-effectiveness View 25

Rethinking the Distributed DNN Training Cluster Design from ...

引用

作者： Lai, Zhiquan Liu, Yujie Wang, Wei Hao, Yanqi Li, Dongsheng College of Computer National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing Changsha China

ISBN: (纸本)9798350330014

As deep learning grows rapidly, model training heavily relies on parallel methods and there exist numerous cluster configurations. However, current preferences for parallel training focus on data centers, overlooking the financial constraints faced by most researchers. To attain the best performance within the cost limitation, we introduce a throughput-cost metric to accurately characterize clusters' cost-effectiveness. Based on this metric, we design a cost-effective cluster featuring the 3090 with NVLink. The experiment results demonstrate that our cluster achieves remarkable cost-effectiveness in various distributed model training schemes. © 2023 IEEE.

关键词： Cost effectiveness

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：