检索结果-内蒙古大学图书馆

9th International Conference on Cyber Security and Information Engineering, ICCSIE 2024

作者： Shen, Ao Lai, Zhiquan Li, Dongsheng National Key Laboratory of Parallel and Distributed Computing National University of Defense Technology China

ISBN: (纸本)9798400718137

Breakthroughs in natural language processing (NLP) by large-scale language models (LLMs) have led to superior performance in multilingual tasks such as translation, summarization, and Q&A. However, the size and complexity of these models raise challenges in terms of computational requirements, memory usage, and energy consumption. Quantization strategies, as a type of model compression technique, have gained attention for their advantages in reducing model size and accelerating inference speed. In this paper, we review the rapid development of quantization techniques for LLMs, and systematically explore methods such as post-training quantization (PTQ), quantization-aware fine-tuning (QAF), and quantization-aware training (QAT), which provide a comprehensive solution to the resource-intensive problem of LLMs from training to post-deployment. We also analyze state-of-the-art benchmarks and datasets to evaluate the effectiveness of quantitative methods in terms of performance retention and computational efficiency. The purpose of this review is to provide researchers with a snapshot of the latest progress in LLM quantization techniques, helping them to quickly grasp the dynamics of the field and understand the key techniques and challenges, so that they can more efficiently devote themselves to this evolving research area. © 2024 Copyright held by the owner/author(s).

关键词： Natural language processing systems

来源：评论

学校读者我要写书评

暂无评论

Detailed and clock-driven simulation for HPC interconnection network

引用

Frontiers of Computer Science 2016年第5期10卷 797-811页

作者： Wenhao ZHOU Juan CHEN Chen CUI Qian WANG Dezun DONG Yuhua TANG State Key Laboratory of High Performance Computing School of Computer National University of Defense Technology Changsha 410073 China Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense Technology Changsha 410073 China

Performance and energy consumption of high performance computing （HPC） interconnection networks have a great significance in the whole supercomputer, and building up HPC interconnection network simulation plat- form is very important for the research on HPC software and hardware technologies. To effectively evaluate the per- formance and energy consumption of HPC interconnection networks, this article designs and implements a detailed and clock-driven HPC interconnection network simulation plat- form, called HPC-NetSim. HPC-NetSim uses application- driven workloads and inherits the characteristics of the de- tailed and flexible cycle-accurate network simulator. Besides, it offers a large set of configurable network parameters in terms of topology and routing, and supports router＇s on/off states. We compare the simulated execution time with the real execution time of Tianhe-2 subsystem and the mean error is only 2.7%. In addition, we simulate the network behaviors with different network structures and low-power modes. The results are also consistent with the theoretical analyses.

关键词： high performance computing clock-driven sim-ulation interconnection network BookSim

来源：评论

学校读者我要写书评

暂无评论

Detecting Duplicate Contributions in Pull-Based Model CombiningTextual and Change Similarities

引用

Journal of Computer Science & Technology 2021年第1期36卷 191-206页

作者： Zhi-Xing Li Yue Yu Tao Wang Gang Yin Xin-Jun Mao Huai-Min Wang Key Laboratory of Parallel and Distributed Computing College of ComputerNational University of Defense Technology Changsha 410073China Laboratory of Software Engineering for Complex Systems College of ComputerNational University of Defense TechnologyChangsha 410073China

Communication and coordination between OSS developers who do not work physically in the same location have always been the challenging *** pull-based development model,as the state-of-art collaborative development mechanism,provides high openness and transparency to improve the visibility of contributors'***,duplicate contributions may still be submitted by more than one contributors to solve the same problem due to the parallel and uncoordinated nature of this *** not detected in time,duplicate pull-requests can cause contributors and reviewers to waste time and energy on redundant *** this paper,we propose an approach combining textual and change similarities to automatically detect duplicate contributions in pull-based model at submission *** a new-arriving contribution,we first compute textual similarity and change similarity between it and other existing *** then our method returns a list of candidate duplicate contributions that are most similar with the new contribution in terms of the combined textual and change *** evaluation shows that 83.4%of the duplicates can be found in average when we use the combined textual and change similarity compared to 54.8%using only textual similarity and 78.2%using only change similarity.

关键词： Pull-request Duplicate detection textual similarity change similarity

来源：评论

学校读者我要写书评

暂无评论

Multi-class contour preserving classification

Multi-class contour preserving classification

引用

13th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2012

作者： Fuangkhon, Piyabute Tanprasert, Thitipong Distributed and Parallel Computing Research Laboratory Faculty of Science and Technology Assumption University Bangkok Thailand

ISBN: (纸本)9783642326387

The original contour preserving classification technique was proposed to improve the robustness and weight fault tolerance of a neural network applied with a two-class linearly separable problem. It was recently found to be improving the level of accuracy of two-class classification. This paper presents an augmentation of the original technique to improve the level of accuracy of multi-class classification by better preservation of the shape or distribution model of a multi-class problem. The test results on six real world multi-class datasets from UCI machine learning repository present that the proposed technique supports multi-class data and can improve the level of accuracy of multi-class classification more effectively. © 2012 Springer-Verlag.

关键词： Neural networks

来源：评论

学校读者我要写书评

暂无评论

A multidimensional approach of evaluating developers 2020

A multidimensional approach of evaluating developers

引用

2nd International Conference on Big Data Engineering, BDE 2020

作者： Zhang, Changqiang Chen, Ming Key Laboratory of Parallel and Distributed Computing College of Computer National University of Defense Technology China

ISBN: (纸本)9781450377225

In this paper, we propose an approach to assess the ability of developers based on their behavior data from OSS. Specifically, we classify developers' ability into code ability, project management ability, and social ability. Code efficiency is related to the developer's commit record and the pull-request record. The developer's project management ability is achieved by tracking the developer's commit record. We use regular matching to map the commit behavior to the project management behavior and calculate the developer's project management ability according to the proportion of different behaviors. The social ability of developers is related to the data that developers interact with in the open-source community. We dug for developer reviews on commit, issue, and gist fragments. By calculating the proportion of positive emotions in developer reviews and the proportion of developers interacting with others in the reviews, the social ability of developers is obtained. We get behavioral data from 50 random developers. Twitter's data is used to test the effect of different machine learning algorithms on the accuracy of developer comment polarity judgments. It is found that the combination of SVM, xgboost and random forest have the highest prediction accuracy. Finally, we select 5 students to use Likert scale to score the results. Our score shows that the results are basically in line with expectations. © 2020 ACM.

关键词： Decision trees

来源：评论

学校读者我要写书评

暂无评论

U-shaped Dual Attention Transformer: An Efficient Transformer Based on Channel and Spatial Attention 4

U-shaped Dual Attention Transformer: An Efficient Transforme...

引用

4th International Conference on Artificial Intelligence, Robotics, and Communication, ICAIRC 2024

作者： Zhai, Zhaoyuan Qiao, Peng Li, Rongchun Zhou, Zhen National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing Changsha China

ISBN: (纸本)9798331531225

Transformer-based methods have demonstrated remarkable performance on image super-resolution tasks. Due to high computational complexity, researchers have been working to achieve a balance between computation costs and performance. Restormer has achieved commendable balance by utilizing global channel attention. However, the performance is limited by insufficient local pixel reconstruction. In this paper, we propose a U-shaped Dual Attention Transformer (UDAT) with local-global receptive field, addressing the limitation of Restormer in local pixel reconstruction. We propose a dense window channel attention to enhance the local feature representation, more efficient in computational complexity. Experiments demonstrate that our UDAT achieve superior performance compared with Restormer on benchmark datasets, surpassing Restormer by 0.57 dB on the Urban100 dataset. On-par with SwinIR, our method reduces computational complexity by 3.2 times and improves inference speed by 2 times, achieving a better balance between computational costs and performance. © 2024 IEEE.

关键词： Pixels

来源：评论

学校读者我要写书评

暂无评论

parallel Accelerating Ultra-Long Read Alignment by Vertical Partitioning Data 13

Parallel Accelerating Ultra-Long Read Alignment by Vertical ...

引用

13th IEEE International Symposium on parallel Architectures, Algorithms and Programming, PAAP 2022

作者： Pan, Deng Zhong, Cheng Chen, Danyang Zhang, Jinxiong Yang, Feng Guangxi University Key Laboratory of Parallel Distributed Computing Technology Guangxi Universities School of Computer Electronics and Information Guangxi Nanning China Guangxi University Key Laboratory of Parallel Distributed Computing School of Computer Electronics and Information Guangxi Nanning China

ISBN: (纸本)9781665452182

The alignment between sequencing reads and genome is a basic work in biological big data analysis. Each read of the third generation sequencing data is getting longer, and the data size is getting larger. To effectively solve the ultra-long read alignment problem with high requirements for computing and memory capacity, a strategy for vertical partitioning ultra-long reads on hybrid CPU/GPU cluster is proposed, and a heap data structure is used to filter the local aligned results in all computing nodes of the parallel cluster system according to the alignment score to reduce the data transmission size. The methods for early termination and parallel merging-splicing are used to accelerate splicing local aligned results. The local aligned results among all computing nodes are collected and extended to obtain the final alignment results. The experimental results on datasets of simulated and real ultra-long reads show that the proposed parallel alignment algorithm can obtain high alignment accuracy, sensitivity and base-level sensitivity as a whole, and accelerate completing alignment between ultra-long reads and reference genome. © 2022 IEEE.

关键词： Alignment

来源：评论

学校读者我要写书评

暂无评论

Optimization of HPF programs with dynamic recompilation technique

Optimization of HPF programs with dynamic recompilation tech...

引用

4th International Symposium on High Performance computing, ISHPC 2002

作者： Araki, Takuya Murai, Hitoshi Kamachi, Tsunehiko Seo, Yoshiki Parallel and Distributed Systems NEC Laboratory Real World Computing Partnership United States

ISBN: (纸本)354043674X

Optimizing compilers perform various optimizations in order to exploit the best performance from computer systems. However, some kinds of optimizations cannot be applied if values of variables or system parameters are not known at compilation time. To solve this problem, we designed and implemented a system which collects such information at run time, and dynamically recompiles part of the program based on it. In our system, recompilation and management of runtime information are carried out on processors other than those which execute user programs. Therefore, recompilation cost does not affect the program execution time, unlike other similar systems. The evaluation result shows that quite high speedup can be attained with this method. © Springer-Verlag Berlin Heidelberg 2002.

关键词： Information management

来源：评论

学校读者我要写书评

暂无评论

Providing dynamic and customizable caching policies 2

Providing dynamic and customizable caching policies

引用

2nd USENIX Symposium on Internet Technologies and Systems, USITS 1999

作者： Barnes, J. Fritz Pandey, Raju Parallel and Distributed Computing Laboratory Computer Science Department University of California DavisCA95616 United States

Web caching has emerged as one solution for improving client latency on the web. Cache effectiveness depends on the policies used to route requests to other caches and servers, to maintain up-to-date web objects and to remove objects from the cache. Traditional caches apply one set of policies, which determines the efficiency as well as the effectiveness of the caches. This set of policies often does not exploit the diversity inherent in different web objects, caches and clients. Policies that do exploit this diversity result in convoluted caching policies that attempt to combine multiple policies and guess at the unknown characteristics of web objects, caches and clients. In this paper, we present an extensible caching infrastructure in which cache administrators, servers, and end users can customize how web objects are cached, replaced, and kept consistent. The infrastructure includes a domain-specific language, CacheL, for defining customizable caching policies that can be changed dynamically. Analysis of our prototype, PoliSquid, shows the benefits of the infrastructure for variable coherency policies, localized removal policies, and early removal of objects from servers. © 1999 by The USENIX Association All Rights Reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Communication Analysis for Multidimensional parallel Training of Large-scale DNN Models 25

Communication Analysis for Multidimensional Parallel Trainin...

引用

25th IEEE International Conferences on High Performance computing and Communications, 9th International Conference on Data Science and Systems, 21st IEEE International Conference on Smart City and 9th IEEE International Conference on Dependability in Sensor, Cloud and Big Data Systems and Applications, HPCC/DSS/SmartCity/DependSys 2023

作者： Lai, Zhiquan Hao, Yanqi Li, Shengwei Li, Dongsheng College of Computer National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing Changsha China

ISBN: (纸本)9798350330014

Multidimensional parallel training has been widely applied to train large-scale deep learning models like GPT-3. The efficiency of parameter communication among training devices/processes is often the performance bottleneck of large model training. Analysis of parameter communication mode and traffic has important reference significance for the research of interconnection network design and computing task scheduling to improve the training performance. In this paper, we analyze the parametric communication modes in typical 3D parallel training (data parallelism, pipeline parallelism, and tensor parallelism), and model the traffic in different communication modes. Finally, taking GPT-3 as an example, we present the communication in its 3D parallel training. © 2023 IEEE.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：