检索结果-内蒙古大学图书馆

IEEE International Conference on High Performance Computing and Communications (HPCC)

作者： Yujie Liu Zhiquan Lai Weijie Liu Wei Wang Dongsheng Li National Key Laboratory of Parallel and Distributed Computing College of Computer National University of Defense Technology Changsha China

Large models have achieved impressive performance in many downstream tasks. Using pipeline parallelism to fine-tune large models on commodity GPU servers is an important way to make the excellent performance of large models available to the general public. Previous solutions fail to achieve an efficient memory-balanced pipeline parallelism. In this poster, we introduce a memory load-balanced pipeline parallel solution. This solution balances memory consumption across stages on commodity GPU servers via NVLink bridges. It establishes a new pathway to offload data from GPU to CPU by using the PCIe link of adjacent GPUs connected by the NVLink bridge. Furthermore, our method orchestrates offload operations to minimize the offload latency during large model fine-tuning. Experiments demonstrate that our solution can balance the memory footprint among pipeline stages without sacrificing training performance.

关键词：

来源：评论

学校读者我要写书评

暂无评论

The parallelized Cuckoo Filter for Cold Data Representation 23

The Parallelized Cuckoo Filter for Cold Data Representation

引用

23rd IEEE International Conference on High Performance Computing and Communications, 7th IEEE International Conference on Data Science and Systems, 19th IEEE International Conference on Smart City and 7th IEEE International Conference on Dependability in Sensor, Cloud and Big Data Systems and Applications, HPCC-DSS-SmartCity-DependSys 2021

作者： Sun, Bowen Luo, Lailong Li, Shangsen Chen, Yingwen Guo, Deke College of Computer National University of Defense Technology China National University of Defense Technology Science and Technology on Information Systems Engineering Laboratory China College of Computer National University of Defense Technology National Lab for Parallel and Distributed Processing China

ISBN: (纸本)9781665494571

Cold data contributes a large portion of the big data today and is usually stored in secondary storage. Various sketch data structures are implemented to represent the stored elements and provide constant-time membership queries. Considering the large volume and infrequent access of cold data, we envision three rationales of cold data representation: 1) parallelized insertion, 2) accurate representation, and 3) constant-time query/deletion. The existing sketches, however, fail to achieve these rationales simultaneously. To this end, we propose a novel sketch named parallelized cuckoo Filter (PCF) in this paper. PCF divides the standard cuckoo filter into multiple homogenous segments. An additional hash function is introduced to assign an element to a segment. The element will be inserted into the selected segment. With such a design, element insertions in PCF can be naturally parallelized. Besides, as a variant of the cuckoo filter, PCF naturally realizes the accurate representation and constant-time query/deletion requirements. Comprehensive experiments further indicate that PCF achieves the three rationales simultaneously. © 2021 IEEE.

关键词： Hash functions Smart cities Big Data Data structures Standards

来源：评论

学校读者我要写书评

暂无评论

Evaluating a New Attention Framework Based on Matrix Blocking for Attention Models on FPGAs

Evaluating a New Attention Framework Based on Matrix Blockin...

引用

International Conference on Tools for Artificial Intelligence (ICTAI)

作者： Xiaohang Liu Jingfei Jiang Jinwei Xu Lei Gao National Laboratory for Parallel and Distributed Processing National University of Defense Technology Changsha China

The attention mechanism has recently shown superior performance in natural language processing and computer vision tasks. But its complex dataflow and large-scale matrix calculation with huge computing and memory overhead pose a great challenge for the design of hardware accelerators. And previous solutions that benefited from matrix partitioning are bounded by the softmax function. In this paper, we propose a new attention framework that can dramatically improve the performance of attention model inference for long sequence tasks on FPGAs. We design a novel accelerator architecture that employs two systolic arrays and a ping-pong structure to accelerate attention calculation. Meanwhile, we propose an analytical model to predict resource usage and performance, which guides a fast design space exploration. Experiments using the state-of-the-art BERT demonstrate the design achieves 4.61 and 1.24× improvement in speed and energy efficiency compared to CPU and GPU on the Xilinx XCZU11EG platform.

关键词： Performance evaluation Graphics processing units Accelerator architectures Systolic arrays Natural language processing Mobile handsets Space exploration

来源：评论

学校读者我要写书评

暂无评论

An empirical study on the structure evolution of deep learning models: taking SAR image processing a case study

An empirical study on the structure evolution of deep learni...

引用

IEEE International Conference on Joint Cloud Computing (JCC)

作者： Huanxi Liu Xiang He Dawei Feng Han Bao National Laboratory for Parallel and Distributed Processing (PDL) National University of Defense Technology China School of Information and Communication National University of Defense Technology China

With the continuous improvement on model performance, deep learning models have been widely deployed and achieved promising outcomes in various fields in recent years. However, due to the escalating volumes of training data and the complexity of application problems, it becomes more and more challenging to design a neural network with better performance by hand. Analysing the evolution of typical neural network structures has important reference significance for designing a network structure. In this paper, we select the open source models in SAR image processing for an empirical analysis on the evolution of neural network structures. We analyse the evolution of 239 open source deep learning models from the aspects of framework, computing unit, model computation amount and the combined use of various computing units. Results reveal that preference and co-occurrence exist in computing units, while the average number of convolution, activation and normalization layer increases significantly over time. Model complexity shows an overall upward trend, and the characteristics of SAR image are more and more taken into consideration during the model structure design.

关键词：

来源：评论

学校读者我要写书评

暂无评论

DaMSTF: Domain Adversarial Learning Enhanced Meta Self-Training for Domain Adaptation

arXiv

引用

arXiv 2023年

作者： Lu, Menglong Huang, Zhen Zhao, Yunxiang Tian, Zhiliang Liu, Yang Li, Dongsheng National Key Laboratory of Parallel and Distributed Computing National University of Defense Technology China Beijing Institute of Biotechnology China

Self-training emerges as an important research line on domain adaptation. By taking the model’s prediction as the pseudo labels of the unlabeled data, self-training bootstraps the model with pseudo instances in the target domain. However, the prediction errors of pseudo labels (label noise) challenge the performance of self-training. To address this problem, previous approaches only use reliable pseudo instances, i.e., pseudo instances with high prediction confidence, to retrain the model. Although these strategies effectively reduce the label noise, they are prone to miss the hard examples. In this paper, we propose a new self-training framework for domain adaptation, namely Domain adversarial learning enhanced Self-Training Framework (DaMSTF). Firstly, DaMSTF involves meta-learning to estimate the importance of each pseudo instance, so as to simultaneously reduce the label noise and preserve hard examples. Secondly, we design a meta constructor for constructing the meta validation set, which guarantees the effectiveness of the meta-learning module by improving the quality of the meta validation set. Thirdly, we find that the meta-learning module suffers from the training guidance vanishment and tends to converge to an inferior optimal. To this end, we employ domain adversarial learning as a heuristic neural network initialization method, which can help the meta-learning module converge to a better optimal. Theoretically and experimentally, we demonstrate the effectiveness of the proposed DaMSTF. On the cross-domain sentiment classification task, DaMSTF improves the performance of BERT with an average of nearly 4%. © 2023, CC BY-NC-SA.

关键词： Forecasting

来源：评论

学校读者我要写书评

暂无评论

DMSA: Decentralized and Multi-keyword Selective Data Sharing and Acquisition

DMSA: Decentralized and Multi-keyword Selective Data Sharing...

引用

International Symposium on parallel and distributed processing with Applications, ISPA

作者： Moheng Lin Peichang Shi Xiang Fu Feng Jiang Guodong Yi National Key Laboratory of Parallel and Distributed Computing College of Computer Science National University of Defense Technology Changsha China Xiangjiang Lab Changsha China

ISBN: (数字)9798331509712

ISBN: (纸本)9798331509729

Blockchain technology has been extensively uti-lized in decentralized data-sharing applications, with the immutability of blockchain providing a witness for the circulation of data. However, current blockchain data-sharing solutions still fail to address the simultaneous screening needs of both the sender and receiver with multi-keywords. Without the capability to support bilateral simultaneous filtering, the disclosure of reasons for matching failures could inadvertently expose sensitive user data. Therefore, the challenge lies in enabling ciphertexts with multiple keywords and receivers with multiple interests to achieve mutual and simultaneous matching. Based on the technical foundations of SE (Searchable Encryption), MABE (Multi-Attribute Based Encryption), and polynomial fitting, this paper proposes a scheme called DMSA (Decentralized and Multi-keyword selective Sharing and selective Acquisition). This scheme can satisfy soundness, enabling ciphertexts carrying multiple keywords and receivers representing multiple interests to match each other simultaneously. We conducted a security analysis that confirms the security of DMSA against chosen-plaintext attacks. Our experimental results demonstrate a significant efficiency improvement, with a 67% increase over single-keyword data-sharing schemes and a 16% enhancement compared to the existing multi-keyword data-sharing solution.

关键词： distributed processing Filtering Data security keyword search Fitting Receivers Polynomials Data models Blockchains Encryption

来源：评论

学校读者我要写书评

暂无评论

HAF: a hybrid annotation framework based on expert knowledge and learning technique

引用

Science China(Information Sciences) 2022年第1期65卷 276-278页

作者： Zhixing LI Yue YU Tao WANG Gang YIN Xinjun MAO Huaimin WANG Key Laboratory of Parallel and Distributed Computing National University of Defense Technology College of Computer National University of Defense Technology

Dear editor,The increasing awareness of the potential value hidden in data has resulted in many data mining studies being conducted. In the domain of software engineering, for example, developers' behavioral data and code review data have been leveraged in social coding sites to automatically recommend relevant projects [1] and candidate reviewers [2, 3].

关键词：

来源：评论

学校读者我要写书评

暂无评论

A survey of script learning

引用

Frontiers of Information Technology & Electronic Engineering 2021年第3期22卷 341-373页

作者： Yi HAN Linbo QIAO Jianming ZHENG Hefeng WU Dongsheng LI Xiangke LIAO Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense TechnologyChangsha 410000China Science and Technology on Information Systems Engineering Laboratory National University of Defense TechnologyChangsha 410000China School of Data and Computer Science Sun Yat-sen UniversityGuangzhou 510006China

Script is the structured knowledge representation of prototypical real-life event *** the commonsense knowledge inside the script can be helpful for machines in understanding natural language and drawing commonsensible *** learning is an interesting and promising research direction,in which a trained script learning system can process narrative texts to capture script knowledge and draw ***,there are currently no survey articles on script learning,so we are providing this comprehensive survey to deeply investigate the standard framework and the major research topics on script *** research field contains three main topics:event representations,script learning models,and evaluation *** each topic,we systematically summarize and categorize the existing script learning systems,and carefully analyze and compare the advantages and disadvantages of the representative *** also discuss the current state of the research and possible future directions.

关键词： Script learning Natural language processing Commonsense knowledge modeling Event reasoning

来源：评论

学校读者我要写书评

暂无评论

A prompt-based approach to adversarial example generation and robustness enhancement

引用

Frontiers of Computer Science 2024年第4期18卷 85-96页

作者： Yuting YANG Pei HUANG Juan CAO Jintao LI Yun LIN Feifei MA Key Lab of Intelligent Information Processing of Chinese Academy of Sciences(CAS) Institute of Computing TechnologyCASBeijing 100190China School of Computer Science and Technology University of Chinese Academy of SciencesBeijing 100049China Department of Computer Science Stanford UniversityCA 94305USA School of Computing National University of SingaporeSingapore 119077Singapore Laboratory of Parallel Software and Computational Science Institute of SoftwareChinese Academy of SciencesBeijing 100190China

Recent years have seen the wide application of natural language processing(NLP)models in crucial areas such as finance,medical treatment,and news media,raising concerns about the model robustness and *** find that prompt paradigm can probe special robust defects of pre-trained language *** prompt texts are first constructed for inputs and a pre-trained language model can generate adversarial examples for victim models via *** results show that prompt paradigm can efficiently generate more diverse adversarial examples besides synonym ***,we propose a novel robust training approach based on prompt paradigm which incorporates prompt texts as the alternatives to adversarial examples and enhances robustness under a lightweight minimax-style optimization *** on three real-world tasks and two deep neural models show that our approach can significantly improve the robustness of models to resist adversarial attacks.

关键词： robustness adversarial example prompt learning pre-trained language model

来源：评论

学校读者我要写书评

暂无评论

F3A: Fairness-Aware AI-Workloads Allocation Considering Multidimensional User Demands in JointCloud

F3A: Fairness-Aware AI-Workloads Allocation Considering Mult...

引用

IEEE International Conference on Joint Cloud Computing (JCC)

作者： Jiacheng Yang Guodong Yi Fei Gao Peichang Shi Huaimin Wang National Key Laboratory of Parallel and Distributed Processing College of Computer Science National University of Defense Technology Changsha China Xiangjiang Lab Changsha China School of Advanced Interdisciplinary Studies Hunan University Of Technology and Business Changsha China

ISBN: (数字)9798350387339

ISBN: (纸本)9798350387346

With the rapid growth of large language models, cloud computing has become an indispensable component of the AI industry. Cloud service providers(CSPs) are establishing AI data centers to service AI workloads. In the face of this surging need for AI computing power, building a connected computing environment across various clouds and forming a JointCloud presents an attractive solution. However, scheduling AI tasks across multiple AI data centers within a JointCloud environment presents a significant challenge: how to balance users’ demands while ensuring CSPs’ fairness in scheduling. Existing research primarily focuses on optimizing scheduling quality with limited consideration for fairness. Therefore, this paper proposes a Fairness-Aware AI-Workloads Allocation method (F3A), a fair cross-cloud allocation technique for AI tasks. F3A utilizes Point and Token to reflect both the resource status and historical task allocations of AI data centers, enabling the consideration of users’ multidimensional demands and facilitating fair task allocation across multiple centers. In order to better assess the fairness of scheduling, we also devised a fairness indicator(FI), based on the Gini coefficient to measure the fairness of task allocation. The experimental results demonstrate that F3A consistently maintains FI within 0.1 across various cluster sizes and different task quantities, representing an improvement of 76.45% compared to classical fair scheduling algorithms round-robin. F3A exhibits commendable performance in ensuring fairness in task allocation while also demonstrating effectiveness in cost reduction and enhancing user satisfaction.

关键词： Industries Data centers Cloud computing Job shop scheduling Costs Scheduling algorithms Large language models

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：