检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Chen, Zhongwu Xu, Chengjin Wang, Dingmin Huang, Zhen Dou, Yong Jiang, Xuhui Guo, Jian National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology China IDEA Research International Digital Economy Academy Department of Computer Science University of Oxford United Kingdom

Retrieval-augmented generation (RAG) has shown promising potential in knowledge intensive question answering (QA). However, existing approaches only consider the query itself, neither specifying the retrieval preferences for the retrievers nor informing the generators of how to refer to the retrieved documents for the answers, which poses a significant challenge to the QA performance. To address these issues, we propose Rule-guided Retrieval-Augmented Generation with LMs, which explicitly introduces rules for in-context learning (RuleRAG-ICL) to guide retrievers to recall related documents in the directions of rules and uniformly guide generators to reason attributed by the same rules. The combination of queries and rules can be used as fine-tuning data to update retrievers and generators, achieving better rule-based instruction-following ability (RuleRAG-FT). Moreover, most existing RAG datasets were constructed without considering rules and Knowledge Graphs (KGs) are recognized as providing high-quality rules. Therefore, we construct five rule-aware RAG benchmarks for QA, RuleQA, based on KGs to stress the significance of retrieval and reasoning with rules. Experiments on RuleQA demonstrate RuleRAG-ICL improves the retrieval quality of +89.2% in Recall@10 and answer accuracy of +103.1% in Exact Match, and RuleRAG-FT yields more enhancement. In addition, experiments on four existing RAG datasets show RuleRAG is also effective by offering rules in RuleQA to them, further proving the generalization of rule guidance in RuleRAG. Code and RuleQA are at https://***/r/RuleRAG. Copyright © 2024, The Authors. All rights reserved.

关键词： Question answering

来源：评论

学校读者我要写书评

暂无评论

Asycome: A JointCloud Data Asynchronous Collaboration Mechanism Based on Blockchain 3rd

Asycome: A JointCloud Data Asynchronous Collaboration Mechan...

引用

3rd International Conference on Blockchain and Trustworthy Systems, Blocksys 2021

作者： Li, Linhui Shi, Peichang Fu, Xiang Zhang, Shengtian Zhong, Tao Chen, Ming National Key Laboratory of Parallel and Distributed Processing College of Computer Science National University of Defense Technology Changsha410073 China

ISBN: (纸本)9789811679926

In the Internet of Everything (IoE), due to its issues of complexity and heterogeneity, message delay cannot be guaranteed, and it is not enough to leverage a centralized model for data collaboration. By leveraging the features of blockchain and asynchronous consensus algorithms, a new trust collaboration model can be established in an untrusted environment. Besides, the scalability issue of blockchain poses a challenge to data asynchronous collaboration. In this paper, we propose Asycome, a JointCloud data asynchronous collaboration mechanism based on Directed Acyclic Graph (DAG). This mechanism introduces the new Gossip about gossip protocol, driven by message-transmitting events, to keep data synchronized in an asynchronous environment. Through two stages of voting and committing. Transactions on the same time series do not need to reach the final confirmed state before proceeding to the next transaction processing. Over time, all transactions are connected to the DAG, ensuring that the consensus is finally reached. We evaluated the performance of the mechanism with experiments on throughput and latency. And the feasibility of Asycome is analyzed from the liveness and security. The results show that higher throughput, lower latency, and Asynchronous Byzantine Fault Tolerance (ABFT) characteristics can meet the technical requirements of complex and asynchronous data collaboration of the IoE to some extent. © 2021, Springer Nature Singapore Pte Ltd.

关键词： Blockchain

来源：评论

学校读者我要写书评

暂无评论

Local-Adaptive Transformer for Multivariate Time Series Anomaly Detection and Diagnosis

Local-Adaptive Transformer for Multivariate Time Series Anom...

引用

IEEE International Conference on Systems, Man and Cybernetics

作者： Xiaohui Zhou Yijie Wang Hongzuo Xu Mingyu Liu Ruyi Zhang National Key Laboratory of Parallel and Distributed Computing College of Computer National University of Defense Technology Changsha China

Time series data are pervasive in varied real-world applications, and accurately identifying anomalies in time series is of great importance. Many current methods are insufficient to model long-term dependence, whereas some anomalies can be only identified through long temporal contextual information. This may finally lead to disastrous outcomes due to false negatives of these anomalies. Prior arts employ Transformers (i.e., a neural network architecture that has powerful capability in modeling long-term dependence and global association) to alleviate this problem; however, Transformers are insensitive in sensing local context, which may neglect subtle anomalies. Therefore, in this paper, we propose a local-adaptive Transformer based on cross-correlation for time series anomaly detection, which unifies both global and local information to capture comprehensive time series patterns. Specifically, we devise a cross-correlation mechanism by employing causal convolution to adaptively capture local pattern variation, offering diverse local information into the long-term temporal learning process. Furthermore, a novel optimization objective is utilized to jointly optimize reconstruction of the entire time series and matrix derived from cross-correlation mechanism, which prevents the cross-correlation from becoming trivial in the training phase. The generated cross-correlation matrix reveals underlying interactions between dimensions of multivariate time series, which provides valuable insights into anomaly diagnosis. Extensive experiments on six real-world datasets demonstrate that our model outperforms state-of-the-art competing methods and achieves 6.8%-27.5% $F_{1}$ score improvement. Our method also has good anomaly interpretability and is effective for anomaly diagnosis.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Rethinking the distributed DNN Training Cluster Design from the Cost-effectiveness View

Rethinking the Distributed DNN Training Cluster Design from ...

引用

IEEE International Conference on High Performance Computing and Communications (HPCC)

作者： Zhiquan Lai Yujie Liu Wei Wang Yanqi Hao Dongsheng Li National Key Laboratory of Parallel and Distributed Computing College of Computer National University of Defense Technology Changsha China

As deep learning grows rapidly, model training heavily relies on parallel methods and there exist numerous cluster configurations. However, current preferences for parallel training focus on data centers, overlooking the financial constraints faced by most researchers. To attain the best performance within the cost limitation, we introduce a throughput-cost metric to accurately characterize clusters' cost-effectiveness. Based on this metric, we design a cost-effective cluster featuring the 3090 with NVLink. The experiment results demonstrate that our cluster achieves remarkable cost-effectiveness in various distributed model training schemes.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Communication Analysis for Multidimensional parallel Training of Large-scale DNN Models

Communication Analysis for Multidimensional Parallel Trainin...

引用

IEEE International Conference on High Performance Computing and Communications (HPCC)

作者： Zhiquan Lai Yanqi Hao Shengwei Li Dongsheng Li National Key Laboratory of Parallel and Distributed Computing College of Computer National University of Defense Technology Changsha China

Multidimensional parallel training has been widely applied to train large-scale deep learning models like GPT-3. The efficiency of parameter communication among training devices/processes is often the performance bottleneck of large model training. Analysis of parameter communication mode and traffic has important reference significance for the research of interconnection network design and computing task scheduling to improve the training performance. In this paper, we analyze the parametric communication modes in typical 3D parallel training (data parallelism, pipeline parallelism, and tensor parallelism), and model the traffic in different communication modes. Finally, taking GPT-3 as an example, we present the communication in its 3D parallel training.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Session-Recommendation Based on Gated Neural Network to Extract Item Structure Feature Graph 22

Session-Recommendation Based on Gated Neural Network to Extr...

引用

5th International Conference on Data Storage and Data Engineering, DSDE 2022

作者： Zhang, Wei Yang, Yan Zhong, Yingli College of Computer Science and Technology Heilongjiang University China Heilongjiang Key Laboratory of Database and Parallel Computing Heilongjiang University China School of Computer Science and Technology Heilongjiang University China

ISBN: (纸本)9781450395724

The issue of inaccurate user portrait recommendations for short-Term conversations is solved. The classical recommendation is based on an assumption: The user's historical behavior can represent the user's stable long-Term preferences, and the result is generated based on the user's historical behavior. However, there is sparseness in the user's historical behavior information record in the actual data set, and the obtained data cannot accurately represent the user's preferences. Research on a session recommendation focuses on user behavior data in the short term, and aims at predict anonymous user behavior based on in-session behavior information. It plays a vital role in application scenarios where user behavior information is scarce. Feature extraction is performed on the user's anonymous behavior information graph generated in a fixed period of time as the embedding representation of user's interest preference;the graph structure of the same item in the global click interaction graph is combined with the input a gated graph neural network as the item attribute embedding representation. The prediction part adopts simulated interactive calculation to produce results. With the help of gated graph neural network, features extraction and analysis of data in short-Term sessions: session-based user behavior data and in-session item attributes are obtained from the graph data structure, which reduces errors caused by inconsistent data processing methods. The user's current interest is captured based on the behavior information in the session, the global inter-item conversion graph is generated based on the sequence of click moments, and the feature vector is obtained through graph neural network processing. The generation of recommendation results is more time-sensitive, complete the anonymity problem due to the scarcity of user's information, and improves the accuracy of recommendations. © 2022 ACM.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Ladder: A Blockchain Model of Low-Overhead Storage 3rd

Ladder: A Blockchain Model of Low-Overhead Storage

引用

3rd International Conference on Blockchain and Trustworthy Systems, Blocksys 2021

作者： Chen, Peng Zhang, Xunhui Liu, Hui Xiang, Liangliang Shi, Peichang National Key Laboratory of Parallel and Distributed Processing College of Computer Science National University of Defense Technology Changsha410073 China

ISBN: (纸本)9789811679926

Emerging blockchain accounting mechanism allow mutually distributed parties to transport trusted information and ensure the correctness of data. Every blockchain node stores the complete block locally. Although this measure improves security, it causes huge storage overhead. We present Ladder, a multi-level low-overhead storage model for blockchain, which can greatly reduce the storage overhead of nodes while ensuring that the blockchain cannot be tampered easily and the block information is complete. Ladder distinguishes the retrieved frequency and importance of different blocks by value-density, dividing them into Hot-Blocks, Warm-Blocks and Cold-Blocks adopting different storage strategies respectively. Through experimental verification of time and space cost of different storage strategies, Ladder can greatly reduce the local storage overhead of nodes. © 2021, Springer Nature Singapore Pte Ltd.

关键词： Blockchain

来源：评论

学校读者我要写书评

暂无评论

DMSA: Dynamic Multi-Scale Unsupervised Semantic Segmentation Based On Adaptive Affinity 48

DMSA: Dynamic Multi-Scale Unsupervised Semantic Segmentation...

引用

48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023

作者： Yang, Kun Lu, Jun Heilongjiang University College of Computer Science and Technology Harbin150080 China Jiaxiang Industrial Technology Research Institute of Hlju Jining China Key Laboratory of Database and Parallel Computing of Heilongjiang Province Harbin China

ISBN: (纸本)9781728163277

The proposed method in this paper proposes an end-to-end unsupervised semantic segmentation architecture DMSA based on four loss functions. The framework uses Atrous Spatial Pyramid Pooling (ASPP) module to enhance feature extraction. At the same time, a dynamic dilation strategy is designed to better capture multi-scale context information. Secondly, a Pixel-Adaptive Refinement (PAR) module is introduced, which can adaptively refine the initial pseudo labels after feature fusion to obtain high quality pseudo labels. Experiments show that the proposed DSMA framework is superior to the existing methods on the saliency dataset. On the COCO 80 dataset, the MIoU is improved by 2.0, and the accuracy is improved by 5.39. On the Pascal VOC 2012 Augmented dataset, the MIoU is improved by 4.9, and the accuracy is improved by 3.4. In addition, the convergence speed of the model is also greatly improved after the introduction of the PAR module. © 2023 IEEE.

关键词： Semantic Segmentation

来源：评论

学校读者我要写书评

暂无评论

A Fast Adversarial Sample Detection Approach for Industrial Internet-of-Things Applications

A Fast Adversarial Sample Detection Approach for Industrial ...

引用

International Workshop on Quality of Service

作者： Shigeng Zhang Yudong Li Shuxin Chen Xuan Li Jian Zhang School of Computer Science and Engineering Central South University College of Computer Science and Electronic Engineering Hunan University The Science and Technology on Parallel and Distributed Processing Laboratory (PDL) China

Adversarial attacks reveal the inherent vulnerability of deep neural networks, which face serious security issues for their security. Among them, the attack against the Deep Neural Network (DNN) application used in the Industrial Internet of Things (IIoT) is a key area in adversarial attacks. Adversarial examples generated by attackers by adding human-undetectable interference to legitimate examples may cause models to make wrong decision results, resulting in serious accidents. Many detection technologies have been proposed to mitigate the harm of adversarial examples to neural networks, among which the methods based on the difference of feature attribution between normal examples and adversarial examples show state-of-the-art detection performance, but they suffer from detection efficiency. In this work, we focus on improving the detection efficiency of the feature-attribution-based detection methods. We observe that there is still a significant difference in the feature attribution distribution of a normal image and an adversarial image even only some pixels in the image are processed, which can be verified by utilizing the Kolmogorov-Smirnov test. Based on this observation, we first adopt a variety of strategies to sample partial pixels in an image and then utilize the selected pixels to train a feature-attribution-based detector for detecting adversarial examples. Extensive experiments conducted on four datasets (MNIST, CIFAR-10, SVHN, CIFAR-100) against various attacks proved that the detection efficiency of the accelerated detection method is improved (for example, the average execution time was increased by 8.7 times on CIFAR-10) while the detection performance maintains state-of-the-art.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Prophet: Fine-grained Load Balancing for parallel Training of Large-scale MoE Models

Prophet: Fine-grained Load Balancing for Parallel Training o...

引用

IEEE International Conference on Cluster Computing

作者： Wei Wang Zhiquan Lai Shengwei Li Weijie Liu Keshi Ge Yujie Liu Ao Shen Dongsheng Li National Laboratory for Parallel and Distributed Processing(PDL) College Of Computer National University Of Defense Technology Changsha China

Mixture of Expert (MoE) has received increasing attention for scaling DNN models to extra-large size with negligible increases in computation. The MoE model has achieved the highest accuracy in several domains. However, a significant load imbalance occurs in the device during the training of a MoE model, resulting in significantly reduced throughput. Previous works on load balancing either harm model convergence or suffer from high execution overhead. To address these issues, we present Prophet: a fine-grained load balancing method for parallel training of large-scale MoE models, which consists of a planner and a scheduler. Prophet planner first employs a fine-grained resource allocation method to determine the possible scenarios for the expert placement in a fine-grained manner, and then efficiently searches for a well-balanced expert placement to balance the load without introducing additional overhead. Prophet scheduler exploits the locality of the token distribution to schedule the resource allocation operations using a layer-wise fine-grained schedule strategy to hide their overhead. We conduct extensive experiments in four clusters and five representative models. The results indicate that Prophet gains up to 2.3x speedup compared to the state-of-the-art MoE frameworks including Deepspeed-MoE and FasterMoE. Additionally, Prophet achieves a load balancing enhancement of up to 12.06x when compared to FasterMoE.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：