检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Shen, Ao Wang, Qiang Lai, Zhiquan Li, Xionglve Li, Dongsheng National Key Laboratory of Parallel and Distributed Computing National University of Defense Technology Hunan Changsha410073 China College of computer National University of Defense Technology Hunan Changsha410073 China

Large Language Models (LLMs) have demonstrated impressive performance across various domains. However, the enormous number of model parameters makes fine-tuning challenging, significantly limiting their application and deployment. Existing solutions combine parameter quantization with Low-Rank Adaptation (LoRA), greatly reducing memory usage but resulting in noticeable performance degradation. In this paper, we identify an imbalance in fine-tuning quantized pre-trained models: overly complex adapter inputs and outputs versus low effective trainability of the adaptation. We propose Quantized LLMs with Balanced-rank Adaptation (Q-BaRA), which simplifies the adapter inputs and outputs while increasing the adapter’s rank to achieve a more suitable balance for finetuning quantized LLMs. Additionally, for scenarios where fine-tuned LLMs need to be deployed as low-precision inference models, we introduce Quantization-Aware Fine-tuning with Higher Rank Adaptation (QA-HiRA), which simplifies the adapter inputs and outputs to align with the pre-trained model’s block-wise quantization while employing a single matrix to achieve a higher rank. Both Q-BaRA and QA-HiRA are easily implemented and offer the following optimizations: (i) Q-BaRA consistently achieves the highest accuracy compared to baselines and other variants, requiring the same number of trainable parameters and computational effort;(ii) QA-HiRA naturally merges adapter parameters into the block-wise quantized model after fine-tuning, achieving the highest accuracy compared to other methods. We apply our Q-BaRA and QA-HiRA to the LLaMA and LLaMA2 model families and validate their effectiveness across different fine-tuning datasets and downstream scenarios. Copyright © 2024, The Authors. All rights reserved.

关键词： Digital elevation model

来源：评论

学校读者我要写书评

暂无评论

Multi-task Pre-training Language Model for Semantic Network Completion

arXiv

引用

arXiv 2022年

作者： Li, Da Yang, Sen Xu, Kele Yi, Ming He, Yukai Wang, Huaimin Tencent Beijing100084 China Beijing Institute of Microbiology and Epidemiology Beijing100071 China State Key Laboratory of Pathogen and Biosecurity Beijing100071 China National University of Defense Technology Changsha410073 China National Key Lab of Parallel and Distributed Processing Changsha410073 China

Semantic networks, such as the knowledge graph, can represent the knowledge leveraging the graph structure. Although the knowledge graph shows promising values in natural language processing, it suffers from incompleteness. This paper focuses on knowledge graph completion by predicting linkage between entities, which is a fundamental yet critical task. Semantic matching is a potential solution for link prediction as it can deal with unseen entities, while the translational distance based methods struggle with the unseen entities. However, to achieve competitive performance as translational distance based methods, semantic matching based methods require large-scale datasets for the training purpose, which are typically unavailable in practical settings. Therefore, we employ the language model and introduce a novel knowledge graph architecture named LP-BERT, which contains two main stages: multi-task pre-training and knowledge graph fine-tuning. In the pre-training phase, three tasks are taken to drive the model to learn the relationship information from triples by predicting either entities or relations. While in the fine-tuning phase, inspired by contrastive learning, we design a triple-style negative sampling in a batch, which greatly increases the proportion of negative sampling while keeping the training time almost unchanged. Furthermore, we propose a new data augmentation method utilizing the inverse relationship of triples to improve the performance and robustness of the model. To demonstrate the effectiveness of our proposed framework, we conduct extensive experiments on three widely-used knowledge graph datasets, WN18RR, FB15k-237, and UMLS. The experimental results demonstrate the superiority of our methods, and our approach achieves state-of-the-art results on the WN18RR and FB15k-237 datasets. Significantly, the Hits@10 indicator is improved by 5% from the previous state-of-the-art result on the WN18RR dataset while reaching 100% on the UMLS dataset. Copy

关键词： Inverse problems

来源：评论

学校读者我要写书评

暂无评论

Density-Attentive Head Detector for Crowd Counting

Density-Attentive Head Detector for Crowd Counting

引用

International Conference on Data Science and Business Analytics (ICDSBA)

作者： Ziyu Zhao Xingyu Shen Long Lan Xiaoyao Yin Xiang Zhang Yonggang Che Zhigang Luo College of Computer National University of Defense Technology Changsha China Institute for Quantum Information & State Key Laboratory of High Performance Computing National University of Defense Technology Changsha China Science and Technology on Parallel and Distributed Processing National University of Defense Technology Changsha China

ISBN: (数字)9781728146447

ISBN: (纸本)9781728146454

In crowd counting, regression-based method shows better performance in extreme density scenes by introduces a density map. However, the regression-based method fails to locate the positions of each head, which significantly restricts its applications. Detection-based method counts each head with their accurate locations but works only in middle or low-level density scenes. In this paper, a joint learning method, named Density-attentive Head Detector (DAHD) is developed to overcome their respective shortcomings via a multi-task training procedure. Specifically, to guarantee the sensitivity of the detector to small or partially occluded heads, we carefully equip the detector with a density map which learned from regression module. Moreover, a novel Dilated Feature Pyramid Network (DFPN) is introduced to our method to enlarge the receptive field of convolutional kernel, bringing confirmative additional benefits to identify small heads. Experiments on the popular ShanghaiTech and Mall datasets confirm the improved performance of DAHD compared with the current detection-based approaches, and a comparable performance to regression-based approaches in term of counting.

关键词： Detectors Feature extraction Object detection Faces Kernel Training Task analysis

来源：评论

学校读者我要写书评

暂无评论

KC-GenRe: A Knowledge-constrained Generative Re-ranking Method Based on Large Language Models for Knowledge Graph Completion

arXiv

引用

arXiv 2024年

作者： Wang, Yilin Hu, Minghao Huang, Zhen Li, Dongsheng Yang, Dong Lu, Xicheng Defense Innovation Institute Academy of Military Sciences China Information Research Center of Military Science China National Key Laboratory of Parallel and Distributed Computing China

The goal of knowledge graph completion (KGC) is to predict missing facts among entities. Previous methods for KGC re-ranking are mostly built on non-generative language models to obtain the probability of each candidate. Recently, generative large language models (LLMs) have shown outstanding performance on several tasks such as information extraction and dialog systems. Leveraging them for KGC re-ranking is beneficial for leveraging the extensive pre-trained knowledge and powerful generative capabilities. However, it may encounter new problems when accomplishing the task, namely mismatch, misordering and omission. To this end, we introduce KC-GenRe, a knowledge-constrained generative re-ranking method based on LLMs for KGC. To overcome the mismatch issue, we formulate the KGC re-ranking task as a candidate identifier sorting generation problem implemented by generative LLMs. To tackle the misordering issue, we develop a knowledge-guided interactive training method that enhances the identification and ranking of candidates. To address the omission issue, we design a knowledge-augmented constrained inference method that enables contextual prompting and controlled generation, so as to obtain valid rankings. Experimental results show that KG-GenRe achieves state-of-the-art performance on four datasets, with gains of up to 6.7% and 7.7% in the MRR and Hits@1 metric compared to previous methods, and 9.0% and 11.1% compared to that without re-ranking. Extensive analysis demonstrates the effectiveness of components in KG-GenRe. Copyright © 2024, The Authors. All rights reserved.

关键词： Knowledge graph

来源：评论

学校读者我要写书评

暂无评论

A GPU-based Fast Solution for Riesz Space Fractional Reaction-Diffusion Equation

A GPU-based Fast Solution for Riesz Space Fractional Reactio...

引用

International Conference on Network-Based Information Systems (NBIS)

作者： Qinglin Wang Jie Liu Chunye Gong Yang Zhang Zuocheng Xing Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense Technology Changsha China College of Aerospace Science and Engineering National University of Defense Technology Changsha China Science and Technology on Space Physics Laboratory Beijing China

The fast numerical solutions of Riesz fractional equation have computational cost of O(NMlogM), where M, N are the number of grid points and time steps. In this paper, we present a GPU-based fast solution for Riesz space fractional equation. The GPU-based fast solution, which is based on the fast method using FFT and implemented with CUDA programming model, consists of parallel FFT, vector-vector addition and vector-vector multiplication on GPU. The experimental results show that the GPU-based fast solution compares well with the exact solution. Compared to the known parallel fast solution on 8-core Intel E5-2670 CPU, the overall performance speedup on NVIDIA GTX650 GPU reaches 2.12 times and that on NVIDIA K20C GPU achieves 10.93 times.

关键词： Graphics processing units Instruction sets Mathematical model Arrays Yttrium Approximation methods parallel processing

来源：评论

学校读者我要写书评

暂无评论

A distributed Relation Detection Approach in the Internet of Things

引用

Mobile Information Systems 2017年第1期2017卷

作者： Zhu, Weiping Lu, Hongliang Cui, Xiaohui Cao, Jiannong International School of Software Wuhan University Wuhan China Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense Technology Changsha China Department of Computing Hong Kong Polytechnic University Kowloon Hong Kong

In the Internet of Things, it is important to detect the various relations among objects for mining useful knowledge. Existing works on relation detection are based on centralized processing, which is not suitable for the Internet of Things owing to the unavailability of a server, one-point failure, computation bottleneck, and moving of objects. In this paper, we propose a distributed approach to detect relations among objects. We first build a system model for this problem that supports generic forms of relations and both physical time and logical time. Based on this, we design the distributed Relation Detection Approach (DRDA), which utilizes a distributed spanning tree to detect relations using in-network processing. DRDA can coordinate the distributed tree-building process of objects and automatically change the depth of the routing tree to a proper value. Optimization among multiple relation detection tasks is also considered. Extensive simulations were performed and the results show that the proposed approach outperforms existing approaches in terms of the energy consumption. © 2017 Weiping Zhu et al.

关键词： Internet of things

来源：评论

学校读者我要写书评

暂无评论

Soft-Constrained Nonnegative Matrix Factorization via Normalization

Soft-Constrained Nonnegative Matrix Factorization via Normal...

引用

International Joint Conference on Neural Networks

作者： Long Lan Naiyang Guan Xiang Zhang Dacheng Tao Zhigang Luo Science and Technology on Parallel and Distributed Processing Laboratory School of Computer Science National University of Defense Technology Centre for Quantum Computation & Intelligent Systems and the Faculty of Engineering and Information Technology University of Technology

ISBN: (纸本)9781479914821

Semi-supervised clustering aims at boosting the clustering performance on unlabeled samples by using labels from a few labeled samples. Constrained NMF (CNMF) is one of the most significant semi-supervised clustering methods, and it factorizes the whole dataset by NMF and constrains those labeled samples from the same class to have identical encodings. In this paper, we propose a novel soft-constrained NMF (SCNMF) method by softening the hard constraint in CNMF. Particularly, SCNMF factorizes the whole dataset into two lower-dimensional factor matrices by using multiplicative update rule (MUR). To utilize the labels of labeled samples, SCNMF iteratively normalizes both factor matrices after updating them with MURs to make encodings of labeled samples close to their label vectors. It is therefore reasonable to believe that encodings of unlabeled samples are also close to their corresponding label vectors. Such strategy significantly boosts the clustering performance even when the labeled samples are rather limited, e.g., each class owns only a single labeled sample. Since the normalization procedure never increases the computational complexity of MUR, SCNMF is quite efficient and effective in practices. Experimental results on face image datasets illustrate both efficiency and effectiveness of SCNMF compared with both NMF and CNMF.

关键词： Matrix decomposition Encoding Vectors Clustering methods Clustering algorithms Time complexity Face

来源：评论

学校读者我要写书评

暂无评论

Box-constrained Projective Nonnegative Matrix Factorization via Augmented Lagrangian Method

Box-constrained Projective Nonnegative Matrix Factorization ...

引用

International Joint Conference on Neural Networks

作者： Xiang Zhang Naiyang Guan Long Lan Dacheng Tao Zhigang Luo Science and Technology on Parallel and Distributed Processing Laboratory School of Computer Science National University of Defense Technology Centre for Quantum Computation & Intelligent Systems and the Faculty of Engineering and Information Technology University of Technology

ISBN: (纸本)9781479914821

Projective non-negative matrix factorization (PNMF) projects a set of examples onto a subspace spanned by a non-negative basis whose transpose is regarded as the projection matrix. Since PNMF learns a natural parts-based representation, it has been successfully used in text mining and pattern recognition. However, it is non-trivial to analyze the convergence of the optimization algorithms for PNMF because its objective function is non-convex. In this paper, we propose a Box-constrained PNMF (BPNMF) method to overcome this deficiency of PNMF. In particular, BPNMF introduces an auxiliary variable, i.e., the coefficients of examples, and incorporates the following two types of constraints: 1) each entry of the basis is non-negative and upper-bounded, i.e., box-constrained, and 2) the coefficients equal to the projected points of the examples. The first box constraint makes the basis to be bound and the second equality constraint keeps its equivalence to PNMF. Similar to PNMF, BPNMF is difficult because the objective function is non-convex. To solve BPNMF, we developed an efficient algorithm in the frame of augmented Lagrangian multiplier (ALM) method and proved that the ALM-based algorithm converges to local minima. Experimental results on two face image datasets demonstrate the effectiveness of BPNMF compared with the representative methods.

关键词： Accuracy Linear programming Training Convergence Optimization Face Educational institutions

来源：评论

学校读者我要写书评

暂无评论

Efficient Flooding for Wireless Ad Hoc Networks with Directional Antennas

Efficient Flooding for Wireless Ad Hoc Networks with Directi...

引用

2007 International Symposium on Communications and Information Technologies (ISCIT 2007), vol.2

作者： Xianlong Jiao Xiaodong Wang Xingming Zhou National Laboratory for Parallel and Distributed Processing National University of Defense Technology Changsha China

In wireless ad hoc networks, the flooding operation is an important utility function. To solve the broadcast storm problem, some efficient flooding schemes have been proposed. However, there also exist many problems in these schemes, such as signal collision. In this paper, we propose an efficient flooding algorithm called EFDA based on the algorithm proposed in [1] (we reference the algorithm as EF1). Through theoretical analysis, our algorithm doesn't affect the local optimality gained by EF1. We have implemented our algorithm in ns-2 simulator, and the results show that exploiting directional antenna can reduce the number of signal collisions and our algorithm achieves better performance than pure flooding and EF1 due to smaller number of signal collisions.

关键词： Mobile ad hoc networks Directional antennas Information technology

来源：评论

学校读者我要写书评

暂无评论

A Connectivity-Enhanced Multi-Task Learning based on Anatomical Priors for 3D Class-Balanced Pulmonary Airway Segmentation

A Connectivity-Enhanced Multi-Task Learning based on Anatomi...

引用

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Yan Jia Yong Dou Peng Qiao Yuqing Cheng Kele Xu Zhouyu He National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology Changsha China College of Systems Engineering National University of Defense Technology Changsha China

ISBN: (数字)9798350386226

ISBN: (纸本)9798350386233

Accurate and efficient airway segmentation is essential for evaluating pulmonary diseases, aiding diagnosis, reducing the preoperative burden of airway identification, and minimizing patient discomfort during prolonged surgeries. However, current pulmonary airway reconstruction techniques are hindered by two major challenges: difficulty in accurately reconstructing fine airway branches due to the tendency to overlook small targets, and insufficient structural connectivity leading to frequent branch discontinuities within the airway tree. These limitations directly affect the clinical applicability of reconstructed airways. To overcome these challenges, a novel 3D pulmonary airway segmentation multi-task framework is proposed, designed to enhance the performance of existing backbone models. This approach integrates Anatomical Prior-Based Multi-Task Learning (AP-MTL) through the use of Gaussian-constructed connectivity-enhanced isosurfaces, significantly improving the network’s ability to maintain airway continuity. Additionally, a Class-Balanced CT Density Distribution Reconstruction mechanism (DDR-CB) is introduced, further refining the model’s capability to detect and segment fine airway branches. As a result of these enhancements, the model demonstrates a 11.5% average improvement in segmentation accuracy and connectivity compared to the baseline. The source code is publicly accessible at https://***/inexhaustible419/APMTLAirwaySegment.

关键词： Image segmentation Three-dimensional displays Accuracy Lungs Atmospheric modeling Supervised learning Surgery Multitasking Image reconstruction Biomedical imaging

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：