检索结果-内蒙古大学图书馆

arXiv 2023年

作者： Guan, Lei Li, Dongsheng Shi, Yanqi Meng, Jian The Department of Mathematics National University of Defense Technology Changsha China The National Key Laboratory of Parallel and Distributed Computing National University of Defense Technology China

In this paper, we propose a general deep learning training framework XGrad which introduces weight prediction into the popular gradient-based optimizers to boost their convergence and generalization when training the deep neural network (DNN) models. In particular, ahead of each mini-batch training, the future weights are predicted according to the update rule of the used optimizer and are then applied to both the forward pass and backward propagation. In this way, during the whole training period, the optimizer always utilizes the gradients w.r.t. the future weights to update the DNN parameters, making the gradient-based optimizer achieve better convergence and generalization compared to the original optimizer without weight prediction. XGrad is rather straightforward to implement yet pretty effective in boosting the convergence of gradient-based optimizers and the accuracy of DNN models. Empirical results concerning five popular optimizers including SGD with momentum, Adam, AdamW, AdaBelief, and AdaM3 demonstrate the effectiveness of our proposal. The experimental results validate that XGrad can attain higher model accuracy than the baseline optimizers when training the DNN models. The code of XGrad will be available at: https://***/guanleics/XGrad. Copyright © 2023, The Authors. All rights reserved.

关键词： Forecasting

来源：评论

学校读者我要写书评

暂无评论

PTQ-SO: A Scale Optimization-based Approach for Post-training Quantization of Edge computing

PTQ-SO: A Scale Optimization-based Approach for Post-trainin...

引用

International Conference on Computer Supported Cooperative Work in Design

作者： Kangkang Liu Ningjiang Chen College of Computer and Electronic Information Guangxi University Nanning China Education Department of Guangxi Zhuang Autonomous Region Key Laboratory of Parallel Distributed and Intelligent Computing Guangxi University Nanning China

ISBN: (数字)9798350349184

ISBN: (纸本)9798350349191

With the increasing performance of deep convolutional neural networks, they have been widely used in many computer vision tasks. However, a huge convolutional neural network model requires a lot of memory and computing resources, which makes it difficult to meet the requirements of low latency and reliability of edge computing when the model is deployed locally on resource-limited devices in edge environments. Quantization is a kind of model compression technology, which can effectively reduce model size, calculation cost and inference delay, but the quantization noise will cause the accuracy of the quantization model to decrease. Aiming at the problem of precision loss caused by model quantization, this paper proposes a post-training quantization method based on scale optimization. By reducing the influence of redundant parameters in the model on the quantization parameters in the process of model quantization, the scale factor optimization is realized to reduce the quantization error and thus improve the accuracy of the quantized model, reduce the inference delay and improve the reliability of edge applications. The experimental results show that under different quantization strategies and different quantization bit widths, the proposed method can improve the accuracy of the quantized model, and the absolute accuracy of the optimal quantization model is improved by 1.36%. The improvement effect is obvious, which is conducive to the application of deep neural network in edge environment.

关键词： Quantization (signal) Accuracy Computational modeling Computer network reliability Delays Reliability Convolutional neural networks

来源：评论

学校读者我要写书评

暂无评论

A Counterfactual Ultrasound Anti-Interference Self-Supervised Network for B-mode Ultrasound Tongue Extraction

A Counterfactual Ultrasound Anti-Interference Self-Supervise...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Yan Jia Yuqing Cheng Kele Xu Yong Dou Peng Qiao Zhouyu He National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology Changsha China College of Systems Engineering National University of Defense Technology Changsha China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

B-mode ultrasound tongue imaging is a non-invasive and real-time method for visualizing vocal tract deformation. However, accurately extracting the tongue’s surface contour remains a significant challenge due to the low signal-to-noise ratio (SNR) and prevalent speckle noise in ultrasound images. Traditional supervised learning models often require large labeled datasets, which are labor-intensive to produce and susceptible to noise interference. To address these limitations, we present a novel Counterfactual Ultrasound Anti-Interference Self-Supervised Network (CUAI-SSN), which integrates self-supervised learning (SSL) with counterfactual data augmentation, progressively disentangles confounding factors, ensuring that the model generalizes well across varied ultrasound conditions. Our approach leverages causal reasoning to decouple noise from relevant features, enabling the model to learn robust representations that focus on essential tongue structures. By generating counterfactual image-label pairs, our method introduces alternative, noise-independent scenarios that enhance model training. Furthermore, we introduce attention mechanisms to enhance the network’s ability to capture fine-grained details even in noisy conditions. Extensive experiments on real ultrasound tongue images demonstrate that CUAI-SSN outperforms existing methods, setting a new benchmark for automated contour extraction in ultrasound tongue imaging. Our code is publicly available at https://***/inexhaustible419/CounterfactualultrasoundAI.

关键词： Training Ultrasonic imaging Tongue Self-supervised learning Data augmentation Data models Cognition Data mining Noise measurement Signal to noise ratio

来源：评论

学校读者我要写书评

暂无评论

Cooperative Air-Ground Instant Delivery by UAVs and Crowdsourced Taxis 40

Cooperative Air-Ground Instant Delivery by UAVs and Crowdsou...

引用

40th IEEE International Conference on Data Engineering, ICDE 2024

作者： Gao, Junhui Wang, Qianru Zhang, Xin Shi, Juan Zhao, Xiang Han, Qingye Pan, Yan School of Computer Science Northwestern Polytechnical University China School of Computer Science and Technology Xidian University China Air Force Engineering University China National University of Defense Technology Laboratory for Big Data and Decision China School of Management Science and Real Estate Chongqing University China National University of Defense Technology National Key Laboratory of Information Systems Engineering China National University of Defense Technology National Key Laboratory of Parallel and Distributed Computing China

ISBN: (纸本)9798350317152

Instant delivery has become a fundamental service in people's daily lives. Different from the traditional express service, the instant delivery has a strict shipping time constraint after being ordered. However, the labor shortage makes it challenging to realize efficient instant delivery. To tackle the problem, researchers have studied to introduce vehicles (i.e., taxis) or Unmanned Aerial Vehicles (UAVs or drones) into instant delivery tasks. Unfortunately, the delivery detour of taxis and the limited battery of UAVs make it hard to meet the rapidly increasing instant delivery demands. Under this circumstance, this paper proposes an air-ground cooperative instant delivery paradigm to maximize the delivery performance and meanwhile minimize the negative effects on the taxi passengers. Specifically, a data-driven delivery potential-demands-aware cooperative strategy is designed to improve the overall delivery performance of both UAVs and taxis as well as the taxi passengers' experience. The experimental results show that the proposed method improves the delivery number by 30.1% and 114.5% compared to the taxi-based and UAV-based instant delivery respectively, and shortens the delivery time by 35.7% compared to the taxi-based instant delivery. © 2024 IEEE.

关键词： Unmanned aerial vehicles (UAV)

来源：评论

学校读者我要写书评

暂无评论

ACCURATE AND EFFICIENT FINE-TUNING OF QUANTIZED LARGE LANGUAGE MODELS THROUGH OPTIMAL BALANCE

arXiv

引用

arXiv 2024年

作者： Shen, Ao Wang, Qiang Lai, Zhiquan Li, Xionglve Li, Dongsheng National Key Laboratory of Parallel and Distributed Computing National University of Defense Technology Hunan Changsha410073 China College of computer National University of Defense Technology Hunan Changsha410073 China

Large Language Models (LLMs) have demonstrated impressive performance across various domains. However, the enormous number of model parameters makes fine-tuning challenging, significantly limiting their application and deployment. Existing solutions combine parameter quantization with Low-Rank Adaptation (LoRA), greatly reducing memory usage but resulting in noticeable performance degradation. In this paper, we identify an imbalance in fine-tuning quantized pre-trained models: overly complex adapter inputs and outputs versus low effective trainability of the adaptation. We propose Quantized LLMs with Balanced-rank Adaptation (Q-BaRA), which simplifies the adapter inputs and outputs while increasing the adapter’s rank to achieve a more suitable balance for finetuning quantized LLMs. Additionally, for scenarios where fine-tuned LLMs need to be deployed as low-precision inference models, we introduce Quantization-Aware Fine-tuning with Higher Rank Adaptation (QA-HiRA), which simplifies the adapter inputs and outputs to align with the pre-trained model’s block-wise quantization while employing a single matrix to achieve a higher rank. Both Q-BaRA and QA-HiRA are easily implemented and offer the following optimizations: (i) Q-BaRA consistently achieves the highest accuracy compared to baselines and other variants, requiring the same number of trainable parameters and computational effort;(ii) QA-HiRA naturally merges adapter parameters into the block-wise quantized model after fine-tuning, achieving the highest accuracy compared to other methods. We apply our Q-BaRA and QA-HiRA to the LLaMA and LLaMA2 model families and validate their effectiveness across different fine-tuning datasets and downstream scenarios. Copyright © 2024, The Authors. All rights reserved.

关键词： Digital elevation model

来源：评论

学校读者我要写书评

暂无评论

Simultaneously Learning Syntactic Dependency and Semantics Reasonability for Relation Extraction

Simultaneously Learning Syntactic Dependency and Semantics R...

引用

International Conference on Image, Vision and Intelligent Systems, ICIVIS 2021

作者： Wang, Xin Yin, Nan Zhang, Xiang Bai, Xinyi Luo, Zhigang College of Computer National University of Defense Technology Changsha China Institute for Quantum and State Key Laboratory of High Performance Computing National University of Defense Technology Changsha China Science and Technology on Parallel and Distributed Laboratory National University of Defense Technology Changsha China

ISBN: (纸本)9789811669620

Relation extraction as an important Natural Language Processing (NLP) task is to identify relations between named entities in text. Recently, graph convolutional networks over dependency trees have been widely used to capture syntactic features and achieved attractive performance. However, most existing dependency-based approaches ignore the positive influence of the words outside the dependency trees, sometimes conveying rich and useful information on relation extraction. In this paper, we propose a novel model, Entity-aware Self-attention Contextualized GCN (ESC-GCN), which efficiently incorporates syntactic structure of input sentences and semantic context of sequences. To be specific, relative position self-attention obtains the overall semantic pairwise correlation related to word position, and contextualized graph convolutional networks capture rich intra-sentence dependencies between words by adequately pruning operations. In this way, our proposed model not only reduces the noisy impact from dependency trees but also obtains easily-ignored entity-related semantic representation. Extensive experiments demonstrate that our model achieves encouraging performance. © 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Graph Structure Learning via Transfer Entropy for Multivariate Time Series Anomaly Detection

Graph Structure Learning via Transfer Entropy for Multivaria...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Mingyu Liu Yijie Wang Xiaohui Zhou Yongjun Wang National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology Changsha China College of Computer Science and Technology National University of Defense Technology Changsha China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Multivariate time series anomaly detection (MTAD) poses a challenge due to temporal and feature dependencies. The critical aspects of enhancing the detection performance lie in accurately capturing the dependencies between variables within the sliding window and effectively leveraging them. Existing studies rely on domain knowledge to pre-set the window size, and overlook the strength of dependencies while calculating direction based on variable similarity. This paper proposes GSLTE, a graph structure learning method for MTAD. GSLTE employs Fast Fourier Transform to conduct iterative segmentation of the whole series, selecting the dominant Fourier frequency as the window size for each subsequence within the minimum interval. GSLTE quantifies the direction and strength of the dependencies based on variable-lag transfer entropy which is achieved through Dynamic Time Warping method to learn asymmetric links between variables. Extensive experiments show that GNN-based MTAD methods applying GSLTE can further improve anomaly detection performance while outperforming state-of-the-art competitors.

关键词： Learning systems Time-frequency analysis Fast Fourier transforms Time series analysis Signal processing Feature extraction Entropy Iterative methods Speech processing Anomaly detection

来源：评论

学校读者我要写书评

暂无评论

parallel Accelerating Ultra-Long Read Alignment by Vertical Partitioning Data

Parallel Accelerating Ultra-Long Read Alignment by Vertical ...

引用

International Symposium on parallel Architectures, Algorithms and Programming (PAAP)

作者： Deng Pan Cheng Zhong Danyang Chen Jinxiong Zhang Feng Yang School of Computer Electronics and Information Key Laboratory of Parallel Distributed Computing Technology in Guangxi Universities Guangxi University Nanning Guangxi China School of Computer Electronics and Information Key Laboratory of Parallel Distributed Computing Guangxi University Nanning Guangxi China

ISBN: (纸本)9781665452199

The alignment between sequencing reads and genome is a basic work in biological big data analysis. Each read of the third generation sequencing data is getting longer, and the data size is getting larger. To effectively solve the ultra-long read alignment problem with high requirements for computing and memory capacity, a strategy for vertical partitioning ultra-long reads on hybrid CPU/GPU cluster is proposed, and a heap data structure is used to filter the local aligned results in all computing nodes of the parallel cluster system according to the alignment score to reduce the data transmission size. The methods for early termination and parallel merging-splicing are used to accelerate splicing local aligned results. The local aligned results among all computing nodes are collected and extended to obtain the final alignment results. The experimental results on datasets of simulated and real ultra-long reads show that the proposed parallel alignment algorithm can obtain high alignment accuracy, sensitivity and base-level sensitivity as a whole, and accelerate completing alignment between ultra-long reads and reference genome.

关键词： Sequential analysis Sensitivity Splicing Merging Memory management Genomics Programming

来源：评论

学校读者我要写书评

暂无评论

A Connectivity-Enhanced Multi-Task Learning based on Anatomical Priors for 3D Class-Balanced Pulmonary Airway Segmentation

A Connectivity-Enhanced Multi-Task Learning based on Anatomi...

引用

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Yan Jia Yong Dou Peng Qiao Yuqing Cheng Kele Xu Zhouyu He National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology Changsha China College of Systems Engineering National University of Defense Technology Changsha China

ISBN: (数字)9798350386226

ISBN: (纸本)9798350386233

Accurate and efficient airway segmentation is essential for evaluating pulmonary diseases, aiding diagnosis, reducing the preoperative burden of airway identification, and minimizing patient discomfort during prolonged surgeries. However, current pulmonary airway reconstruction techniques are hindered by two major challenges: difficulty in accurately reconstructing fine airway branches due to the tendency to overlook small targets, and insufficient structural connectivity leading to frequent branch discontinuities within the airway tree. These limitations directly affect the clinical applicability of reconstructed airways. To overcome these challenges, a novel 3D pulmonary airway segmentation multi-task framework is proposed, designed to enhance the performance of existing backbone models. This approach integrates Anatomical Prior-Based Multi-Task Learning (AP-MTL) through the use of Gaussian-constructed connectivity-enhanced isosurfaces, significantly improving the network’s ability to maintain airway continuity. Additionally, a Class-Balanced CT Density Distribution Reconstruction mechanism (DDR-CB) is introduced, further refining the model’s capability to detect and segment fine airway branches. As a result of these enhancements, the model demonstrates a 11.5% average improvement in segmentation accuracy and connectivity compared to the baseline. The source code is publicly accessible at https://***/inexhaustible419/APMTLAirwaySegment.

关键词： Image segmentation Three-dimensional displays Accuracy Lungs Atmospheric modeling Supervised learning Surgery Multitasking Image reconstruction Biomedical imaging

来源：评论

学校读者我要写书评

暂无评论

Fascinating Supervisory Signals and Where to Find Them: Deep Anomaly Detection with Scale Learning

arXiv

引用

arXiv 2023年

作者： Xu, Hongzuo Wang, Yijie Wei, Juhui Jian, Songlei Li, Yizhou Liu, Ning National Key Laboratory of Parallel and Distributed Computing China College of Computer National University of Defense Technology China College of Science National University of Defense Technology China

Due to the unsupervised nature of anomaly detection, the key to fueling deep models is finding supervisory signals. Different from current reconstruction-guided generative models and transformation-based contrastive models, we devise novel data-driven supervision for tabular data by introducing a characteristic – scale – as data labels. By representing varied sub-vectors of data instances, we define scale as the relationship between the dimensionality of original sub-vectors and that of representations. Scales serve as labels attached to transformed representations, thus offering ample labeled data for neural network training. This paper further proposes a scale learning-based anomaly detection method. Supervised by the learning objective of scale distribution alignment, our approach learns the ranking of representations converted from varied subspaces of each data instance. Through this proxy task, our approach models inherent regularities and patterns within data, which well describes data "normality". Abnormal degrees of testing instances are obtained by measuring whether they fit these learned patterns. Extensive experiments show that our approach leads to significant improvement over state-of-the-art generative/contrastive anomaly detection methods. © 2023, CC BY.

关键词： Anomaly detection

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：