All-reduce is a widely used communication technique for distributed and parallel applications typically implemented using either a tree-based or ring-based scheme. Each of these approaches has its own limitations: tre...
详细信息
All-reduce is a widely used communication technique for distributed and parallel applications typically implemented using either a tree-based or ring-based scheme. Each of these approaches has its own limitations: tree-based schemes struggle with efficiently exchanging large messages, while ring-based solutions assume constant communication throughput,an unrealistic expectation in modern network communication infrastructures. We present FMCC-RT, an all-reduce approach that combines the advantages of tree-and ring-based implementations while mitigating their drawbacks. FMCC-RT dynamically switches between tree and ring-based implementations depending on the size of the message being processed. It utilizes an analytical model to assess the impact of message sizes on the achieved throughput, enabling the derivation of optimal work partitioning parameters. Furthermore, FMCC-RT is designed with an Open MPI-compatible API, requiring no modification to user code. We evaluated FMCC-RT through micro-benchmarks and real-world application tests. Experimental results show that FMCC-RT outperforms state-of-the-art tree-and ring-based methods, achieving speedups of up to 5.6×.
Knowledge distillation(KD) enhances student network generalization by transferring dark knowledge from a complex teacher network. To optimize computational expenditure and memory utilization, self-knowledge distillati...
详细信息
Knowledge distillation(KD) enhances student network generalization by transferring dark knowledge from a complex teacher network. To optimize computational expenditure and memory utilization, self-knowledge distillation(SKD) extracts dark knowledge from the model itself rather than an external teacher network. However, previous SKD methods performed distillation indiscriminately on full datasets, overlooking the analysis of representative samples. In this work, we present a novel two-stage approach to providing targeted knowledge on specific samples, named two-stage approach self-knowledge distillation(TOAST). We first soften the hard targets using class medoids generated based on logit vectors per class. Then, we iteratively distill the under-trained data with past predictions of half the batch size. The two-stage knowledge is linearly combined, efficiently enhancing model performance. Extensive experiments conducted on five backbone architectures show our method is model-agnostic and achieves the best generalization ***, TOAST is strongly compatible with existing augmentation-based regularization methods. Our method also obtains a speedup of up to 2.95x compared with a recent state-of-the-art method.
Rapid and accurate identification of high-quality patents can accelerate the transformation process of scientific and technological achievements, optimize the management of intellectual property rights and enhance the...
详细信息
Rapid and accurate identification of high-quality patents can accelerate the transformation process of scientific and technological achievements, optimize the management of intellectual property rights and enhance the vitality of innovation. Aiming at the shortcomings of the traditional high-value patent assessment method, which is relatively simple and seldom considers the influence of patentees, this paper proposes a high-quality patent method HMFM (High-Value Patent Multi-Feature Fusion Method) that fuses multi-dimensional features. A weighted node importance assessment method in complex network called GLE (Glob-Local-struEntropy) based on improved structural entropy is designed to calculate the influence of the patentee to form the patentee’s features, and the patent text features are extracted by BERT-DPCNN deep learning model, which is supplemented to the basic patent indicator system. Finally a machine learning algorithm is used to assess the value of patents. Experiment results show that our method can identify high-value patents more effectively and accurately.
Accurate traffic flow prediction has a profound impact on modern traffic management. Traffic flow has complex spatial-temporal correlations and periodicity, which poses difficulties for precise prediction. To address ...
详细信息
Accurate traffic flow prediction has a profound impact on modern traffic management. Traffic flow has complex spatial-temporal correlations and periodicity, which poses difficulties for precise prediction. To address this problem, a Multi-head Self-attention and Spatial-Temporal Graph Convolutional Network (MSSTGCN) for multiscale traffic flow prediction is proposed. Firstly, to capture the hidden traffic periodicity of traffic flow, traffic flow is divided into three kinds of periods, including hourly, daily, and weekly data. Secondly, a graph attention residual layer is constructed to learn the global spatial features across regions. Local spatial-temporal dependence is captured by using a T-GCN module. Thirdly, a transformer layer is introduced to learn the long-term dependence in time. A position embedding mechanism is introduced to label position information for all traffic sequences. Thus, this multi-head self-attention mechanism can recognize the sequence order and allocate weights for different time nodes. Experimental results on four real-world datasets show that the MSSTGCN performs better than the baseline methods and can be successfully adapted to traffic prediction tasks.
Embodied visual exploration is critical for building intelligent visual agents. This paper presents the neural exploration with feature-based visual odometry and tracking-failure-reduction policy(Ne OR), a framework f...
详细信息
Embodied visual exploration is critical for building intelligent visual agents. This paper presents the neural exploration with feature-based visual odometry and tracking-failure-reduction policy(Ne OR), a framework for embodied visual exploration that possesses the efficient exploration capabilities of deep reinforcement learning(DRL)-based exploration policies and leverages feature-based visual odometry(VO) for more accurate mapping and positioning results. An improved local policy is also proposed to reduce tracking failures of feature-based VO in weakly textured scenes through a refined multi-discrete action space, keyframe fusion, and an auxiliary task. The experimental results demonstrate that Ne OR has better mapping and positioning accuracy compared to other entirely learning-based exploration frameworks and improves the robustness of feature-based VO by significantly reducing tracking failures in weakly textured scenes.
The integration of artificial intelligence(AI)technology,particularly large language models(LLMs),has become essential across various sectors due to their advanced language comprehension and generation *** their trans...
详细信息
The integration of artificial intelligence(AI)technology,particularly large language models(LLMs),has become essential across various sectors due to their advanced language comprehension and generation *** their transformative impact in fields such as machine translation and intelligent dialogue systems,LLMs face significant *** challenges include safety,security,and privacy concerns that undermine their trustworthiness and effectiveness,such as hallucinations,backdoor attacks,and privacy *** works often conflated safety issues with security *** contrast,our study provides clearer and more reasonable definitions for safety,security,and privacy within the context of *** on these definitions,we provide a comprehensive overview of the vulnerabilities and defense mechanisms related to safety,security,and privacy in ***,we explore the unique research challenges posed by LLMs and suggest potential avenues for future research,aiming to enhance the robustness and reliability of LLMs in the face of emerging threats.
We consider the effects of the mass of graviton on both the waveform of gravitational waves(GWs)and the antenna response to *** determine that the effect on the response function is negligible for small *** the Fisher...
详细信息
We consider the effects of the mass of graviton on both the waveform of gravitational waves(GWs)and the antenna response to *** determine that the effect on the response function is negligible for small *** the Fisher matrix method,we perform parameter estimations with space-based GW detectors for massive binary black holes(BBHs)in massive gravity *** wavelength of massive graviton can be constrained to beλg>1.91×1019m and the mass can be constrained to be mg<1.16×10-61kg by 1-year observation of equal-mass massive BBHs with space-based GW detectors.
Traditional structured illumination microscopy techniques still face issues such as complex hardware structures, high optical path costs, low efficiency of optical sectioning reconstruction algorithms, and long image ...
详细信息
With the rapid development of intelligent video surveillance technology,pedestrian re-identification has become increasingly important inmulti-camera surveillance *** technology plays a critical role in enhancing publ...
详细信息
With the rapid development of intelligent video surveillance technology,pedestrian re-identification has become increasingly important inmulti-camera surveillance *** technology plays a critical role in enhancing public ***,traditional methods typically process images and text separately,applying upstream models directly to downstream *** approach significantly increases the complexity ofmodel training and computational ***,the common class imbalance in existing training datasets limitsmodel performance *** address these challenges,we propose an innovative framework named Person Re-ID Network Based on Visual Prompt technology andMulti-Instance Negative Pooling(VPM-Net).First,we incorporate the Contrastive Language-Image Pre-training(CLIP)pre-trained model to accurately map visual and textual features into a unified embedding space,effectively mitigating inconsistencies in data distribution and the training *** enhancemodel adaptability and generalization,we introduce an efficient and task-specific Visual Prompt Tuning(VPT)technique,which improves the model’s relevance to specific ***,we design two key modules:the Knowledge-Aware Network(KAN)and theMulti-Instance Negative Pooling(MINP)*** KAN module significantly enhances the model’s understanding of complex scenarios through deep contextual semantic *** module handles samples,effectively improving the model’s ability to distinguish fine-grained *** experimental outcomes across diverse datasets underscore the remarkable performance of *** results vividly demonstrate the unique advantages and robust reliability of VPM-Net in fine-grained retrieval tasks.
Knowledge graphs(KGs) effectively mitigate data sparsity in recommendation systems(RSs) by providing valuable auxiliary information [1]. However, traditional centralized KG-based RSs increase the risk of user privacy ...
Knowledge graphs(KGs) effectively mitigate data sparsity in recommendation systems(RSs) by providing valuable auxiliary information [1]. However, traditional centralized KG-based RSs increase the risk of user privacy *** learning(FL) enhances RS's privacy by enabling model training on decentralized data [2]. Although integrating KG and FL can address both data sparsity and privacy issues in RSs [3], several challenges persist. CH1,Each client's local model relies on a consistent global model from the server, limiting personalized deployment to endusers.
暂无评论