检索结果-内蒙古大学图书馆

FEDGT: FEDERATED NODE CLASSIFICATION WITH SCALABLE GRAPH TRANSFORMER

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Zaixi Hu, Qingyong Yu, Yang Gao, Weibo Liu, Qi Anhui Province Key Lab of Big Data Analysis and Application School of Computer Science and Technology University of Science and Technology of China China State Key Laboratory of Cognitive Intelligence Anhui Hefei China Hong Kong University of Science and Technology Hong Kong

Graphs are widely used to model relational data. As graphs are getting larger and larger in real-world scenarios, there is a trend to store and compute subgraphs in multiple local systems. For example, recently proposed subgraph federated learning methods train Graph Neural Networks (GNNs) distributively on local subgraphs and aggregate GNN parameters with a central server. However, existing methods have the following limitations: (1) The links between local subgraphs are missing in subgraph federated learning. This could severely damage the performance of GNNs that follow message-passing paradigms to update node/edge features. (2) Most existing methods overlook the subgraph heterogeneity issue, brought by subgraphs being from different parts of the whole graph. To address the aforementioned challenges, we propose a scalable Federated Graph Transformer (FedGT) in the paper. Firstly, we design a hybrid attention scheme to reduce the complexity of the Graph Transformer to linear while ensuring a global receptive field with theoretical bounds. Specifically, each node attends to the sampled local neighbors and a set of curated global nodes to learn both local and global information and be robust to missing links. The global nodes are dynamically updated during training with an online clustering algorithm to capture the data distribution of the corresponding local subgraph. Secondly, FedGT computes clients' similarity based on the aligned global nodes with optimal transport. The similarity is then used to perform weighted averaging for personalized aggregation, which well addresses the data heterogeneity problem. Finally, extensive experimental results on 6 datasets and 2 subgraph settings demonstrate the superiority of FedGT. © 2024, CC BY.

关键词： Graph neural networks

Research on Instance Segmentation of High-Resolution Remote Sensing Images

学校读者我要写书评

暂无评论

Research on Instance Segmentation of High-Resolution Remote ...

Artificial Intelligence technology (ACAIT), Asian Conference on

作者： Xiaoying Zhang Houqun Yang Xuanyu Li Dangui Yang Jie Shen Gaodi Xu College of Computer Science and Technology Hainan University Haikou China Haikou Key Laboratory of Deep Learning and Big Data Application Technology Hainan University Haikou China

ISBN: (数字)9798350359145

ISBN: (纸本)9798350359152

Increasingly, high-resolution remote sensing images are being used in real life as a result of the continuous advancement of high-definition remote sensing imaging technology in recent years. They play a crucial role in urban planning, agriculture, oceans, ecological protection, and other fields. A growing amount of remote sensing image data has led to a growing interest in using image instance segmentation to decode remote sensing images by automatically extracting valuable information. The present paper examines and analyzes a large number of instance segmentation methods on remote sensing images. It begins with an explanation of the classification and research progress of natural image instance segmentation techniques, which are categorized into two-stage instance segmentation, single-stage instance segmentation, multi-stage instance segmentation and weakly supervised instance segmentation, and also analyzed in terms of their advantages and shortcomings. And then we discussed it in more detail of remote sensing image instance segmentation using deep learning, and it is explained that each of these methods can be classified into three types based on its technical characteristics: methods that combine multiscale feature fusion, methods that combine dilated convolution and methods that combine attention mechanisms, as well as the benefits and limitations of each method. The results of classical experiments on different datasets are presented, along with an explanation of the commonly used datasets and evaluation metrics in instance segmentation. This paper presents an overview of the current problems and challenges associated with remote sensing image instance segmentation, as well as future trends and developments.

关键词： Instance segmentation Deep learning Training Convolution Buildings Urban planning Vegetation mapping

VISION TRANSFORMER BASED RANDOM WALK FOR GROUP RE-IDENTIFICATION

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Guoqing Liu, Tianqi Fang, Wenxuan Zheng, Yuhui School of Computer Science Nanjing University of Information Science and Technology Nanjing China The State Key Laboratory of Tibetan Intelligent Information Processing and Application Qinghai Normal University Xining China

Group re-identification (re-ID) aims to match groups with the same people under different cameras, mainly involves the challenges of group members and layout changes well. Most existing methods usually use the k-nearest neighbor algorithm to update node features to consider changes in group membership, but these methods cannot solve the problem of group layout changes. To this end, we propose a novel vision transformer based random walk framework for group re-ID. Specifically, we design a vision transformer based on a monocular depth estimation algorithm to construct a graph through the average depth value of pedestrian features to fully consider the impact of camera distance on group members relationships. In addition, we propose a random walk module to reconstruct the graph by calculating affinity scores between target and gallery images to remove pedestrians who do not belong to the current group. Experimental results show that our framework is superior to most methods. Copyright © 2024, The Authors. All rights reserved.

关键词： Nearest neighbor search

APADGCN: Adaptive Partial Attention Diffusion Graph Convolutional Network for Traffic Flow Forecasting 4th

学校读者我要写书评

暂无评论

APADGCN: Adaptive Partial Attention Diffusion Graph Convolut...

4th International Conference on Spatial Data and Intelligence, SpatialDI 2023

作者： Zhang, Bowen Li, Bohan Wei, Jinzhan Wen, Hao College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing211106 China Key Laboratory of Safety-Critical Software Ministry of Industry and Information Technology Beijing China National Engineering Laboratory for Integrated Aero-Space-Ground Ocean Big Data Application Technology Xi’an China School of Electronic Information and Automation Guilin University of Aerospace Technology Guilin541010 China College of Aerospace Engineering Nanjing University of Aeronautics and Astronautics Nanjing211106 China

ISBN: (纸本)9783031329098

Traffic flow forecasting is a core task of urban governance and plays a vital role in the development of ITS. Because of the complexity and uncertainty of traffic patterns, it is of great challenge to capture spatial-temporal correlations. Recent researches mainly focus on the pre-defined adjacency matrix based on prior knowledge as the basis of spatial-temporal correlation modeling, but the fixed graph structure cannot adequately describe the dependency between traffic sensors. To tackle this issue, a novel deep learning model framework is proposed in this paper: Adaptive Partial Attention Diffusion Graph Convolutional Network(APADGCN), which consists of three main parts: 1) the Multi-Component module that divides the historical traffic flow into recent, daily-periodic, and weekly-periodic, to capture the traffic patterns of different periodic;2) the spatial correlation modeling which can dynamically capture node relationships and model spatial dependency, and enhance the aggregation ability of low-order information;3) the temporal correlation modeling which models long-term time dependencies using convolution and gating. The final result is obtained by the weighted fusion of the results of the multi-components. We compared our APADGCN with various baseline models in the four real datasets from the Caltrans Performance Measurement System (PeMS). The experimental results show that the prediction accuracy of APADGCN is better than that of the baseline models. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

关键词： Convolution

Towards a Holistic Understanding of Mathematical Questions with Contrastive Pre-training

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ning, Yuting Huang, Zhenya Lin, Xin Chen, Enhong Tong, Shiwei Gong, Zheng Wang, Shijin Anhui Province Key Laboratory of Big Data Analysis and Application School of Computer Science and Technology University of Science and Technology of China China State Key Laboratory of Cognitive Intelligence China iFLYTEK Co. Ltd China

Understanding mathematical questions effectively is a crucial task, which can benefit many applications, such as difficulty estimation. Researchers have drawn much attention to designing pre-training models for question representations due to the scarcity of human annotations (e.g., labeling difficulty). However, unlike general free-format texts (e.g., user comments), mathematical questions are generally designed with explicit purposes and mathematical logic, and usually consist of more complex content, such as formulas, and related mathematical knowledge (e.g., Function). Therefore, the problem of holistically representing mathematical questions remains underexplored. To this end, in this paper, we propose a novel contrastive pre-training approach for mathematical question representations, namely QuesCo, which attempts to bring questions with more similar purposes closer. Specifically, we first design two-level question augmentations, including content-level and structure-level, which generate literally diverse question pairs with similar purposes. Then, to fully exploit hierarchical information of knowledge concepts, we propose a knowledge hierarchy-aware rank strategy (KHAR), which ranks the similarities between questions in a fine-grained manner. Next, we adopt a ranking contrastive learning task to optimize our model based on the augmented and ranked questions. We conduct extensive experiments on two real-world mathematical datasets. The experimental results demonstrate the effectiveness of our model. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine learning

Enhanced Fine-Tuning of Lightweight Domain-Specific Q&A Model Based on Large Language Models

学校读者我要写书评

暂无评论

Enhanced Fine-Tuning of Lightweight Domain-Specific Q&A Mode...

IEEE International Conference on Software Reliability Engineering Workshops (ISSRE Wksp)

作者： Shenglin Zhang Pengtian Zhu Minghua Ma Jiagang Wang Yongqian Sun Dongwen Li Jingyu Wang Qianying Guo Xiaolei Hua Lin Zhu Dan Pei Nankai University Haihe Laboratory of Information Technology Application Innovation Microsoft Tsinghua University Tianjin Key Laboratory of Software Experience and Human Computer Interaction China Mobile Research Institute Beijing National Research Center for Information Science and Technology

ISBN: (数字)9798350367041

ISBN: (纸本)9798350367058

Large language models (LLMs) excel at general question-answering (Q&A) but often fall short in specialized domains due to a lack of domain-specific knowledge. Commercial companies face the dual challenges of privacy protection and resource constraints when involving LLMs for fine-tuning. This paper propose a novel framework, Self-Evolution, designed to address these issues by leveraging lightweight open-source LLMs through multiple iterative fine-tuning rounds. To enhance the efficiency of iterative fine-tuning, Self-Evolution employ a strategy that filters and reinforces the knowledge with higher value during the iterative process. We employed Self-Evolution on Qwen1.5-7B-Chat using 4,000 documents containing rich domain knowledge from China Mobile, achieving a performance score 174% higher on domain-specific question-answering evaluations than Qwen1.5-7B-Chat and even 22% higher than Qwen1.5-72B-Chat. Self-Evolution has been deployed in China Mobile’s daily operation and maintenance for 117 days, and it improves the efficiency of locating alarms, fixing problems, and finding related reports, with an average efficiency improvement of over 18.6%. In addition, we release Self-Evolution framework code in https://***/Zero-Pointer/Self-Evolution.

关键词： Privacy Filters Large language models Conferences Software reliability Maintenance Iterative methods Data mining Protection Faces

Reliability Analysis of Heterogeneous Sensor-Cloud Systems against Targeted Attacks

学校读者我要写书评

暂无评论

China Communications 2022年第8期19卷 181-197页

作者： Hao Peng Zhen Qian Guangquan Xu Kejie Mao Kangtian Li Dandan Zhao Department of Computer Science and Engineering Zhejiang Normal UniversityJinhua Zhejiang 321004China Key Laboratory of Intelligent Education Technology and Application of Zhejiang Province Zhejiang Normal UniversityJinhua Zhejiang 321004China The Big Data School Qingdao Huanghai UniversityQingdao Shandong 266427China College of Intelligence and Computing Tianjin UniversityTianjin 300072China

Based on the wide application of cloud computing and wireless sensor networks in various fields,the Sensor-Cloud System(SCS)plays an indispensable role between the physical world and the network ***,due to the close connection and interdependence between the physical resource network and computing resource network,there are security problems such as cascading failures between systems in the *** this paper,we propose a model with two interdependent networks to represent a sensor-cloud ***,based on the percolation theory,we have carried out a formulaic theoretical analysis of the whole process of cascading *** the system’s subnetwork presents a steady state where there is no further collapse,we can obtain the largest remaining connected subgroup components and the penetration ***,this result is the critical maximum that the coupled SCS can *** verify the correctness of the theoretical results,we further carried out actual simulation *** results show that a scale-free network priority attack’s percolation threshold is always less than that of ER network which is priority ***,when the scale-free network is attacked first,adding the power law exponentλcan be more intuitive and more effective to improve the network’s reliability.

关键词： sensor-cloud systems targeted attacks heterogeneous networks reliability analysis system security

Bit-mask Robust Contrastive Knowledge Distillation for Unsupervised Semantic Hashing

学校读者我要写书评

暂无评论

arXiv 2024年

作者： He, Liyang Chen, Enhong Huang, Zhenya Wang, Fei Wang, Shijin Liu, Jiayu Sha, Jing School of Computer Science and Technology University of Science and Technology of China State Key Laboratory of Cognitive Intelligence Hefei China Anhui Province Key Laboratory of Big Data Analysis and Application University of Science and Technology of China State Key Laboratory of Cognitive Intelligence Hefei China State Key Laboratory of Cognitive Intelligence Hefei China School of Data Science University of Science and Technology of China State Key Laboratory of Cognitive Intelligence Hefei China iFLYTEK AI Research State Key Laboratory of Cognitive Intelligence Hefei China

Unsupervised semantic hashing has emerged as an indispensable technique for fast image search, which aims to convert images into binary hash codes without relying on labels. Recent advancements in the field demonstrate that employing large-scale backbones (e.g., ViT) in unsupervised semantic hashing models can yield substantial improvements. However, the inference delay has become increasingly difficult to overlook. Knowledge distillation provides a means for practical model compression to alleviate this delay. Nevertheless, the prevailing knowledge distillation approaches are not explicitly designed for semantic hashing. They ignore the unique search paradigm of semantic hashing, the inherent necessities of the distillation process, and the property of hash codes. In this paper, we propose an innovative Bit-mask Robust Contrastive knowledge Distillation (BRCD) method, specifically devised for the distillation of semantic hashing models. To ensure the effectiveness of two kinds of search paradigms in the context of semantic hashing, BRCD first aligns the semantic spaces between the teacher and student models through a contrastive knowledge distillation objective. Additionally, to eliminate noisy augmentations and ensure robust optimization, a cluster-based method within the knowledge distillation process is introduced. Furthermore, through a bit-level analysis, we uncover the presence of redundancy bits resulting from the bit independence property. To mitigate these effects, we introduce a bit mask mechanism in our knowledge distillation objective. Finally, extensive experiments not only showcase the noteworthy performance of our BRCD method in comparison to other knowledge distillation methods but also substantiate the generality of our methods across diverse semantic hashing models and backbones. The code for BRCD is available at https://***/hly1998/BRCD. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

Long and Recent Preference Learning with Recent-k Items Distribution for Recommender System

学校读者我要写书评

暂无评论

IEEE Transactions on Multimedia 2025年

作者： Gao, Yongbiao Niu, Sijie Lv, Guohua Ling, Miaogen Geng, Xin Qilu University of Technology Shandong Academy of Sciences Key Laboratory of Computing Power Network and Information Security Ministry of Education Shandong Computer Science Center National Supercomputer Center in Jinan Jinan China Shandong Fundamental Research Center for Computer Science Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing Jinan China Southeast University Ministry of Education Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Application China University of Jinan Shandong Provincial Key Laboratory of Ubiquitous Intelligent Computing School of Information Science and Engineering China Nanjing University of Information Science and Technology School of Computer Science China Southeast University School of Computer Science and Engineering China

Reinforcement learning (RL) aims to formulate the recommendation task as a Markov decision process (MDP) and trains an agent to automatically learn the optimal recommendation policy from interaction trajectories through trial-and-error and reward mechanisms. However, most existing RL-based approaches overlook the correlation between items and the dynamics of user interests implied in temporally close interactions. Therefore, in this paper, we propose a reinforcement learning method that incorporates a "recent-k items"distribution to capture users' local preferences. Specifically, we model the output layer as two distinct branches. The "recent-k items"branch, formulated with a Kullback-Leibler divergence loss, learns the recent interests of users, whereas the other branch utilizes a one-step temporal difference error to capture long-term preferences. The proposed structure is integrated into deep Q-learning and actor-critics, resulting in two enhanced methods named R k Q and R k AC, respectively. Furthermore, a novel soft inter-reward is carefully designed to enhance the proposed method, and we theoretically prove the convergence of the proposed algorithm. We perform extensive experiments on two large real-world datasets and conduct further analysis of the influences of different action sequences, time intervals, and enhancement capabilities for state-of-the-art models. The experimental results demonstrate the efficacy of our proposed methods. © 1999-2012 IEEE.

关键词： Deep reinforcement learning