检索结果-内蒙古大学图书馆

GradSA: Gradient Sparsification and Accumulation for Communication-Efficient Distributed Deep Learning 15th

学校读者我要写书评

暂无评论

GradSA: Gradient Sparsification and Accumulation for Communi...

15th International Conference on Green, Pervasive, and Cloud computing, GPC 2020

作者： Liu, Bo Jiang, Wenbin Zhao, Shaofeng Jin, Hai He, Bingsheng National Engineering Research Center for Big Data Technology and System Service Computing Technology and System Lab Cluster and Grid Computing Lab Huazhong University of Science and Technology Wuhan China Wuhan National Laboratory for Optoelectronics Key Laboratory of Information Storage System Engineering Research Center of Data Storage Systems and Technology Huazhong University of Science and Technology Wuhan China Library Henan University of Economics and Law Zhengzhou China Department of Computer Science School of Computing National University of Singapore Singapore Singapore

ISBN: (纸本)9783030642426

Large-scale distributed deep learning is of great importance in various applications. For distributed training, the inter-node gradient communication often becomes the performance bottleneck. Gradient sparsification has been proposed to reduce the communication overhead. However, the sparsification only arbitrarily selects a small fraction of gradients, while the efficiency of selection dimension has been overlooked. Furthermore, gradient staleness is inevitable after applying the sparsification which will ultimately lead to model divergence. In this paper, we propose a staleness-compensated sparse stochastic gradient descent algorithm, GradSA, to improve the training efficiency. Layer-level gradients are sparsified to reduce the communication overhead, which conform to the characteristics of the network structure, and historical accumulation of the approximated gradients is utilized to speed up convergence. We demonstrate the model convergence acceleration and the efficiency of our layer-level selection over existing state-of-the-art works, such as DGC, TernGrad, and 8-Bit quantization. © 2020, Springer Nature Switzerland AG.

关键词： Efficiency

Network Security Situational Awareness Model Based on Threat Intelligence 14th

学校读者我要写书评

暂无评论

Network Security Situational Awareness Model Based on Threat...

14th EAI International Conference on Mobile Multimedia Communications, Mobimedia 2021

作者： Zhang, Hongbin Yin, Yan Zhao, Dongmei Liu, Bin Gao, Hongbin School of Information Science and Engineering Hebei University of Science and Technology Shijiazhuang050018 China Hebei Key Laboratory of Network and Information Security Hebei Normal University Hebei Shijiazhuang050024 China School of Economics and Management Hebei University of Science and Technology Shijiazhuang050000 China Research Center of Big Data and Social Computing Hebei University of Science Shijiazhuang05000 China

ISBN: (纸本)9783030898137

In order to deal with the problems that the increasing scale of the network in the real environment leads to the continuous high incidence of network attacks, the threat intelligence was applied to situational awareness, and the situational awareness model based on random game was constructed. Threat perception of the target system was performed by comparing the similarity between the exogenous threat intelligence and the internal security events of the system. At the same time, internal threat intelligence was generated based on the threat information inside the system. In this process, game theory was used to quantify the current network security situation of the system, evaluate the security status of the network. Finally, the prediction of the network security situation was realized. The experimental results show that the network security situation awareness method based on threat intelligence can reflect the changes in the network security situation and predict attack behaviors accurately. © 2021, ICST Institute for Computer Sciences, Social Informatics and Telecommunications engineering.

关键词： Game theory

A Node-collaboration-informed Graph Convolutional Network for Precise Representation to Undirected Weighted Graphs

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Ying Yuan, Ye Luo, Xin The School of Computer Science and Technology Chongqing University of Posts and Telecommunications Chongqing400065 China The Chongqing Key Laboratory of Big Data and Intelligent Computing Chongqing Engineering Research Center of Big Data Application for Smart Cities Chongqing Institute of Green and Intelligent Technology Chinese Academy of Sciences Chongqing400714 China The College of Computer and Information Science Southwest University Chongqing400715 China

An undirected weighted graph (UWG) is frequently adopted to describe the interactions among a solo set of nodes from real applications, such as the user contact frequency from a social network services system. A graph convolutional network (GCN) is widely adopted to perform representation learning to a UWG for subsequent pattern analysis tasks such as clustering or missing data estimation. However, existing GCNs mostly neglects the latent collaborative information hidden in its connected node pairs. To address this issue, this study proposes to model the node collaborations via a symmetric latent factor analysis model, and then regards it as a node-collaboration module for supplementing the collaboration loss in a GCN. Based on this idea, a Node-collaboration-informed Graph Convolutional Network (NGCN) is proposed with three-fold ideas: a) Learning latent collaborative information from the interaction of node pairs via a node-collaboration module;b) Building the residual connection and weighted representation propagation to obtain high representation capacity;and c) Implementing the model optimization in an end-to-end fashion to achieve precise representation to the target UWG. Empirical studies on UWGs emerging from real applications demonstrate that owing to its efficient incorporation of node-collaborations, the proposed NGCN significantly outperforms state-of-the-art GCNs in addressing the task of missing weight estimation. Meanwhile, its good scalability ensures its compatibility with more advanced GCN extensions, which will be further investigated in our future studies. Copyright © 2022, The Authors. All rights reserved.

关键词： Convolution

AdAUC: End-to-end Adversarial AUC Optimization Against Long-tail Problems

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hou, Wenzheng Xu, Qianqian Yang, Zhiyong Bao, Shilong He, Yuan Huang, Qingming Key Laboratory of Intelligent Information Processing Institute of Computing Technology CAS Beijing China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing China State Key Laboratory of Information Security Institute of Information Engineering CAS Beijing China School of Cyber Security University of Chinese Academy of Sciences Beijing China Alibaba Group Beijing China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing China Artificial Intelligence Research Center Peng Cheng Laboratory Shenzhen China

It is well-known that deep learning models are vulnerable to adversarial examples. Existing studies of adversarial training have made great progress against this challenge. As a typical trait, they often assume that the class distribution is overall balanced. However, long-tail datasets are ubiquitous in a wide spectrum of applications, where the amount of head class instances is larger than the tail classes. Under such a scenario, AUC is a much more reasonable metric than accuracy since it is insensitive toward class distribution. Motivated by this, we present an early trial to explore adversarial training methods to optimize AUC. The main challenge lies in that the positive and negative examples are tightly coupled in the objective function. As a direct result, one cannot generate adversarial examples without a full scan of the dataset. To address this issue, based on a concavity regularization scheme, we reformulate the AUC optimization problem as a saddle point problem, where the objective becomes an instance-wise function. This leads to an end-to-end training protocol. Furthermore, we provide a convergence guarantee of the proposed algorithm. Our analysis differs from the existing studies since the algorithm is asked to generate adversarial examples by calculating the gradient of a min-max problem. Finally, the extensive experimental results show the performance and robustness of our algorithm in three long-tail datasets. © 2022, CC BY.

关键词： Deep learning

PointCA: Evaluating the Robustness of 3D Point Cloud Completion Models Against Adversarial Examples

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hu, Shengshan Zhang, Junwei Liu, Wei Hou, Junhui Li, Minghui Zhang, Leo Yu Jin, Hai Sun, Lichao School of Cyber Science and Engineering Huazhong University of Science and Technology China School of Computer Science and Technology Huazhong University of Science and Technology China School of Software Engineering Huazhong University of Science and Technology China Cluster and Grid Computing Lab China National Engineering Research Center for Big Data Technology and System Hubei Engineering Research Center on Big Data Security China Hubei Key Laboratory of Distributed System Security China Services Computing Technology and System Lab China City University of Hong Kong Hong Kong Deakin University Australia Lehigh University United States

Point cloud completion, as the upstream procedure of 3D recognition and segmentation, has become an essential part of many tasks such as navigation and scene understanding. While various point cloud completion models have demonstrated their powerful capabilities, their robustness against adversarial attacks, which have been proven to be fatally malicious towards deep neural networks, remains unknown. In addition, existing attack approaches towards point cloud classifiers cannot be applied to the completion models due to different output forms and attack purposes. In order to evaluate the robustness of the completion models, we propose PointCA, the first adversarial attack against 3D point cloud completion models. PointCA can generate adversarial point clouds that maintain high similarity with the original ones, while being completed as another object with totally different semantic information. Specifically, we minimize the representation discrepancy between the adversarial example and the target point set to jointly explore the adversarial point clouds in the geometry space and the feature space. Furthermore, to launch a stealthier attack, we innovatively employ the neighbourhood density information to tailor the perturbation constraint, leading to geometry-aware and distribution-adaptive modification for each point. Extensive experiments against different premier point cloud completion networks show that PointCA can cause a performance degradation from 77.9% to 16.7%, with the structure chamfer distance kept below 0.01. We conclude that existing completion models are severely vulnerable to adversarial examples, and state-of-the-art defenses for point cloud classification will be partially invalid when applied to incomplete and uneven point cloud data. Copyright © 2022, The Authors. All rights reserved.

关键词： Deep neural networks

DITE: Achieving Distributed Intelligent Traffic engineering With Sparse Network-Wide Guidance

学校读者我要写书评

暂无评论

IEEE Transactions on Network Science and engineering 2025年

作者： Guo, Yingya Zhou, Weihong Hu, Cheng Lin, Bin Deng, Yuhui Min, Geyong Fuzhou University College of Computer and Data Science Fuzhou 350025 China Fuzhou University Fujian Provincial Key Laboratory of Network Computing and Intelligent Information Processing Fuzhou 350025 China Ministry of Education Engineering Research Center of Big Data Intelligence Fuzhou 350025 China Guangdong University of Foreign Studies School of Information Science and Technology Guangzhou 510515 China Jinan University Department of Computer Science Guangzhou 510632 China University of Exeter Department of Computer Science College of Engineering Mathematics and Physical Sciences Exeter EX4 4PY United Kingdom

Distributed Traffic engineering (TE) adopts decentralized routing and offers natural merits over centralized TE in rapidly responding to dynamic network flows. However, based solely on local network observations, traditional distributed TE methods often lack sufficient routing evidence necessary for generating optimal policies. Furthermore, reinforcement learning, commonly used in distributed TE methods to avoid the need for labeled data, faces challenges in accumulating valuable routing experiences due to its trial-and-error exploration. To handle these challenges, we propose a novel Distributed Intelligent Traffic engineering (DITE) framework that empowers geographically distributed agents to rapidly and adaptively optimize routing policies for dynamic network flows with low communication overhead. Specifically, to address the limitation of optimizing local routing based solely on partial network observations, we propose enriching the routing evidence for each agent with sparse network-wide information. To promote agents to learn cooperative routing polices efficiently, we explore a new paradigm for training routing agents by seamlessly integrating imitation learning with reinforcement learning. Additionally, to efficiently describe the implicit spatiotemporal features among the routing evidence, we customize a transformer-based agent to utilize the attention mechanism for establishing the relationships between agent states and routing policies. Extensive comparative evaluations on three real network topologies and traffic traces demonstrate the superior performance of DITE in achieving distributed TE. Compared to the existing distributed TE approaches, DITE achieves a notable reduction of 35.95%–42.35% in maximum link utilization and a reduction of 23.62%-68.39% in end-to-end latency. Moreover, the experimental results indicate DITE is robust to network failures and traffic changes. © 2013 IEEE.

关键词： Distributed traffic engineering imitation learning network-wide guidance reinforcement learning transformer

Confidence-Regulated Generative Diffusion Models for Reliable AI Agent Migration in Vehicular Metaverses

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Kang, Yingkai Kang, Jiawen Wen, Jinbo Zhang, Tao Yang, Zhaohui Niyato, Dusit Zhang, Yan School of Automation Guangdong University of Technology Guangzhou510006 China College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing210016 China School of Cyberspace Science and Technology Beijing Jiaotong University Beijing100044 China College of Information Science and Electronic Engineering Zhejiang Provincial Key Lab of information processing communication and networking Zhejiang University Hangzhou310007 China College of Computing and Data Science Nanyang Technological University Singapore Department of Informatics University of Oslo the Simula Research Laboratory Norway

Vehicular metaverses are an emerging paradigm that merges intelligent transportation systems with virtual spaces, leveraging advanced digital twin and Artificial Intelligence (AI) technologies to seamlessly integrate vehicles, users, and digital environments. In this paradigm, vehicular AI agents are endowed with environment perception, decision-making, and action execution capabilities, enabling real-time processing and analysis of multi-modal data to provide users with customized interactive services. Since vehicular AI agents require substantial resources for real-time decision-making, given vehicle mobility and network dynamics conditions, the AI agents are deployed in RoadSide Units (RSUs) with sufficient resources and dynamically migrated among them. However, AI agent migration requires frequent data exchanges, which may expose vehicular metaverses to potential cyber attacks. To this end, we propose a reliable vehicular AI agent migration framework, achieving reliable dynamic migration and efficient resource scheduling through cooperation between vehicles and RSUs. Additionally, we design a trust evaluation model based on the theory of planned behavior to dynamically quantify the reputation of RSUs, thereby better accommodating the personalized trust preferences of users. We then model the vehicular AI agent migration process as a partially observable markov decision process and develop a Confidence-regulated Generative Diffusion Model (CGDM) to efficiently generate AI agent migration decisions. Numerical results demonstrate that the CGDM algorithm significantly outperforms baseline methods in reducing system latency and enhancing robustness against cyber attacks. Copyright © 2025, The Authors. All rights reserved.

关键词： Markov processes

PreZ-DGGAN: A Drug Graph GAN Based on Pre-Learning of Implicit Variables 2nd

学校读者我要写书评

暂无评论

PreZ-DGGAN: A Drug Graph GAN Based on Pre-Learning of Implic...

2nd International Conference on Applied Intelligence, ICAI 2024

作者： Liu, Yixin Fan, Yueqin Li, Zhipeng Zhang, Qinhu Big Data and Intelligent Computing Research Center Guangxi Academy of Science Nanning530007 China School of Mechanical Engineering Guangxi University Nanning530004 China Ningbo Institute of Digital Twin Eastern Institute of Technology Ningbo315201 China Institute for Regenerative Medicine Medical Innovation Center and State Key Laboratory of Cardiology School of Medicine Shanghai East Hospital Tongji University Shanghai200123 China College of Advanced Agricultural Sciences Zhejiang Agriculture and Forestry University Hangzhou311300 China

ISBN: (纸本)9789819619061

In the field of drug discovery and development, deep learning techniques have become a powerful tool to accelerate the discovery and development of new drugs. In the design and optimization of lead molecules, generative adversarial network and graph neural network are one of the prime techniques for accelerating research and development, because the former is based on the idea of two-player game to learn drug features to generate effective molecules, while the unstructured inputs of the latter are well-suited for the analysis of molecular data. Therefore, this study develops a new graph generative adversarial model, called PreZ-DGGAN, by inserting the graph neural network layer into the generative adversarial network. Within PreZ-DGGAN, the latent variable input of the generator is pre-learned by an improved graph variational autoencoder, which effectively avoids the mode collapse of the generative adversarial network, whereas the hidden layer of the discriminator is graph neural network layer, which is able to better learn the real molecules and judge the generated molecules as true or false. Experiments on two molecule datasets show that the molecules generated by PreZ-DGGAN have higher validity (100% on average) and uniqueness (91.5% on average), and the generated molecules are more diverse in structure. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Generative adversarial networks

EEGMatch: Learning with Incomplete Labels for Semi-Supervised EEG-based Cross-Subject Emotion Recognition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhou, Rushuang Ye, Weishan Zhang, Zhiguo Luo, Yanyang Zhang, Li Li, Linling Huang, Gan Dong, Yining Zhang, Yuan-Ting Liang, Zhen The School of Biomedical Engineering Medical School Shenzhen University Shenzhen518060 China The Guangdong Provincial Key Laboratory of Biomedical Measurements and Ultrasound Imaging Shenzhen518060 China Department of Biomedical Engineering City University of Hong Kong Hong Kong Hong Kong Science and Technology Park Hong Kong Institute of Computing and Intelligence Harbin Institute of Technology Shenzhen China Peng Cheng Laboratory Shenzhen China School of Data Science City University of Hong Kong Hong Kong Science and Technology Park Hong Kong

Electroencephalography (EEG) is an objective tool for emotion recognition and shows promising performance. However, the label scarcity problem is a main challenge in this field, which limits the wide application of EEG-based emotion recognition. In this paper, we propose a novel semi-supervised transfer learning framework (EEGMatch) to leverage both labeled and unlabeled EEG data. First, an EEG-Mixup based data augmentation method is developed to generate more valid samples for model learning. Second, a semi-supervised two-step pairwise learning method is proposed to bridge prototype-wise and instance-wise pairwise learning, where the prototype-wise pairwise learning measures the global relationship between EEG data and the prototypical representation of each emotion class and the instance-wise pairwise learning captures the local intrinsic relationship among EEG data. Third, a semi-supervised multi-domain adaptation is introduced to align the data representation among multiple domains (labeled source domain, unlabeled source domain, and target domain), where the distribution mismatch is alleviated. Extensive experiments are conducted on two benchmark databases (SEED and SEED-IV) under a cross-subject leave-one-subject-out cross-validation evaluation protocol. The results show the proposed EEGMatch performs better than the state-of-the-art methods under different incomplete label conditions (with 6.89% improvement on SEED and 1.44% improvement on SEED-IV), which demonstrates the effectiveness of the proposed EEGMatch in dealing with the label scarcity problem in emotion recognition using EEG signals. The source code is available at https://***/KAZABANA/EEGMatch. Copyright © 2023, The Authors. All rights reserved.

关键词： Self-supervised learning