检索结果-内蒙古大学图书馆

Motif Graph Neural Network

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Chen, Xuexin Cai, Ruichu Fang, Yuan Wu, Min Li, Zijian Hao, Zhifeng The School of Computer Science Guangdong University of Technology Guangzhou510006 China The School of Computer Science Guangdong University of Technology Guangdong Provincial Key Laboratory of Public Finance and Taxation with Big Data Application Guangzhou China Peng Cheng Laboratory Shenzhen518066 China The School of Computing and Information Systems Singapore Management University 178902 Singapore A*STAR 138632 Singapore The College of Science Shantou University Shantou515063 China

Graphs can model complicated interactions between entities, which naturally emerge in many important applications. These applications can often be cast into standard graph learning tasks, in which a crucial step is to learn low-dimensional graph representations. Graph neural networks (GNNs) are currently the most popular model in graph embedding approaches. However, standard GNNs in the neighborhood aggregation paradigm suffer from limited discriminative power in distinguishing high-order graph structures as opposed to low-order structures. To capture high-order structures, researchers have resorted to motifs and developed motif-based GNNs. However, existing motif-based GNNs still often suffer from less discriminative power on high-order structures. To overcome the above limitations, we propose Motif Graph Neural Network (MGNN), a novel framework to better capture high-order structures, hinging on our proposed motif redundancy minimization operator and injective motif combination. First, MGNN produces a set of node representations w.r.t. each motif. The next phase is our proposed redundancy minimization among motifs which compares the motifs with each other and distills the features unique to each motif. Finally, MGNN performs the updating of node representations by combining multiple representations from different motifs. In particular, to enhance the discriminative power, MGNN utilizes an injective function to combine the representations w.r.t. different motifs. We further show that our proposed architecture increases the expressive power of GNNs with a theoretical analysis. We demonstrate that MGNN outperforms state-of-the-art methods on seven public benchmarks on both node classification and graph classification tasks. Copyright © 2021, The Authors. All rights reserved.

关键词： Graph neural networks

Enhancing Event Tagger with Automatic Speech Recognizer for Audio Multi-task Scenarios by Distillation with Pre-Trained Large Models

学校读者我要写书评

暂无评论

Enhancing Event Tagger with Automatic Speech Recognizer for ...

IEEE International Conference on Systems, Man and Cybernetics

作者： Jianfeng cheng Ye Liu Jian Yin Liangdao Wang Yan Pan Artificial Intelligence Department Lizhi Inc. Beijing & Guangzhou China School of Computer Science and Engineering Sun Yat-sen University Guangzhou China Guangdong Key Laboratory of Big Data Analysis and Processing Guangzhou China School of Artificial Intelligence Sun Yat-sen University Zhuhai China

ISBN: (数字)9781665410205

ISBN: (纸本)9781665410212

With the continuous expansion of robotics and digital humans in practical applications, the demand for the auditory system is becoming deeper, usually requiring more efficient speech recognition framework capabilities to handle multiple tasks and use fewer resources. In previous audio processing frameworks, each type of audio processing task typically requires constructing a standalone deep network model for training, which results in more training data and higher training time when constructing models for multi-task audio scenarios simultaneously. The recent improvement of audio models based on transformers have brought about methods that can handle multiple audio tasks concurrently. However, recent related methods still require retraining multi-task targets with an amount of data, and achieve the general effect after training for multi-task scenarios than the simple combination of standalone methods processed separately. In order to better build a model that can handle multiple audio tasks, we propose a novel framework of distillation through pre-trained large models for enhancing event tagger with automatic speech recognizer. Through multiple rounds of experiments on several audio datasets, it has been verified that the proposed framework can achieve better results than the baseline for multitasking, and comparative results with less parameters compared to the baselines for single-task scenarios.

关键词： Training Adaptation models Analytical models Training data Speech recognition Speech enhancement Multitasking Transformers data models Robots

Opara: Exploiting Operator Parallelism for Expediting DNN Inference on GPUs

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Chen, Aodong Xu, Fei Han, Li Dong, Yuan Chen, Li Zhou, Zhi Liu, Fangming The Shanghai Key Laboratory of Multidimensional Information Processing School of Computer Science and Technology East China Normal University 3663 N. Zhongshan Road Shanghai200062 China The School of Software Engineering East China Normal University 3663 N. Zhongshan Road Shanghai200062 China The School of Computing and Informatics University of Louisiana at Lafayette 301 East Lewis Street LafayetteLA70504 United States The Guangdong Key Laboratory of Big Data Analysis and Processing School of Computer Science and Engineering Sun Yat-sen University 132 E. Waihuan Road Guangzhou510006 China Peng Cheng Laboratory Huazhong University of Science and Technology China

GPUs have become the defacto hardware devices for accelerating Deep Neural Network (DNN) inference workloads. However, the conventional sequential execution mode of DNN operators in mainstream deep learning frameworks cannot fully utilize GPU resources, even with the operator fusion enabled, due to the increasing complexity of model structures and a greater diversity of operators. Moreover, the inadequate operator launch order in parallelized execution scenarios can lead to GPU resource wastage and unexpected performance interference among operators. In this paper, we propose Opara, a resource- and interference-aware DNN Operator parallel scheduling framework to accelerate DNN inference on GPUs. Specifically, Opara first employs CUDA Streams and CUDA Graph to parallelize the execution of multiple operators automatically. To further expedite DNN inference, Opara leverages the resource demands of operators to judiciously adjust the operator launch order on GPUs, overlapping the execution of compute-intensive and memory-intensive operators. We implement and open source a prototype of Opara based on PyTorch in a non-intrusive manner. Extensive prototype experiments with representative DNN and Transformer-based models demonstrate that Opara outperforms the default sequential CUDA Graph in PyTorch and the state-of-the-art operator parallelism systems by up to 1.68× and 1.29×, respectively, yet with acceptable runtime overhead. Copyright © 2023, The Authors. All rights reserved.

关键词： Deep neural networks

AdAUC: End-to-end Adversarial AUC Optimization Against Long-tail Problems

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hou, Wenzheng Xu, Qianqian Yang, Zhiyong Bao, Shilong He, Yuan Huang, Qingming Key Laboratory of Intelligent Information Processing Institute of Computing Technology CAS Beijing China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing China State Key Laboratory of Information Security Institute of Information Engineering CAS Beijing China School of Cyber Security University of Chinese Academy of Sciences Beijing China Alibaba Group Beijing China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing China Artificial Intelligence Research Center Peng Cheng Laboratory Shenzhen China

It is well-known that deep learning models are vulnerable to adversarial examples. Existing studies of adversarial training have made great progress against this challenge. As a typical trait, they often assume that the class distribution is overall balanced. However, long-tail datasets are ubiquitous in a wide spectrum of applications, where the amount of head class instances is larger than the tail classes. Under such a scenario, AUC is a much more reasonable metric than accuracy since it is insensitive toward class distribution. Motivated by this, we present an early trial to explore adversarial training methods to optimize AUC. The main challenge lies in that the positive and negative examples are tightly coupled in the objective function. As a direct result, one cannot generate adversarial examples without a full scan of the dataset. To address this issue, based on a concavity regularization scheme, we reformulate the AUC optimization problem as a saddle point problem, where the objective becomes an instance-wise function. This leads to an end-to-end training protocol. Furthermore, we provide a convergence guarantee of the proposed algorithm. Our analysis differs from the existing studies since the algorithm is asked to generate adversarial examples by calculating the gradient of a min-max problem. Finally, the extensive experimental results show the performance and robustness of our algorithm in three long-tail datasets. © 2022, CC BY.

关键词： Deep learning

DRL-M4MR: An Intelligent Multicast Routing Approach Based on DQN Deep Reinforcement Learning in SDN

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhao, Chenwei Ye, Miao Xue, Xingsi Lv, Jianhui Jiang, Qiuxiang Wang, Yong School of Information and Communication Guilin University of Electronic Technology Guilin541004 China Guangxi Key Laboratory of Wireless Wideband Communication and Signal Processing Guilin University of Electronic Technology Guilin541004 China Fujian Provincial Key Laboratory of Big Data Mining and Applications Fujian University of Technology Fujian Fuzhou350118 China Peng Cheng Lab. Guangdong Shenzhen518038 China School of Computer Science and Information Security Guilin University of Electronic Technology Guilin541004 China

Traditional multicast routing methods have some problems in constructing a multicast tree, such as limited access to network state information, poor adaptability to dynamic and complex changes in the network, and inflexible data forwarding. To address these defects, the optimal multicast routing problem in software-defined networking (SDN) is tailored as a multi-objective optimization problem, and an intelligent multicast routing algorithm DRL-M4MR based on the deep Q network (DQN) deep reinforcement learning (DRL) method is designed to construct a multicast tree in SDN. First, the multicast tree state matrix, link bandwidth matrix, link delay matrix, and link packet loss rate matrix are designed as the state space of the DRL agent by combining the global view and control of the SDN. Second, the action space of the agent is all the links in the network, and the action selection strategy is designed to add the links to the current multicast tree under four cases. Third, single-step and final reward function forms are designed to guide the intelligence to make decisions to construct the optimal multicast tree. The experimental results show that, compared with existing algorithms, the multicast tree construct by DRL-M4MR can obtain better bandwidth, delay, and packet loss rate performance after training, and it can make more intelligent multicast routing decisions in a dynamic network environment. Copyright © 2022, The Authors. All rights reserved.

关键词： Reinforcement learning

Not All Diffusion Model Activations Have Been Evaluated as Discriminative Features

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Meng, Benyuan Xu, Qianqian Wang, Zitai Cao, Xiaochun Huang, Qingming Institute of Information Engineering CAS China School of Cyber Security University of Chinese Academy of Sciences China Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS China Peng Cheng Laboratory China School of Cyber Science and Tech. Shenzhen Campus of Sun Yat-sen University China School of Computer Science and Tech. University of Chinese Academy of Sciences China Key Laboratory of Big Data Mining and Knowledge Management CAS China

Diffusion models are initially designed for image generation. Recent research shows that the internal signals within their backbones, named activations, can also serve as dense features for various discriminative tasks such as semantic segmentation. Given numerous activations, selecting a small yet effective subset poses a fundamental problem. To this end, the early study of this field performs a large-scale quantitative comparison of the discriminative ability of the activations. However, we find that many potential activations have not been evaluated, such as the queries and keys used to compute attention scores. Moreover, recent advancements in diffusion architectures bring many new activations, such as those within embedded ViT modules. Both combined, activation selection remains unresolved but overlooked. To tackle this issue, this paper takes a further step with a much broader range of activations evaluated. Considering the significant increase in activations, a full-scale quantitative comparison is no longer operational. Instead, we seek to understand the properties of these activations, such that the activations that are clearly inferior can be filtered out in advance via simple qualitative evaluation. After careful analysis, we discover three properties universal among diffusion models, enabling this study to go beyond specific models. On top of this, we present effective feature selection solutions for several popular diffusion models. Finally, the experiments across multiple discriminative tasks validate the superiority of our method over the SOTA competitors. Our code is available at this url. © 2024, CC BY.

关键词： Semantics

Multiple residual dense networks for reconfigurable intelligent surfaces cascaded channel estimation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Jin, Yu Zhang, Jiayi Huang, Chongwen Yang, Liang Xiao, Huahua Ai, Bo Wang, Zhiqin The School of Electronic and Information Engineering Beijing Jiaotong University Beijing100044 China The Frontiers Science Center for Smart High-speed Railway System Beijing Jiaotong University Beijing100044 China Zhejiang Provincial Key Lab of Information Processing Communication and Networking Zhejiang University Hangzhou310007 China College of Computer Science and Electronic Engineering Hunan University Changsha410082 China ZTE Corporation State Key Laboratory of Mobile Network Mobile Multimedia Technology Shenzhen518057 China State Key Laboratory of Rail Traffic Control and Safety Beijing Jiaotong University Beijing100044 China Henan Joint International Research Laboratory of Intelligent Networking and Data Analysis Zhengzhou University Zhengzhou450001 China Research Center of Networks and Communications Peng Cheng Laboratory Shenzhen518055 China China Academy of Information and Communications Technology Beijing100191 China

Reconfigurable intelligent surface (RIS) constitutes an essential and promising paradigm that relies programmable wireless environment and provides capability for space-intensive communications, due to the use of low-cost massive reflecting elements over the entire surfaces of man-made structures. However, accurate channel estimation is a fundamental technical prerequisite to achieve the huge performance gains from RIS. By leveraging the low rank structure of RIS channels, three practical residual neural networks, named convolutional blind denoising network, convolutional denoising generative adversarial networks and multiple residual dense network, are proposed to obtain accurate channel state information, which can reflect the impact of different methods on the estimation performance. Simulation results reveal the evolution direction of these three methods and reveal their superior performance compared with existing benchmark schemes. Copyright © 2021, The Authors. All rights reserved.

关键词： Channel estimation