检索结果-内蒙古大学图书馆

Disentangled Noisy Correspondence Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Dang, Zhuohang Luo, Minnan Wang, Jihong Jia, Chengyou Han, Haochen Wan, Herun Dai, Guang Chang, Xiaojun Wang, Jingdong The School of Computer Science and Technology The Ministry of Education Key Laboratory of Intelligent Networks and Network Security The Shaanxi Province Key Laboratory of Big Data Knowledge Engineering Xi'an Jiaotong University Shaanxi Xi'An710049 China SGIT AI Lab State Grid Shaanxi Electric Power Company Limited State Grid Corporation of China Shaanxi China The School of Information Science and Technology University of Science and Technology China United Arab Emirates The Baidu Inc China

Cross-modal retrieval is crucial in understanding latent correspondences across modalities. However, existing methods implicitly assume well-matched training data, which is impractical as real-world data inevitably involves imperfect alignments, i.e., noisy correspondences. Although some works explore similarity-based strategies to address such noise, they suffer from sub-optimal similarity predictions influenced by modality-exclusive information (MEI), e.g., background noise in images and abstract definitions in texts. This issue arises as MEI is not shared across modalities, thus aligning it in training can markedly mislead similarity predictions. Moreover, although intuitive, directly applying previous cross-modal disentanglement methods suffers from limited noise tolerance and disentanglement efficacy. Inspired by the robustness of information bottlenecks against noise, we introduce DisNCL, a novel information-theoretic framework for feature Disentanglement in Noisy Correspondence Learning, to adaptively balance the extraction of MII and MEI with certifiable optimal cross-modal disentanglement efficacy. DisNCL then enhances similarity predictions in modality-invariant subspace, thereby greatly boosting similarity-based alleviation strategy for noisy correspondences. Furthermore, DisNCL introduces soft matching targets to model noisy many-to-many relationships inherent in multi-modal input for noise-robust and accurate cross-modal alignment. Extensive experiments confirm DisNCL's efficacy by 2% average recall improvement. Mutual information estimation and visualization results show that DisNCL learns meaningful MII/MEI subspaces, validating our theoretical analyses. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

A Low-rank Matching Attention based Cross-modal Feature Fusion Method for Conversational Emotion Recognition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Shou, Yuntao Liu, Huan Cao, Xiangyong Meng, Deyu Dong, Bo School of Computer Science and Technology The Ministry of Education Key Laboratory for Intelligent Networks and Network Security The Shaanxi Province Key Laboratory of Big Data Knowledge Engineering Xi’an Jiaotong Shaanxi Xi’an710049 China School of Computer Science and Technology Xi’an Jiaotong Shaanxi Xi’an710049 China The BigKE Joint Innovation Center SHMEEA Shanghai China School of Mathematics and Statistics Xi’an Jiaotong University Xi’an China

Conversational emotion recognition (CER) is an important research topic in human-computer interactions. Although recent advancements in transformer-based cross-modal fusion methods have shown promise in CER tasks, they tend to overlook the crucial intra-modal and inter-modal emotional interaction or suffer from high computational complexity. To address this, we introduce a novel and lightweight cross-modal feature fusion method called Low-Rank Matching Attention Method (LMAM). LMAM effectively captures contextual emotional semantic information in conversations while mitigating the quadratic complexity issue caused by the self-attention mechanism. Specifically, by setting a matching weight and calculating inter-modal features attention scores row by row, LMAM requires only one-third of the parameters of self-attention methods. We also employ the low-rank decomposition method on the weights to further reduce the number of parameters in LMAM. As a result, LMAM offers a lightweight model while avoiding overfitting problems caused by a large number of parameters. Moreover, LMAM is able to fully exploit the intra-modal emotional contextual information within each modality and integrates complementary emotional semantic information across modalities by computing and fusing similarities of intra-modal and inter-modal features simultaneously. Experimental results verify the superiority of LMAM compared with other popular cross-modal fusion methods on the premise of being more lightweight. Also, LMAM can be embedded into any existing state-of-the-art CER methods in a plug-and-play manner, and can be applied to other multi-modal recognition tasks, e.g., session recommendation and humour detection, demonstrating its remarkable generalization ability. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics

Generative AI for Deep Reinforcement Learning: Framework, Analysis, and Use Cases

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sun, Geng Xie, Wenwen Niyato, Dusit Mei, Fang Kang, Jiawen Du, Hongyang Mao, Shiwen College of Computer Science and Technology Jilin University Changchun130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education Jilin University Changchun130012 China College of Computing and Data Science Nanyang Technological University Singapore639798 Singapore School of Automation Guangdong University of Technology Guangzhou510006 China Department of Electrical and Electronic Engineering The University of Hong Kong 999077 Hong Kong Department of Electrical and Computer Engineering Auburn University Auburn36830 United States

As a form of artificial intelligence (AI) technology based on interactive learning, deep reinforcement learning (DRL) has been widely applied across various fields and has achieved remarkable accomplishments. However, DRL faces certain limitations, including low sample efficiency and poor generalization. Therefore, we present how to leverage generative AI (GAI) to address these issues above and enhance the performance of DRL algorithms in this paper. We first introduce several classic GAI and DRL algorithms and demonstrate the applications of GAI-enhanced DRL algorithms. Then, we discuss how to use GAI to improve DRL algorithms from the data and policy perspectives. Subsequently, we introduce a framework that demonstrates an actual and novel integration of GAI with DRL, i.e., GAI-enhanced DRL. Additionally, we provide a case study of the framework on UAV-assisted integrated near-field/far-field communication to validate the performance of the proposed framework. Moreover, we present several future directions. Finally, the related code is available at: https://***/GAI-enhanced-DRL. Copyright © 2024, The Authors. All rights reserved.

关键词： Deep reinforcement learning

ZSTAD: Zero-Shot Temporal Activity Detection

学校读者我要写书评

暂无评论

ZSTAD: Zero-Shot Temporal Activity Detection

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Lingling Zhang Xiaojun Chang Jun Liu Minnan Luo Sen Wang Zongyuan Ge Alexander Hauptmann School of Computer Science and Technology Xi'an Jiaotong University Xian China Ministry of Education Key Lab For Intelligent Networks and Network Security Xian China Faculty of Information Technology Monash University Australia National Engineering Lab for Big Data Analytics Xi'an Jiaotong University Xian China School of Information Technology and Electrical Engineering The University of Queensland Australia School of Computer Science Carnegie Mellon University USA

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

An integral part of video analysis and surveillance is temporal activity detection, which means to simultaneously recognize and localize activities in long untrimmed videos. Currently, the most effective methods of temporal activity detection are based on deep learning, and they typically perform very well with large scale annotated videos for training. However, these methods are limited in real applications due to the unavailable videos about certain activity classes and the time-consuming data annotation. To solve this challenging problem, we propose a novel task setting called zero-shot temporal activity detection (ZSTAD), where activities that have never been seen in training can still be detected. We design an end-to-end deep network based on R-C3D as the architecture for this solution. The proposed network is optimized with an innovative loss function that considers the embeddings of activity labels and their super-classes while learning the common semantics of seen and unseen activities. Experiments on both the THUMOS'14 and the Charades datasets show promising performance in terms of detecting unseen activities.

关键词： Semantics Proposals Training Task analysis Feature extraction Microsoft Windows Three-dimensional displays

CPED: A Large-Scale Chinese Personalized and Emotional Dialogue dataset for Conversational AI

学校读者我要写书评

暂无评论

TechRxiv

TechRxiv 2022年

作者： Chen, Yirong Fan, Weiquan Xing, Xiaofen Pang, Jianxin Huang, Minlie Han, Wenjing Tie, Qianfeng Xu, Xiangmin The UBTECH-SCUT Joint Research Lab Engineering Research Center Ministry of Education on Human Body Data Perception School of Electronic and Information Engineering South China University of Technology Guangzhou510641 China UBTECH Robotics Corp Shenzhen518000 China CoAI Group State Key Lab of Intelligent Technology and Systems Beijing National Research Center for Information Science and Technology Tsinghua University Beijing100084 China

Human language expression is based on the subjective construal of the situation instead of the objective truth conditions, which means that speakers’ personalities and emotions after cognitive processing have an important influence on conversation. However, most existing datasets for conversational AI ignore human personalities and emotions, or only consider part of them. It’s difficult for dialogue systems to understand speakers’ personalities and emotions although large-scale pre-training language models have been widely used. In order to consider both personalities and emotions in the process of conversation generation, we propose CPED, a large-scale Chinese personalized and emotional dialogue dataset, which consists of multi-source knowledge related to empathy and personal characteristic. These knowledge covers gender, Big Five personality traits, 13 emotions, 19 dialogue acts and 10 scenes. CPED contains more than 12K dialogues of 392 speakers from 40 TV shows. We release the textual dataset with audio features and video features according to the copyright claims, privacy issues, terms of service of video platforms. We provide detailed description of the CPED construction process and introduce three tasks for conversational AI, including personality recognition, emotion recognition in conversations as well as personalized and emotional conversation generation. Finally, we provide baseline systems for these tasks and consider the function of speakers’ personalities and emotions on conversation. Our motivation is to propose a dataset to be widely adopted by the NLP community as a new open benchmark for conversational AI research. The full dataset is available1 © 2022, CC BY-NC-SA.

关键词： Emotion Recognition

Triplet Deep Subspace Clustering via Self-Supervised data Augmentation

学校读者我要写书评

暂无评论

Triplet Deep Subspace Clustering via Self-Supervised Data Au...

IEEE International Conference on data Mining (ICDM)

作者： Zhao Zhang Xianzhen Li Haijun Zhang Yi Yang Shuicheng Yan Meng Wang School of Computer Science and Information Engineering Hefei University of Technology Hefei China Key Laboratory of Knowledge Engineering with Big Data (Ministry of Education) & Intelligent Interconnected Systems Laboratory of Anhui Province Hefei University of Technology Hefei China School of Computer Science and Technology Soochow University Suzhou China Harbin Institute of Technology (Shenzhen) Shenzhen China Centre for Artificial Intelligence University of Technology Sydney Sydney NSW Australia Sea AI Lab (SAIL) & National University of Singapore Singapore

ISBN: (纸本)9781665423991

Deep subspace clustering (DSC) with the auto-encoder and self-expression layer is of great concern due to encouraging performance. However, existing methods usually adopt a “single-task” strategy based on a single dataset, without considering other related tasks or data. As such, they cannot discover other useful information to improve the clustering task. Besides, the local structure preservation of the latent codes in mapping is usually ignored. In this paper, we therefore present an effective “multi-task” strategy via the self-supervised data augmentation, and propose a new end-to-end trainable Triplet Deep Subspace Clustering Network (TDSC-net). Specifically, TDSC-net firstly generates triplet data (i.e., anchor, positive and negative data) from input data by a spectral clustering module and a self-supervised data augmentation module. This can enable it to inherit the merits of self-supervised learning and multitask learning implicitly. After that, TDSC-net builds a triplet deep autoencoder network with a self-expression layer, which takes the triplet data as input, where they share the common network layers (i.e., autoencoder and self-expression layers) over the triple tasks for complementary learning and mutual supervision. A triplet loss is also included to retain the local information of deep latent codes, which will also benefit the self-expression. Furthermore, TDSC-net separates the self-expression layer from decoding process to improve the efficiency of reconstruction. Extensive results on several public datasets demonstrate the effectiveness of our triplet-task DSC strategy.

关键词： Codes Conferences Multitasking Decoding data mining Task analysis

CPED: A Large-Scale Chinese Personalized and Emotional Dialogue dataset for Conversational AI

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Chen, Yirong Fan, Weiquan Xing, Xiaofen Pang, Jianxin Huang, Minlie Han, Wenjing Tie, Qianfeng Xu, Xiangmin The UBTECH-SCUT Joint Research Lab Engineering Research Center of Ministry of Education on Human Body Data Perception School of Electronic and Information Engineering South China University of Technology Guangzhou510641 China UBTECH Robotics Corp Shenzhen518000 China CoAI Group State Key Lab of Intelligent Technology and Systems Beijing National Research Center for Information Science and Technology Tsinghua University Beijing100084 China

Human language expression is based on the subjective construal of the situation instead of the objective truth conditions, which means that speakers' personalities and emotions after cognitive processing have an important influence on conversation. However, most existing datasets for conversational AI ignore human personalities and emotions, or only consider part of them. It's difficult for dialogue systems to understand speakers' personalities and emotions although large-scale pre-training language models have been widely used. In order to consider both personalities and emotions in the process of conversation generation, we propose CPED, a large-scale Chinese personalized and emotional dialogue dataset, which consists of multi-source knowledge related to empathy and personal characteristic. These knowledge covers gender, Big Five personality traits, 13 emotions, 19 dialogue acts and 10 scenes. CPED contains more than 12K dialogues of 392 speakers from 40 TV shows. We release the textual dataset with audio features and video features according to the copyright claims, privacy issues, terms of service of video platforms. We provide detailed description of the CPED construction process and introduce three tasks for conversational AI, including personality recognition, emotion recognition in conversations as well as personalized and emotional conversation generation. Finally, we provide baseline systems for these tasks and consider the function of speakers' personalities and emotions on conversation. Our motivation is to propose a dataset to be widely adopted by the NLP community as a new open benchmark for conversational AI research. The full dataset is available. © 2022, CC BY-NC-ND.

关键词： Speech processing

Dictionary Pair-based data-Free Fast Deep Neural Network Compression

学校读者我要写书评

暂无评论

Dictionary Pair-based Data-Free Fast Deep Neural Network Com...

IEEE International Conference on data Mining (ICDM)

作者： Yangcheng Gao Zhao Zhang Haijun Zhang Mingbo Zhao Yi Yang Meng Wang School of Computer Science and Information Engineering Hefei University of Technology Hefei China Key Laboratory of Knowledge Engineering with Big Data (Ministry of Education) & Intelligent Interconnected Systems Laboratory of Anhui Province Hefei University of Technology Hefei China Harbin Institute of Technology (Shenzhen) Xili University Town Shenzhen China City University of Hong Kong Hong Kong SAR Centre for Artificial Intelligence University of Technology Sydney Sydney NSW Australia

ISBN: (纸本)9781665423991

Deep neural network (DNN) compression can reduce the memory footprint of deep networks effectively, so that the deep model can be deployed on the portable devices. However, most of the existing model compression methods cost lots of time, e.g., vector quantization or pruning, which makes them inept to the real-world applications that need fast online computation. In this paper, we therefore explore how to accelerate the model compression process by reducing the computation cost. Then, we propose a new deep model compression method, termed Dictionary Pair-based data-Free Fast DNN Compression, which aims at reducing the memory consumption of DNNs without extra training and can greatly improve the compression efficiency. Specifically, our proposed method performs tensor decomposition on the DNN model with a fast dictionary pair learning-based reconstruction approach, which can be deployed on different layers (e.g., convolution and fully-connection layers). Given a pre-trained DNN model, we first divide the parameters (i.e., weights) of each layer into a series of partitions for dictionary pair-based fast reconstruction, which can potentially discover more fine-grained information and provide the possibility for parallel model compression. Then, dictionaries of less memory occupation are learned to reconstruct the weights. Extensive experiments on popular DNNs (i.e., VGG-16, ResNet-18 and ResNet-50) showed that our proposed weight compression method can significantly reduce the memory footprint and speed up the compression process, with less performance loss.

关键词： Deep learning Training Performance evaluation Dictionaries Tensors Costs Computational modeling

Noise-Tolerant Learning for Audio-Visual Action Recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Han, Haochen Zheng, Qinghua Luo, Minnan Miao, Kaiyao Tian, Feng Chen, Yan The National Engineering Lab for Big Data Analytics Xi’an Jiaotong University Xi’an710049 China The School of Computer Science and Technology Xi’an Jiaotong University Xi’an710049 China The Key Laboratory of Intelligent Networks and Network Security Xi’an Jiaotong University Ministry of Education Xi’an710049 China The School of Cyber Science and Engineering Xi’an Jiaotong University Xi’an710049 China

Recently, video recognition is emerging with the help of multi-modal learning, which focuses on integrating distinct modalities to improve the performance or robustness of the model. Although various multi-modal learning methods have been proposed and offer remarkable recognition results, almost all of these methods rely on high-quality manual annotations and assume that modalities among multi-modal data provide semantically relevant information. Unfortunately, the widely used video datasets are usually coarse-annotated or collected from the Internet. Thus, it inevitably contains a portion of noisy labels and noisy correspondence. To address this challenge, we use the audio-visual action recognition task as a proxy and propose a noise-tolerant learning framework to find anti-interference model parameters against both noisy labels and noisy correspondence. Specifically, our method consists of two phases that aim to rectify noise by the inherent correlation between modalities. First, a noise-tolerant contrastive training phase is performed to make the model immune to the possible noisy-labeled data. Despite the benefits brought by contrastive training, it would overfit the noisy correspondence and thus provide false supervision. To alleviate the influence of noisy correspondence, we propose a cross-modal noise estimation component to adjust the consistency between different modalities. As the noisy correspondence existed at the instance level, we further propose a category-level contrastive loss to reduce its interference. Second, in the hybrid-supervised training phase, we calculate the distance metric among features to obtain corrected labels, which are used as complementary supervision to guide the training. Furthermore, due to the lack of suitable datasets, we establish a benchmark of real-world noisy correspondence in audio-visual data by relabeling the Kinetics dataset. Extensive experiments on a wide range of noisy levels demonstrate that our method significant

关键词： Modal analysis