检索结果-内蒙古大学图书馆

TM2SP: A Transformer-based Multi-Level Spatiotemporal Feature Pyramid Network for Video Saliency Prediction

IEEE Transactions on Circuits and Systems for Video Technology 2025年第6期35卷 5236-5250页

作者： Li, Chenming Liu, Shiguang Tianjin University School of Computer Science and Technology College of Intelligence and Computing Tianjin300350 China Tianjin University Tianjin Key Laboratory of Cognitive Computing and Application Tianjin300350 China

This paper proposes an end-to-end video saliency prediction network model, termed TM2SP-Net (Transformer-based Multi-level Spatiotemporal Feature Pyramid Network). Leveraging the strong encoding learning capability of Video Swin Transformer for video data, we design a Multi-level Spatiotemporal Feature Pyramid Network (MLSTFPN) that effectively detects and enriches salient regions and spatial details across different scales. In particular, a pre-trained image saliency detection encoder is employed to extract salient features from each frame, serving as prior knowledge to guide the multi-scale spatiotemporal feature fusion and decoding processes. Additionally, we introduce an Inception Gate-Controlled Fusion (IGCF) and Layered Self-Attention Aggregation Fusion (LSAF) mechanisms to efficiently merge spatiotemporal features across various stages. Finally, extensive experiments conducted on the DHF1K, Hollywood-2, UCF-Sports, and six audio-visual saliency datasets demonstrate the superiority of our method over existing state-of-the-art approaches. © 1991-2012 IEEE.

关键词： Signal encoding

来源：评论

学校读者我要写书评

暂无评论

Diversity-Enhanced Learning for Unsupervised Syntactically Controlled Paraphrase Generation 27

Diversity-Enhanced Learning for Unsupervised Syntactically C...

引用

27th European Conference on Artificial Intelligence, ECAI 2024

作者： Wu, Shaojuan Li, Jitong Sun, Yue Zhang, Xiaowang Feng, Zhiyong College of Intelligence and Computing Tianjin University Tianjin Key Laboratory of Cognitive Computing and Application Tianjin China

ISBN: (纸本)9781643685489

Syntactically controlled paraphrase generation is to generate diverse sentences that have the same semantics as the given original sentence but conform to the target syntactic structure. An optimal opportunity to enhance diversity is to make word substitutions during rephrasing based on syntactic control. Existing unsupervised methods have made great progress in syntactic control, but the generated paraphrases rarely have substitutions due to the limitation of training data. In this paper, we propose a Diversity syntactically controlled Paraphrase generation framework (DiPara), in which a novel training strategy is designed to obtain semantic sentences while using the given sentence as training objects. As diverse words vary the syntactic structure around them, we propose a phrase-aware attention mechanism to capture the syntactic structure associated with the current word. To achieve it, the linearized triple sequence is introduced to represent structure singly. Experiment results on two datasets show that DiPara outperforms strong baselines, especially diversity (Self-BLEU4) is improved by 10.18% in ParaNMT-Small. © 2024 The Authors.

关键词： Unsupervised learning

来源：评论

学校读者我要写书评

暂无评论

Adaptive Deep Graph Convolutional Network for Dialogical Speech Emotion Recognition 18th

Adaptive Deep Graph Convolutional Network for Dialogical Sp...

引用

18th National Conference on Man-Machine Speech Communication, NCMMSC 2023

作者： Liu, Jiaxing Wu, Sheng Wang, Longbiao Dang, Jianwu Tianjin Key Laboratory of Cognitive Computing and Application College of Intelligence and Computing Tianjin University Tianjin China

ISBN: (纸本)9789819706006

With the increasing demand for humanization of human-computer interaction, dialogical speech emotion recognition (SER) has attracted the attention from researchers, and it is more aligned with actual scenarios. In this paper, we propose a dialogical SER approach that includes two modules, first module is a pre-trained model with an adapter, the other module is the proposed adaptive deep graph convolutional network (ADGCN). Since emotional speech data, especially dialogical data is scarce, it is difficult for models to extract enough information through limited data. A self-supervised pre-trained framework Data2vec is introduced. An adapter is designed to integrate pre-trained model to reduce the training cost while maintaining its extensive speech-related knowledge. The representations learned by adapter are from independent utterances (intra-utterance level), and ADGCN is proposed to model dialogical contextual information in one dialogue (inter-utterance level). Two residual mechanisms, adaptive residual and dynamic local residual are designed in the proposed ADGCN, which keep it from over-smoothing issues when increasing the number of layers to model global inter-utterance level contextual information. All experiments in this paper are conducted on IEMOCAP and achieved 76.49% in term of weighted accuracy. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.

关键词： Emotion Recognition

来源：评论

学校读者我要写书评

暂无评论

An Event-based Abductive Learning for Hard Time-sensitive Question Answering 30

An Event-based Abductive Learning for Hard Time-sensitive Qu...

引用

Joint 30th International Conference on Computational Linguistics and 14th International Conference on Language Resources and Evaluation, LREC-COLING 2024

作者： Wu, Shaojuan Li, Jitong Zhang, Xiaowang Feng, Zhiyong College of Intelligence and Computing Tianjin University Tianjin Key Laboratory of Cognitive Computing and Application Tianjin China

ISBN: (纸本)9782493814104

Time-Sensitive Question Answering (TSQA) is to answer questions qualified for a certain timestamp based on the given document. It is split into easy and hard modes depending on whether the document contains time qualifiers mentioned in the question. While existing models have performed well on easy mode, their performance is significantly reduced for answering hard time-sensitive questions, whose time qualifiers are implicit in the document. The intuitive idea is to match temporal events in the given document by treating time-sensitive questions as a temporal event of missing objects. However, not all temporal events extracted from the document have explicit time qualifiers. In this paper, we propose an Event-AL framework in which a graph pruning model is designed to locate the timespan of implicit temporal events by capturing temporal relations between events. Moreover, we present an abductive reasoning module to determine proper objects while providing explanations. Besides, as the same relation may be scattered throughout the document in diverse expressions, a relation-based prompt is introduced to instruct LLMs in extracting candidate temporal events. Extensive experiments and results show that Event-AL outperforms strong baselines for hard time-sensitive questions, with a 12.7% improvement in EM scores. In addition, it also exhibits great superiority for multi-answer and beyond hard time-sensitive questions. © 2024 ELRA Language Resource Association: CC BY-NC 4.0.

关键词： Abductive Reasoning Graph Pruning Temporal Event Time-Sensitive Question Answering

来源：评论

学校读者我要写书评

暂无评论

Global and Local Hierarchical Prompt Tuning Framework for Multi-level Implicit Discourse Relation Recognition 30

Global and Local Hierarchical Prompt Tuning Framework for Mu...

引用

Joint 30th International Conference on Computational Linguistics and 14th International Conference on Language Resources and Evaluation, LREC-COLING 2024

作者： Zeng, Lei He, Ruifang Sun, Haowen Xu, Jing Liu, Chang Wang, Bo College of Intelligence and Computing Tianjin University Tianjin China Tianjin Key Laboratory of Cognitive Computing and Application Tianjin China

ISBN: (纸本)9782493814104

Multi-level implicit discourse relation recognition (MIDRR) is a challenging task to recognize the hierarchical discourse relations between the arguments with the absence of connectives. Recent methods tend to incorporate the static hierarchical structure containing all senses (defined as global hierarchy) into prompt tuning through a path prompt template or hierarchical label refining. Howerver, hierarchical modeling is independent of the verbalizer, resulting in a failure to effectively utilize the output probability distribution information of verbalizer. Besides, they ignore the utilization of the dynamic hierarchical label sequence for each instance (defined as local hierarchy) in prompt tuning. In this paper, we propose a global and local hierarchical prompt tuning (GLHPT) framework, which utilize prior knowledge of PLMs while better incorporating hierarchical information from two aspects. We leverage bottom-up propagated probability as the global hierarchy to inject it into multi-level verbalizer (MLV). Furthermore, we design a local hierarchy-driven contrastive learning (LHCL) to improve the probability distribution of MLV. Finally, our model achieves competitive results on two benchmacks. © 2024 ELRA Language Resource Association: CC BY-NC 4.0.

关键词： Probability distributions

来源：评论

学校读者我要写书评

暂无评论

Representation Degeneration Problem in Prompt-based Models for Natural Language Understanding 30

Representation Degeneration Problem in Prompt-based Models f...

引用

Joint 30th International Conference on Computational Linguistics and 14th International Conference on Language Resources and Evaluation, LREC-COLING 2024

作者： Zhao, Qingyan He, Ruifang Zhang, Jinpeng Liu, Chang Wang, Bo College of Intelligence and Computing Tianjin University Tianjin China Tianjin Key Laboratory of Cognitive Computing and Application Tianjin China

ISBN: (纸本)9782493814104

Prompt-based fine-tuning (PF), by aligning with the training objective of pre-trained language models (PLMs), has shown improved performance on many few-shot natural language understanding (NLU) benchmarks. However, the word embedding space of PLMs exhibits anisotropy, which is called the representation degeneration problem. In this paper, we explore the self-similarity within the same context and identify the anisotropy of the feature embedding space in PF model. Given that the performance of PF models is dependent on feature embeddings, we inevitably pose the hypothesis: this anisotropy limits the performance of the PF models. Based on our experimental findings, we propose CLMA, a Contrastive Learning framework based on the [MASK] token and Answers, to alleviate the anisotropy in the embedding space. By combining our proposed counter-intuitive SSD, a Supervised Signal based on embedding Distance, our approach outperforms mainstream methods on the many NLU benchmarks in the few-shot experimental settings. In subsequent experiments, we analyze the capability of our method to capture deep semantic cues and the impact of the anisotropy in the feature embedding space on the performance of the PF model. © 2024 ELRA Language Resource Association: CC BY-NC 4.0.

关键词： Anisotropy

来源：评论

学校读者我要写书评

暂无评论

Speech and Noise Dual-Stream Spectrogram Refine Network With Speech Distortion Loss For Robust Speech Recognition 48

Speech and Noise Dual-Stream Spectrogram Refine Network With...

引用

48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023

作者： Lu, Haoyu Li, Nan Song, Tongtong Wang, Longbiao Dang, Jianwu Wang, Xiaobao Zhang, Shiliang Tianjin University Tianjin Key Laboratory of Cognitive Computing and Application College of Intelligence and Computing Tianjin China

ISBN: (纸本)9781728163277

In recent years, the joint training of speech enhancement front-end and automatic speech recognition (ASR) back-end has been widely used to improve the robustness of ASR systems. Traditional joint training methods only use enhanced speech as input for the backend. However, it is difficult for speech enhancement systems to directly separate speech from input due to the diverse types of noise with different intensities. Furthermore, speech distortion and residual noise are often observed in enhanced speech, and the distortion of speech and noise is different. Most existing methods focus on fusing enhanced and noisy features to address this issue. In this paper, we propose a dual-stream spectrogram refine network to simultaneously refine the speech and noise and decouple the noise from the noisy input. Our proposed method can achieve better performance with a relative 8.6% CER reduction. © 2023 IEEE.

关键词： joint training refine network residual noise robust speech recognition speech distortion

来源：评论

学校读者我要写书评

暂无评论

Unleashing Pre-trained Masked Language Model Knowledge for Label Signal Guided Event Detection 28th

Unleashing Pre-trained Masked Language Model Knowledge for L...

引用

28th International Conference on Database Systems for Advanced applications, DASFAA 2023

作者： Xiao, Mengnan He, Ruifang Zhang, Junwei Ma, Jinsong Zhao, Haodong Tianjin Key Laboratory of Cognitive Computing and Application College of Intelligence and Computing Tianjin University Tianjin300350 China

ISBN: (纸本)9783031306747

Event detection (ED) aims to recognize triggers and their types in sentences. Previous work employs distantly supervised methods or pre-trained language models to generate sentences containing events to alleviate data scarcity. Further, determining the spans and types of triggers is complex and may have deviations. In this paper, we propose to unleash Pre-trained Masked Language Model (PMLM) knowledge for label signal guided ED by a novel trigger augmentation. We directly generate triggers by leveraging the rich knowledge of PMLM through masking triggers. However, these newly replaced triggers may not correspond to the label of the masked trigger. To control such trigger augmentation noises, we design a label signal guided classification mechanism with event type-subtype guidance. To ensure the quality of generated triggers, a semantic consistency mechanism is introduced. Experimental results on the ACE2005 and FewEvent show the effectiveness of our proposed approach. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

Dual Hierarchical Contrastive Learning for Multi-level Implicit Discourse Relation Recognition 12th

Dual Hierarchical Contrastive Learning for Multi-level Impl...

引用

12th National CCF Conference on Natural Language Processing and Chinese computing, NLPCC 2023

作者： Xu, Jing He, Ruifang Zhao, Haodong Wang, Huijie Zeng, Lei College of Intelligence and Computing Tianjin University Tianjin China Tianjin Key Laboratory of Cognitive Computing and Application Tianjin China

ISBN: (纸本)9783031446955

Implicit discourse relation recognition (IDRR) is a challenging but vital task in discourse analysis, which aims at classifying logical relations between arguments. Previous work infuses ACCL takes external knowledge or label semantics to alleviate data scarcity, which either brings noise or underutilizes semantic information contained in label embedding. Meanwhile, it is difficult to model label hierarchy. In this paper, we make full use of label embedding as positives and negatives for our dual hierarchical contrastive learning framework, which contains two parts: 1) hierarchical label contrastive loss (HLCL), which promotes fine-grained labels to be more similar to correlative medium-grained labels than related coarse-grained labels. 2) arguments and connectives contrastive loss (ACCL), which makes arguments aggregate around correlative fine-grained labels. The two modules interact with each other, making the similarity between arguments and correlative fine-grained labels are higher than that with related coarse-grained labels. In this process, the multi-level label semantics are integrated to arguments, which provides guidance for classification. Experimental results show that our method achieves competitive performance against state-of-the-art system. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.

关键词： Embeddings

来源：评论

学校读者我要写书评

暂无评论

Task-Adaptive Generative Adversarial Network Based Speech Dereverberation for Robust Speech Recognition 1

引用

18th National Conference on Man-Machine Speech Communication, NCMMSC 2023

作者： Liu, Ji Li, Nan Ge, Meng Fu, Yanjie Wang, Longbiao Dang, Jianwu Tianjin Key Laboratory of Cognitive Computing and Application College of Intelligence and Computing Tianjin University Tianjin China Japan Advanced Institute of Science and Technology Ishikawa Nomi Japan

ISBN: (数字)9789819706013

ISBN: (纸本)9789819706006

Reverberation is known to severely affect speech recognition performance when speech is recorded in an enclosed space. Deep learning-based speech dereverberation has been remarkably successful in recent years, achieving superior recognition performance for far-field speech applications. However, the output from conventional dereverberation systems cannot be guaranteed suitable for back-end recognition systems because of their different task goals. To bridge the gap between the front-end dereverberation and the back-end recognition, we propose a novel task-adaptive speech dereverberation generative adversarial network (GAN) based speech dereverberation model called Task-adaptive GAN. Specifically, we propose to replace the binary-valued discriminator in a regular generative adversarial network with a novel senone-predicted discriminator, and also introduce a well-designed recognition-aware generator as a dereverberation system. By doing so, the corresponding output distribution will be more suitable for the recognition task. Experimental results on the REVERB corpus show that our proposed approach achieves a relative 18.6% and 8.6% word error rate reduction than the traditional GAN-based baseline system on the simulated set and real set, respectively. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.

关键词： Generative adversarial networks

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：