检索结果-内蒙古大学图书馆

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Yu, Liu Guo, Ludie Kuang, Ping Zhou, Fan University of Electronic Science and Technology of China Sichuan Chengdu610054 China

ISBN: (纸本)9798350368741

pre-trained language models (PLMs) are trained on data that inherently contains gender biases, leading to undesirable impacts. Traditional debiasing methods often rely on external corpora, which may lack quality, diversity, or demographic balance, affecting the effectiveness of debiasing. With the rise of large language models and their extensive knowledge, we propose enhancing fairness (Fair-Gender) in PLMs by absorbing coherent, attribute-balanced, and semantically rich sentences. However, these sentences cannot be directly used for debiasing due to alignment issues and the risk of negative transfer. We address this by applying causal analysis to estimate causal effects, filtering out unaligned sentences, and identifying aligned ones for incorporation into PLMs, thereby ensuring positive transfer. Experiments show that our approach significantly reduces gender biases in PLMs while preserving their language expressiveness. © 2025 IEEE.

关键词： Fairness pre-trained language model social debiasing

来源：评论

学校读者我要写书评

暂无评论

TF-BERT: Tensor-based fusion BERT for multimodal sentiment analysis

引用

NEURAL NETWORKS 2025年 185卷 107222页

作者： Hou, Jingming Omar, Nazlia Tiun, Sabrina Saad, Saidah He, Qian Univ Kebangsaan Malaysia Fac Informat Sci & Technol Ctr Artificial Intelligence Technol Bangi 43600 Selangor Malaysia Guilin Univ Elect Technol State & Local Joint Engn Res Ctr Satellite Nav & L Guangxi Collaborat Innovat Ctr Cloud Comp & Big Da Guangxi Key Lab Cryptog & Informat Secur Guilin Peoples R China

Multimodal Sentiment Analysis (MSA) has gained significant attention due to the limitations of unimodal sentiment recognition in complex real-world applications. Traditional approaches typically focus on using the Transformer for fusion. However, these traditional approaches often fall short because the Transformer can only process two modalities simultaneously, leading to insufficient information exchange and potential loss of emotional data. To address the limitation of traditional Crossmodal Transformer models, which can only process two modalities at a time, we propose a novel Tensor-based Fusion BERT model (TF-BERT). The core of TF-BERT is the Tensor-based Crossmodal Fusion (TCF) module, which seamlessly integrates into the pre- trained BERT language model. By embedding the TCF module into multiple layers of BERT's Transformer, we progressively achieve dynamic complementation between different modalities. Additionally, we designed the Tensor-based Crossmodal Transformer (TCT) module, which introduces a tensor-based Transformer mechanism capable of simultaneously processing three different modalities. This allows for comprehensive information exchange between the target modality and the other two source modalities, thus strengthening the representation of the target modality. The TCT overcomes the limitation of existing the Crossmodal Transformer structures, which can only handle relationships between two modalities. Furthermore, to validate the effectiveness of TF-BERT, we conducted extensive experiments on the CMU-MOSI and CMU-MOSEI datasets. TF-BERT not only achieved the best results across most metrics compared to recent state-of-the-art baselines, but also demonstrated the effectiveness of its two modules through ablation studies. The findings suggest that TFBERT effectively addresses the limitations of previous models by progressively integrating and simultaneously capturing complex emotional interactions across all modalities.

关键词： Multimodal sentiment analysis pre-trained language model Modality interactions Transformer

来源：评论

学校读者我要写书评

暂无评论

Parameter-efficient online knowledge distillation for pretrained language models

引用

EXPERT SYSTEMS WITH APPLICATIONS 2025年 265卷

作者： Wang, Yukun Wang, Jin Zhang, Xuejie Yunnan Univ Sch Informat Sci & Engn Kunming Peoples R China

With the advancement of natural language processing (NLP), the size of datasets and pre-trained language models (PLMs) has exponentially grown. These vast models exhibit robust capabilities in generation, comprehension, and multimodal processing. However, the complexity of parameters poses challenges for real-time and edge computing applications. Knowledge distillation (KD) has emerged as a common approach to address this issue by transferring prior knowledge from a large teacher to a smaller student model. previous KD methods relied on offline full-parameter fine-tuning distillation, where the primary objective of the teacher model was to optimize downstream tasks, often neglecting the focus on KD itself. Such inefficient KD methods limit the potential of both teacher and student models. This study introduces a parameter-efficient online distillation method, PEKD, to tackle these challenges. In contrast to previous KD approaches, PEKD streamlines the distillation process into a single training stage and leverages parallel adapters to enhance fine-tuning efficiency. The teacher and student models, utilizing only 0.5% of the parameters on downstream tasks, achieve or surpass the accuracy of full fine-tuning methods. Extensive experiments on the GLUE benchmark dataset demonstrate the competitiveness of PEKD. The code for this study is accessible at https://***/Kawlyh/PEKD.

关键词： Online knowledge distillation pre-trained language model Parameter compression

来源：评论

学校读者我要写书评

暂无评论

MCKP: Multi-aspect contextual knowledge-enhanced prompting for conversational recommender systems

引用

INFORMATION SCIENCES 2025年 686卷

作者： Wang, Yulin Zhang, Yihao Zhu, Junlin Li, Yao Zhou, Wei Chongqing Univ Technol Sch Artificial Intelligence Chongqing 400054 Peoples R China Chongqing Univ Sch Big Data & Software Engn Chongqing 400044 Peoples R China

Empowering conversational recommender systems (CRSs) with knowledge facilitates the generation of high-quality human-like recommendation proposals to users. Despite substantial endeavors in developing knowledge-based CRSs, they mainly focus on modeling entity knowledge by introducing external data, which cannot sufficiently contribute to understanding complex contextual semantics. Additionally, the source of knowledge relies on external data, ignoring the effective use of original training data, significantly increasing the cost of building efficient CRSs. To tackle the issues above, we explore how to leverage raw dialogue data for modeling and exploiting contextual knowledge to compensate for existing knowledge's shortcomings. Specifically, we first train knowledge generators by constructing multiple standard hybrid prompt templates and answer spaces, which is conducive to perceiving the reasons behind contextual connections from multiple aspects and storing corresponding contextual knowledge. Then, we design answer-guided prompt templates based on the answer space, closing the gap between inference and training of knowledge generators and facilitating the generation of aspect-specific contextual knowledge. Next, we extend our approach by harnessing external knowledge graphs to construct entity knowledge. We propose a local-to-global semantic fusion strategy to fuse knowledge representations from different sources better. Finally, based on the fused knowledge representations, we tailor contextual knowledge-enhanced prompt templates for the CRS to stimulate the reasoning ability of pre-trained language models on dialogue and recommendation tasks. We evaluate the performance of our method on separate evaluation datasets, including the ReDial and the INSPIRED datasets. Based on the Recall, MRR, NDCG, and Distinct evaluation indicators, we demonstrate the effectiveness of the proposed method in dialogue tasks and recommendation tasks. At the same time, we demonstrate t

关键词： Conversational recommendation system Recommendation system pre-trained language model Prompt learning

来源：评论

学校读者我要写书评

暂无评论

Leveraging sensory knowledge into Text-to-Text Transfer Transformer for enhanced emotion analysis

引用

INFORMATION PROCESSING & MANAGEMENT 2025年第1期62卷

作者： Zhao, Qingqing Xia, Yuhan Long, Yunfei Xu, Ge Wang, Jia Chinese Acad Social Sci Inst Linguist Beijing 100732 Peoples R China Univ Essex Sch Comp Sci & Elect Engn Colchester CO4 3SQ Essex England Minjiang Univ Coll Comp & Control Engn Fuzhou 350108 Peoples R China Xian Jiaotong Liverpool Univ Dept Intelligent Sci Suzhou 215123 Peoples R China

This study proposes an innovative model (i.e., SensoryT5), which integrates sensory knowledge into the T5 (Text-to-Text Transfer Transformer) framework for emotion classification tasks. By embedding sensory knowledge within the T5 model's attention mechanism, SensoryT5 not only enhances the model's contextual understanding but also elevates its sensitivity to the nuanced interplay between sensory information and emotional states. Experiments on four emotion classification datasets, three sarcasm classification datasets one subjectivity analysis dataset, and one opinion classification dataset (ranging from binary to 32-class tasks) demonstrate that our model outperforms state-of-the-art baseline models (including the baseline T5 model) significantly. Specifically, SensoryT5 achieves a maximal improvement of 3.0% in both the accuracy and the F1 score for emotion classification. In sarcasm classification tasks, the model surpasses the baseline models by the maximal increase of 1.2% in accuracy and 1.1% in the F1 score. Furthermore, SensoryT5 continues to demonstrate its superior performances for both subjectivity analysis and opinion classification, with increases in ACC and the F1 score by 0.6% for the subjectivity analysis task and increases in ACC by 0.4% and the F1 score by 0.6% for the opinion classification task, when compared to the second-best models. These improvements underscore the significant potential of leveraging cognitive resources to deepen NLP models' comprehension of emotional nuances and suggest an interdisciplinary research between the areas of NLP and neuro-cognitive science.

关键词： Emotion analysis Sensory knowledge Attention mechanism pre-trained language model

来源：评论

学校读者我要写书评

暂无评论

PRIME, a temperature-guided language model revolutionizes protein engineering

引用

Acta Pharmaceutica Sinica B 2025年

作者： Yuanxi Yu Qianhui Wang Yike Zou Zhangjiang Institute for Advanced Study Shanghai Jiao Tong University Shanghai 201203 China School of Pharmaceutical Sciences Shanghai Jiao Tong University Shanghai 200240 China

来源：评论

学校读者我要写书评

暂无评论

Do pretrained language models Indeed Understand Software Engineering Tasks?

引用

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING 2023年第10期49卷 4639-4655页

作者： Li, Yao Zhang, Tao Luo, Xiapu Cai, Haipeng Fang, Sen Yuan, Dawei Macau Univ Sci & Technol Sch Comp Sci & Engn Macau 999078 Peoples R China Hong Kong Polytech Univ Dept Comp Hong Kong 999077 Peoples R China Washington State Univ Sch Elect Engn & Comp Sci Pullman WA 99163 USA

Artificial intelligence (AI) for software engineering (SE) tasks has recently achieved promising performance. In this article, we investigate to what extent the pre-trained language model truly understands those SE tasks such as code search, code summarization, etc. We conduct a comprehensive empirical study on a board set of AI for SE (AI4SE) tasks by feeding them with variant inputs: 1) with various masking rates and 2) with sufficient input subset method. Then, the trained models are evaluated on different SE tasks, including code search, code summarization, and duplicate bug report detection. Our experimental results show that pre-trained language models are insensitive to the given input, thus they achieve similar performance in these three SE tasks. We refer to this phenomenon as overinterpretation, where a model confidently makes a decision without salient features, or where a model finds some irrelevant relationships between the final decision and the dataset. Our study investigates two approaches to mitigate the overinterpretation phenomenon: whole word mask strategy and ensembling. To the best of our knowledge, we are the first to reveal this overinterpretation phenomenon to the AI4SE community, which is an important reminder for researchers to design the input for the models and calls for necessary future work in understanding and implementing AI4SE tasks.

关键词： Overinterpretation deep learning pre-trained language model software engineering

来源：评论

学校读者我要写书评

暂无评论

SETEM: Self-ensemble training with pre-trained language models for Entity Matching

引用

KNOWLEDGE-BASED SYSTEMS 2024年 293卷

作者： Ding, Huahua Dai, Chaofan Wu, Yahui Ma, Wubin Zhou, Haohao Natl Univ Def Technol Sci & Technol Informat Syst Engn Lab Changsha 410073 Peoples R China

Entity Matching (EM) aims to determine whether records in two datasets refer to the same real -world entity. Existing work often uses pre -trained language models (PLMs) for feature representation, converting EM to a binary classification task. However, due to the dependence of PLMs on large labeled datasets and the overlap between train and test sets in current EM benchmarks, these methods often underperform in realworld scenarios (e.g., small data size, hard negative samples, and unseen entities). To address the limitations of existing methods, we propose SETEM, a self -ensemble training method leveraging the stability and strong generalization of ensemble models to tackle these challenges in real -world scenarios. Additionally, we develop a faster training method for low -resource applications. Experiments on benchmark datasets show that SETEM outperforms Ditto and HierGAT on the F 1 score. In particular, SETEM shows the greatest advantage with small datasets and a high proportion of unseen entities in the test set, achieving up to a 9.61% F 1 score increment over baselines on the WDC dataset.

关键词： Entity Matching pre-trained language model Self-ensemble Knowledge distillation

来源：评论

学校读者我要写书评

暂无评论

On the Effectiveness of pre-trained language models for Legal Natural language Processing: An Empirical Study

引用

IEEE ACCESS 2022年 10卷 75835-75858页

作者： Song, Dezhao Gao, Sally He, Baosheng Schilder, Frank Thomson Reuters Eagan MN 55123 USA Thomson Reuters New York NY 10036 USA Meta Platforms Inc Menlo Pk CA 94025 USA

We present the first comprehensive empirical evaluation of pre-trained language models (PLMs) for legal natural language processing (NLP) in order to examine their effectiveness in this domain. Our study covers eight representative and challenging legal datasets, ranging from 900 to 57K samples, across five NLP tasks: binary classification, multi-label classification, multiple choice question answering, summarization and information retrieval. We first run unsupervised, classical machine learning and/or non-PLM based deep learning methods on these datasets, and show that baseline systems' performance can be 4%similar to 35% lower than that of PLM-based methods. Next, we compare general-domain PLMs and those specifically pre-trained for the legal domain, and find that domain-specific PLMs demonstrate 1%similar to 5% higher performance than general-domain models, but only when the datasets are extremely close to the pre-training corpora. Finally, we evaluate six general-domain state-of-the-art systems, and show that they have limited generalizability to legal data, with performance gains from 0.1% to 1.2% over other PLM-based methods. Our experiments suggest that both general-domain and domain-specific PLM-based methods generally achieve better results than simpler methods on most tasks, with the exception of the retrieval task, where the best-performing baseline outperformed all PLM-based methods by at least 5%. Our findings can help legal NLP practitioners choose the appropriate methods for different tasks, and also shed light on potential future directions for legal NLP research.

关键词： Legal natural language processing pre-trained language model deep learning machine learning

来源：评论

学校读者我要写书评

暂无评论

Semantic Importance-Aware Communications Using pre-trained language models

引用

IEEE COMMUNICATIONS LETTERS 2023年第9期27卷 2328-2332页

作者： Guo, Shuaishuai Wang, Yanhu Li, Shujing Saeed, Nasir Shandong Univ Sch Control Sci & Engn Jinan 250061 Peoples R China Shandong Univ Shandong Key Lab Wireless Commun Technol Jinan 250061 Peoples R China United Arab Emirates Univ UAEU Dept Elect & Commun Engn Al Ain U Arab Emirates

This letter proposes a semantic importance-aware communication (SIAC) scheme using pre-trained language models (e.g., ChatGPT, BERT, etc.). Specifically, we propose a cross-layer design with a pre-trained language model embedded in/connected by the cross-layer manager. The pre-trained language model is utilized to quantify the semantic importance of data frames. Based on the quantified semantic importance, we investigate semantic importance-aware power allocation. Unlike existing deep joint source-channel coding (Deep-JSCC)-based semantic communication schemes, SIAC can be directly embedded into current communication systems by only introducing a cross-layer manager. Our experimental results show that the proposed SIAC scheme can achieve lower semantic loss than existing equal-priority communications.

关键词： Semantics Chatbots Bit error rate Licenses Cross layer design Resource management Communication systems Semantic communications pre-trained language model power allocation data importance

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：