检索结果-内蒙古大学图书馆

P-BERT: Toward Long sequence Modeling by Enabling Language Representation With prefix sequence compression, Soft Position Embedding, and Data Augmentation for Patent Relevance Assessment

引用

IEEE ACCESS 2025年 13卷 41928-41942页

作者： Wang, Fei Shi, Xingchen Wang, Dongsheng Lou, Yinxia Jiangsu Univ Sci & Technol Sch Comp Sci Zhenjiang 212003 Peoples R China Jianghan Univ Sch Artificial Intelligence Wuhan 430056 Peoples R China

Recent works have increasingly adopted pre-trained language models, such as BERT, to model technical semantics for patent relevance assessment. However, existing truncation and divide-merge strategies, used to handle input length constraints, results in feature loss and semantic isolation. This limits the effectiveness of pre-trained language models in capturing long sequence semantics for patent relevance assessment. Because patent relevance is evaluated based on local sequence semantics, this paper proposes a patent-oriented bidirectional encoder representation from transformers (P-BERT) to enable the semantic modeling ability of long sequences. The characteristics of P-BERT lie in a prefix sequence compression method, a soft position embedding method and a data augmentation method. The first one can retrain more technical features within the input constraints via transforming full sequences into partial sequences while the second one reconstructs local sequence semantics for partial sequences. As for the last one, it contains sample and feature augmentation to improve the generalization and robustness. Due to the scarcity of training corpus for patent relevance assessment, a dual constraint strategy is proposed to generate a supervised corpus, DualCons. Experimental results demonstrate that DualCons can help BERT obtain the best ranking performance compared with other strategies. Model evaluations show the effectiveness of P-BERT. It reaches an improvement of 2%-6% in recall-oriented metrics and an improvement of 1%-5% in accuracy-oriented indicators over the state-of-the-art baselines. All those would promote the engineering applications of pre-trained language models in patent relevance assessment.

关键词： prefix sequence compression soft position embedding data augmentation patent relevance assessment prefix sequence compression soft position embedding data augmentation patent relevance assessment

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：