检索结果-内蒙古大学图书馆

19th International Conference on Advanced Intelligent Computing Technology and Applications (ICIC)

作者： Meng, Fan-Jun He, Ji-Fei Xu, Xing-Jian Zhao, Ya-Juan Sun, Li-Jun Inner Mongolia Normal Univ Coll Comp Sci & Technol Hohhot 010022 Peoples R China

ISBN: (纸本)9789819947515;9789819947522

Multi-choice Machine Reading Comprehension (MRC) task involves selecting the correct answer from a limited set of options given a passage and a question. MRCtasks have experienced two main peaks: the explosion of deep neural networks and the evolution of contextual language models. However, the traditional paradigm suffers from inconsistency between the pre-trained and fine-tuning phases of the task. pre-trained language models (PrLMs) are limited to a fixed input length, which can introduce noisy information irrelevant to the question. To address these issues, we propose a MacBERT-based Chinese MRC model that discards the traditional mask and adopts a similar word approach. This approach provides better contextual information and solves the problem of multiple meanings of words, better fitting the nature of the MRC task. Additionally, an unsupervised algorithm is used to filter passage sentences, controlling the model input and solving the problem of too much irrelevant content in long reading materials. We demonstrate that the proposed methods achieve better results compared to the baseline model on the Chinese MRC dataset C3.

关键词： Chinese machine reading comprehension pre-trained language model Deep neural network

来源：评论

学校读者我要写书评

暂无评论

pre-trained language model based Ranking in Baidu Search 21

Pre-trained Language Model based Ranking in Baidu Search

引用

27th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD)

作者： Zou, Lixin Zhang, Shengqiang Cai, Hengyi Ma, Dehong Cheng, Suqi Wang, Shuaiqiang Shi, Daiting Cheng, Zhicong Yin, Dawei Baidu Inc Beijing Peoples R China

ISBN: (纸本)9781450383325

As the heart of a search engine, the ranking system plays a crucial role in satisfying users' information demands. More recently, neural rankers fine-tuned from pre-trained language models (PLMs) establish state-of-the-art ranking effectiveness. However, it is non-trivial to directly apply these PLM-based rankers to the large-scale web search system due to the following challenging issues: (1) the prohibitively expensive computations of massive neural PLMs, especially for long texts in the web-document, prohibit their deployments in an online ranking system that demands extremely low latency;(2) the discrepancy between existing ranking-agnostic pre-training objectives and the ad-hoc retrieval scenarios that demand comprehensive relevance modeling is another main barrier for improving the online ranking system;(3) a real-world search engine typically involves a committee of ranking components, and thus the compatibility of the individually fine-tuned ranking model is critical for a cooperative ranking system. In this work, we contribute a series of successfully applied techniques in tackling these exposed issues when deploying the stateof-the-art Chinese pre-trained language model, i.e., ERNIE, in the online search engine system. We first articulate a novel practice to cost-efficiently summarize the web document and contextualize the resultant summary content with the query using a cheap yet powerful Pyramid-ERNIE architecture. Then we endow an innovative paradigm to finely exploit the large-scale noisy and biased post-click behavioral data for relevance-oriented pre-training. We also propose a human-anchored fine-tuning strategy tailored for the online ranking system, aiming to stabilize the ranking signals across various online components. Extensive offline and online experimental results show that the proposed techniques significantly boost the search engine's performance.

关键词： pre-trained language model Learning to Rank

来源：评论

学校读者我要写书评

暂无评论

A knowledge extraction framework for domain-specific application with simplified pre-trained language model and attention-based feature extractor

引用

SERVICE ORIENTED COMPUTING AND APPLICATIONS 2022年第2期16卷 121-131页

作者： Zhang, Jian Qin, Bo Zhang, Yufei Zhou, Junhua Wang, Hongwei Zhejiang Univ ZJU UIUC Inst Haining 314400 Zhejiang Peoples R China Beijing Inst Elect Syst Engn Beijing Simulat Ctr Beijing 100000 Peoples R China

With the advancement of industrial informatics, intelligent algorithms are increasingly applied in various industrial products and applications. In this paper, we proposed a knowledge extraction framework for domain-specific text. This framework can extract entities from text the subsequent tasks such as knowledge graph construction. The proposed framework contains three modules, namely domain feature pre-trained model, LSTM-based named entity recognition and the attention-based nested named entity recognition. The domain feature pre-trained model can effectively learn the features of domain corpus such as professional terms that are not included in the general domain corpus. Flat named entity recognition can use the vector from pre-trained model to obtain the entity from domain-specific text. The nested named entity recognition based on the attention mechanism and the weight sliding balance strategy can effectively identify entity types with higher nesting rates. The framework achieves good results in the field of nuclear power plant maintenance reports, and the methods for domain pre-trained model and LSTM-based flat named entity recognition have been successfully applied to practical tasks.

关键词： Knowledge extraction Named entity recognition pre-trained language model Attention mechanism

来源：评论

学校读者我要写书评

暂无评论

A pre-trained language model for emergency department intervention prediction using routine physiological data and clinical narratives

引用

INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS 2024年 191卷 105564页

作者： Huang, Ting-Yun Chong, Chee-Fah Lin, Heng-Yu Chen, Tzu-Ying Chang, Yung-Chun Lin, Ming-Chin Taipei Med Univ Shuang Ho Hosp Emergency Dept Taipei Taiwan Shin Kong Wu Ho Su Mem Hosp Emergency Dept Taipei Taiwan Taipei Med Univ Grad Inst Data Sci Taipei Taiwan Taipei Med Univ Hosp Clin Big Data Res Ctr Taipei Taiwan Taipei Med Univ Grad Inst Biomed Informat Taipei Taiwan Taipei Med Univ Shuang Ho Hosp Dept Neurosurg Taipei Taiwan Taipei Med Univ Dept Neurosurg Taipei Municipal Wanfang Hosp Taipei Taiwan

Introduction: The urgency and complexity of emergency room (ER) settings require precise and swift decision- making processes for patient care. Ensuring the timely execution of critical examinations and interventions is vital for reducing diagnostic errors, but the literature highlights a need for innovative approaches to optimize diagnostic accuracy and patient outcomes. In response, our study endeavors to create predictive models for timely examinations and interventions by leveraging the patient's symptoms and vital signs recorded during triage, and in so doing, augment traditional diagnostic methodologies. Methods: Focusing on four key areas-medication dispensing, vital interventions, laboratory testing, and emergency radiology exams, the study employed Natural language Processing (NLP) and seven advanced machine learning techniques. The research was centered around the innovative use of BioClinicalBERT, a state-of-the-art NLP framework. Results: BioClinicalBERT emerged as the superior model, outperforming others in predictive accuracy. The integration of physiological data with patient narrative symptoms demonstrated greater effectiveness compared to models based solely on textual data. The robustness of our approach was confirmed by an Area Under the Receiver Operating Characteristic curve (AUROC) score of 0.9. Conclusion: The findings of our study underscore the feasibility of establishing a decision support system for emergency patients, targeting timely interventions and examinations based on a nuanced analysis of symptoms. By using an advanced natural language processing technique, our approach shows promise for enhancing diagnostic accuracy. However, the current model is not yet fully mature for direct implementation into daily clinical practice. Recognizing the imperative nature of precision in the ER environment, future research endeavors must focus on refining and expanding predictive models to include detailed timely examinations and interventions. Al

关键词： Natural language processing pre-trained language model Clinical decision support system Critical interventions Clinical narratives

来源：评论

学校读者我要写书评

暂无评论

A Light Bug Triage Framework for Applying Large pre-trained language model 7

A Light Bug Triage Framework for Applying Large Pre-trained ...

引用

37th IEEE/ACM International Conference on Automated Software Engineering (ASE)

作者： Lee, Jaehyung Han, Kisun Yu, Hwanjo Pohang Univ Sci & Technol POSTECH Pohang Gyeongsangbuk D South Korea Samsung Res Seoul South Korea

ISBN: (纸本)9781450394758

Assigning appropriate developers to the bugs is one of the main challenges in bug triage. Demands for automatic bug triage are increasing in the industry, as manual bug triage is labor-intensive and time-consuming in large projects. The key to the bug triage task is extracting semantic information from a bug report. In recent years, large pre-trained language models (PLMs) including BERT [4] have achieved dramatic progress in the natural language processing (NLP) domain. However, applying large PLMs to the bug triage task for extracting semantic information has several challenges. In this paper, we address the challenges and propose a novel framework for bug triage named LBT-P, standing for Light Bug Triage framework with a pre-trained language model. It compresses a large PLM into small and fast models using knowledge distillation techniques and also prevents catastrophic forgetting of PLM by introducing knowledge preservation fine-tuning. We also develop a new loss function exploiting representations of earlier layers as well as deeper layers in order to handle the overthinking problem. We demonstrate our proposed framework on the real-world private dataset and three public real-world datasets [11]: Google Chromium, Mozilla Core, and Mozilla Firefox. The result of the experiments shows the superiority of LBT-P.

关键词： Bug triage pre-trained language model BERT Knowledge distillation Catastrophic forgetting Overthinking

来源：评论

学校读者我要写书评

暂无评论

pre-trained language model augmented adversarial training network for Chinese clinical event detection

引用

MATHEMATICAL BIOSCIENCES AND ENGINEERING 2020年第4期17卷 2825-2841页

作者： Zhang, Zhichang Zhang, Minyu Zhou, Tong Qiu, Yanlong Northwest Normal Univ Coll Comp Sci & Engn 967 Anning East Rd Lanzhou 730070 Peoples R China

Clinical event detection (CED) is a hot topic and essential task in medical artificial intelligence, which has attracted the attention from academia and industry over the recent years. However, most studies focus on English clinical narratives. Owing to the limitation of annotated Chinese medical corpus, there is a lack of relevant research about Chinese clinical narratives. The existing methods ignore the importance of contextual information in semantic understanding. Therefore, it is urgent to research multilingual clinical event detection. In this paper, we present a novel encoder-decoder structure based on pre-trained language model for Chinese CED task, which integrates contextual representations into Chinese character embeddings to assist model in semantic understanding. Compared with existing methods, our proposed strategy can help model harvest a language inferential skill. Besides, we introduce the punitive weight to adjust the proportion of loss on each category for coping with class imbalance problem. To evaluate the effectiveness of our proposed model, we conduct a range of experiments on test set of our manually annotated corpus. We compare overall performance of our proposed model with baseline models on our manually annotated corpus. Experimental results demonstrate that our proposed model achieves the best precision of 83.73%, recall of 86.56% and F1-score of 85.12%. Moreover, we also evaluate the performance of our proposed model with baseline models on minority category samples. We discover that our proposed model obtains a significant increase on minority category samples.

关键词： medical artificial intelligence Chinese clinical narratives Chinese clinical event detection semantic understanding class imbalance problem transfer learning pre-trained language model adversarial training network

来源：评论

学校读者我要写书评

暂无评论

Japanese ASR-Robust pre-trained language model with Pseudo-Error Sentences Generated by Grapheme-Phoneme Conversion 23

Japanese ASR-Robust Pre-trained Language Model with Pseudo-E...

引用

Interspeech Conference

作者： Ohsugi, Yasuhito Saito, Itsumi Nishida, Kyosuke Yoshida, Sen NTT Corp NTT Human Informat Labs Tokyo Japan

Spoken language understanding systems typically consist of a pipeline of automatic speech recognition (ASR) and natural language processing (NLP) modules. Although pre-trained language models (PLMs) have been successful in NLP by training on large corpora of written texts;spoken language with serious ASR errors that change its meaning is difficult to understand. We propose a method for pre-training Japanese LMs robust against ASR errors without using ASR. With the proposed method using written texts, sentences containing pseudo-ASR errors are generated using a pseudo-error dictionary constructed using grapheme-to-phoneme and phoneme-to-grapheme models based on neural networks. Experiments on spoken dialogue summarization showed that the ASR-robust LM pre-trained with the proposed method outperformed the LM pre-trained with standard masked language modeling by 3.17 points on ROUGE-L when fine-tuning with dialogues including ASR errors.

关键词： pre-trained language model ASR-robust spoken dialogue summarization

来源：评论

学校读者我要写书评

暂无评论

Chinese-Korean Weibo Sentiment Classification Based on pre-trained language model and Transfer Learning 2

Chinese-Korean Weibo Sentiment Classification Based on Pre-t...

引用

IEEE 2nd International Conference on Computer Communication and Artificial Intelligence (CCAI)

作者： Wang, Hengxuan Zhang, Zhenguo Cui, Xu Cui, Rongyi Yanbian Univ Dept Comp Sci & Technol 977 Gongyuan Rd Yanji 133002 Peoples R China

ISBN: (纸本)9781665496636

Korean is the native and official language spoken by Chinese-Korean people, and Weibo is a social media software with a huge number of users in China. Currently, there is few studies related to sentiment analysis of Korean-language Weibo texts posted by Chinese-Korean users. In this paper, we propose a sentiment classification method for Chinese-Korean Weibo based on pre-trained language model and transfer learning. Firstly, we crawled the Chinese-Korean Weibo data from Sina Weibo and label them with sentiment to get the Chinese-Korean Weibo sentiment analysis (CKWSA) dataset. Secondly, to solve the problem of few training samples of the Chinese-Korean Weibo sentiment analysis dataset, we fine-tune the classifier based on the pre-trained Korean language model on the Korean Twitter sentiment analysis dataset to obtain the Korean Twitter sentiment classification model;and further fine-tune the model on CKWSA dataset to get Chinese-Korean Weibo sentiment classification model. The experiments show that the proposed classification method based on pre-trained language model and transfer learning has great performance,and there is an improvement compared other baselines on the Chinese-Korean Weibo sentiment analysis dataset.

关键词： Weibo sentiment classification pre-trained language model Transfer learning Korean natural language processing

来源：评论

学校读者我要写书评

暂无评论

MEM-KGC: Masked Entity model for Knowledge Graph Completion With pre-trained language model

引用

IEEE ACCESS 2021年 9卷 132025-132032页

作者： Choi, Bonggeun Jang, Daesik Ko, Youngjoong Sungkyunkwan Univ Dept Elect & Comp Engn Suwon 16419 Gyeonggi Do South Korea Sungkyunkwan Univ Dept Comp Sci & Engn Suwon 16419 Gyeonggi Do South Korea

The knowledge graph completion (KGC) task aims to predict missing links in knowledge graphs. Recently, several KGC models based on translational distance or semantic matching methods have been proposed and have achieved meaningful results. However, existing models have a significant shortcoming-they cannot train entity embedding when an entity does not appear in the training phase. As a result, such models use randomly initialized embeddings for entities that are unseen in the training phase and cause a critical decrease in performance during the test phase. To solve this problem, we propose a new approach that performs KGC task by utilizing the masked language model (MLM) that is used for a pre-trained language model. Given a triple (head entity, relation, tail entity), we mask the tail entity and consider the head entity and the relation as a context for the tail entity. The model then predicts the masked entity from among all entities. Then, the task is conducted by the same process as an MLM, which predicts a masked token with a given context of tokens. Our experimental results show that the proposed model achieves significantly improved performances when unseen entities appear during the test phase and achieves state-of-the-art performance on the WN18RR dataset.

关键词： Task analysis predictive models Training Bit error rate Semantics Micromechanical devices Licenses Knowledge graph completion link prediction masked language model pre-trained language model

来源：评论

学校读者我要写书评

暂无评论

CLIP-Llama: A New Approach for Scene Text Recognition with a pre-trained Vision-language model and a pre-trained language model

引用

SENSORS 2024年第22期24卷 7371页

作者： Zhao, Xiaoqing Xu, Miaomiao Silamu, Wushour Li, Yanbing Xinjiang Univ Coll Comp Sci & Technol 777 Huarui St Urumqi 830017 Peoples R China

This study focuses on Scene Text Recognition (STR), which plays a crucial role in various applications of artificial intelligence such as image retrieval, office automation, and intelligent transportation systems. Currently, pre-trained vision-language models have become the foundation for various downstream tasks. CLIP exhibits robustness in recognizing both regular (horizontal) and irregular (rotated, curved, blurred, or occluded) text in natural images. As research in scene text recognition requires substantial linguistic knowledge, we introduce the pre-trained vision-language model CLIP and the pre-trained language model Llama. Our approach builds upon CLIP's image and text encoders, featuring two encoder-decoder branches: one visual branch and one cross-modal branch. The visual branch provides initial predictions based on image features, while the cross-modal branch refines these predictions by addressing the differences between image features and textual semantics. We incorporate the large language model Llama2-7B in the cross-modal branch to assist in correcting erroneous predictions generated by the decoder. To fully leverage the potential of both branches, we employ a dual prediction and refinement decoding scheme during inference, resulting in improved accuracy. Experimental results demonstrate that CLIP-Llama achieves state-of-the-art performance on 11 STR benchmark tests, showcasing its robust capabilities. We firmly believe that CLIP-Llama lays a solid and straightforward foundation for future research in scene text recognition based on vision-language models.

关键词： scene text recognition vision-language model pre-trained language model

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：