检索结果-内蒙古大学图书馆

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering

作者： Fang Liu Ge Li Yunfei Zhao Zhi Jin Peking University Beijing China

ISBN: (纸本)9781450367684

Code completion is one of the most useful features in the Integrated Development Environments (IDEs), which can accelerate software development by suggesting the next probable token based on the contextual code in real-time. Recent studies have shown that statistical language modeling techniques can improve the performance of code completion tools through learning from large-scale software repositories. However, these models suffer from two major drawbacks: a) Existing research uses static embeddings, which map a word to the same vector regardless of its context. The differences in the meaning of a token in varying contexts are lost when each token is associated with a single representation; b) Existing language model based code completion models perform poor on completing identifiers, and the type information of the identifiers is ignored in most of these models. To address these challenges, in this paper, we develop a multi-task learning based pre-trained language model for code understanding and code generation with a Transformer-based neural architecture. We pre-train it with hybrid objective functions that incorporate both code understanding and code generation tasks. Then we fine-tune the pre-trained model on code completion. During the completion, our model does not directly predict the next token. Instead, we adopt multi-task learning to predict the token and its type jointly and utilize the predicted type to assist the token prediction. Experiments results on two real-world datasets demonstrate the effectiveness of our model when compared with state-of-the-art methods.

关键词： transformer networks code completion multi-task learning pre-trained language model

来源：评论

学校读者我要写书评

暂无评论

Enhancing Prosodic Features by Adopting pre-trained language model in Bahasa Indonesia Speech Synthesis

Enhancing Prosodic Features by Adopting Pre-trained Language...

引用

作者： Lixuan Zhao Jian Yang Qinglai Qin School of Information Science and Engineering Yunnan University

Deep neural network text-to-speech(TTS) systems can produce high-quality audio. However, modern TTS systems usually need a sizable of studio-quality

关键词： Indonesia speech synthesis End-to-End pre-trained language model Prosody

来源：评论

学校读者我要写书评

暂无评论

A Light Bug Triage Framework for Applying Large pre-trained language model 22

A Light Bug Triage Framework for Applying Large Pre-trained ...

引用

Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering

作者： Jaehyung Lee Kisun Han Hwanjo Yu The Department of Computer Science and Engineering Pohang University of Science and Technology Republic of Korea Samsung Research Republic of Korea

ISBN: (纸本)9781450394758

Assigning appropriate developers to the bugs is one of the main challenges in bug triage. Demands for automatic bug triage are increasing in the industry, as manual bug triage is labor-intensive and time-consuming in large projects. The key to the bug triage task is extracting semantic information from a bug report. In recent years, large pre-trained language models (PLMs) including BERT [4] have achieved dramatic progress in the natural language processing (NLP) domain. However, applying large PLMs to the bug triage task for extracting semantic information has several challenges. In this paper, we address the challenges and propose a novel framework for bug triage named LBT-P, standing for Light Bug Triage framework with a pre-trained language model. It compresses a large PLM into small and fast models using knowledge distillation techniques and also prevents catastrophic forgetting of PLM by introducing knowledge preservation fine-tuning. We also develop a new loss function exploiting representations of earlier layers as well as deeper layers in order to handle the overthinking problem. We demonstrate our proposed framework on the real-world private dataset and three public real-world datasets [11]: Google Chromium, Mozilla Core, and Mozilla Firefox. The result of the experiments shows the superiority of LBT-P.

关键词： pre-trained language model BERT Catastrophic forgetting Bug triage Knowledge distillation Overthinking

来源：评论

学校读者我要写书评

暂无评论

A Person-job Matching Method Based on BM25 and pre-trained language model 23

A Person-job Matching Method Based on BM25 and Pre-trained L...

引用

Proceedings of the 2023 6th International Conference on Machine Learning and Natural language Processing

作者： Jiaqi Tang Heyun Chen Zifan Chen Jiahua Pan Yijiang He Jie Zhao Zhen Li PKU-Changsha Institute for Computing and Digital Economy China Center for Data Science Peking University China National Engineering Laboratory for Big Data Analysis and Applications Peking University China

ISBN: (纸本)9798400709241

To address the inefficiency of traditional methods for person-job matching and the lack of interpretability of deep learning approaches, a novel approach for person-job matching based on BM25 and pre-trained language models is proposed. First, the BM25 algorithm is utilized to perform an initial screening of resumes based on job descriptions. By calculating the matching degree of keywords, it rapidly filters out candidate resumes that do not align with the job requirements. Subsequently, the pre-trained language model BERT is employed to perform fine-grained sorting of the resumes that passed the initial screening, identifying the candidates that best match with the job. By combining the screening capability of the BM25 algorithm with the semantic understanding ability of pre-trained language models, this method enhances the accuracy and interpretability of person-job matching results while maintaining matching speed. Furthermore, to improve model generalization, large-scale language models are employed for data augmentation. Experimental results indicate the significant potential for the proposed approach in practical applications, offering improved solutions for the challenges of talent recruitment across diverse job demands.

关键词： BM25 data augmentation large language model person-job matching pre-trained language model

来源：评论

学校读者我要写书评

暂无评论

Integrating pre-trained protein language model and multiple window scanning deep learning networks for accurate identification of secondary active transporters in membrane proteins

引用

METHODS 2023年 220卷 11-20页

作者： Malik, Muhammad Shahid Ou, Yu-Yen Yuan Ze Univ Dept Comp Sci & Engn Chungli 32003 Taiwan Yuan Ze Univ Grad Program Biomed Informat Chungli 32003 Taiwan Karakoram Int Univ Dept Comp Sci Gilgit Baltistan Pakistan

Secondary active transporters play pivotal roles in regulating ion and molecule transport across cell membranes, with implications in diseases like cancer. However, studying transporters via biochemical experiments poses challenges. We propose an effective computational approach to identify secondary active transporters from membrane protein sequences using pre-trained language models and deep learning neural *** dataset comprised 290 secondary active transporters and 5,420 other membrane proteins from UniProt. Three types of features were extracted - one-hot encodings, position-specific scoring matrix profiles, and contextual embeddings from the ProtTrans language model. A multi-window convolutional neural network architecture scanned the ProtTrans embeddings using varying window sizes to capture multi-scale sequence *** proposed model combining ProtTrans embeddings and multi-window convolutional neural networks achieved 86% sensitivity, 99% specificity and 98% overall accuracy in identifying secondary active transporters, outperforming conventional machine learning *** work demonstrates the promise of integrating pre-trained language models like ProtTrans with multiscale deep neural networks to effectively interpret transporter sequences for functional analysis. Our approach enables more accurate computational identification of secondary active transporters, advancing membrane protein research.

关键词： Secondary active transporters Multiple window scanning Deep learning pre-trained language model

来源：评论

学校读者我要写书评

暂无评论

Classifying Code Comments via pre-trained Programming language model 2

Classifying Code Comments via Pre-trained Programming Langua...

引用

2nd IEEE/ACM International Workshop on Natural language-Based Software Engineering (NLBSE)

作者： Li, Ying Wang, Haibo Zhang, Huaien Tan, Shin Hwei Southern Univ Sci & Technol Dept Comp Sci & Engn Res Inst Trustworthy Autonomous Syst Shenzhen Peoples R China Southern Univ Sci & Technol Hong Kong Polytech Univ Shenzhen Peoples R China

ISBN: (纸本)9798350301786

previous studies have categorized code comments for various programming languages to produce high-quality code comments that can improve code readability and benefit maintenance. However, it still requires more effort to identify the main information contained in code comments. pre-trained language model has shown promising results for solving software engineering tasks. In this paper, we propose a model for code comment classified using the recent pre-trained language model specialized for code-specific tasks (i.e., CodeT5). We introduce expert-predefined features to enhance the model's classification performance. Our evaluation on the official dataset shows that it outperforms the baseline by improving the precision (+65.9%), recall (+147.3%) and the F1-score (+112.5%) of the classification.

关键词： pre-trained language model code comment

来源：评论

学校读者我要写书评

暂无评论

A pre-trained Clinical language model for Acute Kidney Injury 8

A Pre-trained Clinical Language Model for Acute Kidney Injur...

引用

8th IEEE International Conference on Healthcare Informatics (ICHI)

作者： Mao, Chengsheng Yao, Liang Luo, Yuan Northwestern Univ Dept Prevent Med Chicago IL 60611 USA

ISBN: (纸本)9781728153827

pre-trained contextual language models such as BERT have dramatically improved performances for many NLP tasks recently. However, few have explored BERT on specific medical domain tasks such as early prediction for Acute Kidney Injury (AKI). Since much of the clinical information is contained in clinical notes that are largely unstructured text, in this paper, we present an AKI domain-specific pre-trained language model based on BERT (AKI-BERT) that could be used to mine the clinical notes for AKI early prediction. Our experiments on MIMIC-III dataset demonstrate that AKI-BERT can yield performance improvements for AKI early prediction.

关键词： acute kidney injury pre-trained language model BERT clinical decision support natural language processing

来源：评论

学校读者我要写书评

暂无评论

ConPep: prediction of peptide contact maps with pre-trained biological language model and multi-view feature extracting strategy

引用

COMPUTERS IN BIOLOGY AND MEDICINE 2023年 167卷 107631-107631页

作者： Wei, Qingxin Wang, Ruheng Jiang, Yi Wei, Leyi Sun, Yu Geng, Jie Su, Ran Shandong Univ Sch Software Jinan Peoples R China Shandong Univ Joint SDU NTU Ctr Artificial Intelligence Res C FA Jinan Peoples R China Beidahuang Ind Grp Gen Hosp Harbin Peoples R China Tianjin Chest Hosp Dept Cardiol Tianjin Peoples R China Tianjin Univ Coll Intelligence & Comp Tianjin Peoples R China Macao Polytech Univ Fac Appl Sci Ctr Artificial Intelligence driven Drug Discovery Macau Peoples R China

The accurate prediction of peptide contact maps remains a challenging task due to the difficulty in obtaining the interactive information between residues on short sequences. To address this challenge, we propose ConPep, a deep learning framework designed for predicting the contact map of peptides based on sequences only. To sufficiently incorporate the sequential semantic information between residues in peptide sequences, we use a pretrained biological language model and transfer prior knowledge from large scale databases. Additionally, to extract and integrate sequential local information and residue-based global correlations, our model incorporates Bidirectional Gated Recurrent Unit and attention mechanisms. They can obtain multi-view features and thus enhance the accuracy and robustness of our prediction. Comparative results on independent tests demonstrate that our proposed method significantly outperforms state-of-the-art methods even with short peptides. Notably, our method exhibits superior performance at the sequence level, suggesting the robust ability of our model compared with the multiple sequence alignment (MSA) analysis-based methods. We expect it can be meaningful research for facilitating the wide use of our method.

关键词： Peptide contact map prediction pre-trained language model Multi-view feature fusion

来源：评论

学校读者我要写书评

暂无评论

TextBlockV2: Towards precise-Detection-Free Scene Text Spotting with pre-trained language model

引用

ACM Transactions on Multimedia Computing, Communications, and Applications 1000年

作者： Jiahao Lyu Jin Wei Gangyan Zeng Zeng Li Enze Xie Wei Wang Can Ma Yu Zhou Institute of Information Engineering Chinese Academy of Sciences & School of Cyber Security University of Chinese Academy of Sciences China Lenovo Research China School of Cyber Science and Engineering Nanjing University of Science and Technology China Shanghai Artificial Intelligence Laboratory China Institute of Information Engineering Chinese Academy of Sciences China VCIP & TMCC & DISSec College of Computer Science Nankai University China

Existing scene text spotters are designed to locate and transcribe texts from images. However, it is challenging for a spotter to achieve precise detection and recognition of scene texts simultaneously. Inspired by the glimpse-focus spotting pipeline of human beings and impressive performances of pre-trained language models (PLMs) on visual tasks, we ask: 1) “Can machines spot texts without precise detection just like human beings?”, and if yes, 2)“Is text block another alternative for scene text spotting other than word or character?” To this end, our proposed scene text spotter leverages advanced PLMs to enhance performance without fine-grained detection. Specifically, we first use a simple detector for block-level text detection to obtain rough positional information. Then, we finetune a PLM using a large-scale OCR dataset to achieve accurate recognition. Benefiting from the comprehensive language knowledge gained during the pre-training phase, the PLM-based recognition module effectively handles complex scenarios, including multi-line, reversed, occluded, and incomplete-detection texts. Taking advantage of the fine-tuned language model on scene recognition benchmarks and the paradigm of text block detection, extensive experiments demonstrate the superior performance of our scene text spotter across multiple public benchmarks. Additionally, we attempt to spot texts directly from an entire scene image to demonstrate the potential of PLMs, even Large language models (LLMs).

关键词： Scene Text Spotting pre-trained language model

来源：评论

学校读者我要写书评

暂无评论

pre-trained protein language model sheds new light on the prediction of Arabidopsis protein-protein interactions

引用

PLANT METHODS 2023年第1期19卷 141-141页

作者： Zhou, Kewei Lei, Chenping Zheng, Jingyan Huang, Yan Zhang, Ziding China Agr Univ Coll Biol Sci State Key Lab Anim Biotech Breeding Beijing 100193 Peoples R China

BackgroundProtein-protein interactions (PPIs) are heavily involved in many biological processes. Consequently, the identification of PPIs in the model plant Arabidopsis is of great significance to deeply understand plant growth and development, and then to promote the basic research of crop improvement. Although many experimental Arabidopsis PPIs have been determined currently, the known interactomic data of Arabidopsis is far from complete. In this context, developing effective machine learning models from existing PPI data to predict unknown Arabidopsis PPIs conveniently and rapidly is still urgently *** used a large-scale pre-trained protein language model (pLM) called ESM-1b to convert protein sequences into high-dimensional vectors and then used them as the input of multilayer perceptron (MLP). To avoid the performance overestimation frequently occurring in PPI prediction, we employed stringent datasets to train and evaluate the predictive model. The results showed that the combination of ESM-1b and MLP (i.e., ESMAraPPI) achieved more accurate performance than the predictive models inferred from other pLMs or baseline sequence encoding schemes. In particular, the proposed ESMAraPPI yielded an AUPR value of 0.810 when tested on an independent test set where both proteins in each protein pair are unseen in the training dataset, suggesting its strong generalization and extrapolating ability. Moreover, the proposed ESMAraPPI model performed better than several state-of-the-art generic or plant-specific PPI *** sequence embeddings from the pre-trained model ESM-1b contain rich protein semantic information. By combining with the MLP algorithm, ESM-1b revealed excellent performance in predicting Arabidopsis PPIs. We anticipate that the proposed predictive model (ESMAraPPI) can serve as a very competitive tool to accelerate the identification of Arabidopsis interactome.

关键词： Arabidopsis Protein-protein interactions Machine learning pre-trained language model Natural language processing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：