检索结果-内蒙古大学图书馆

Medical text classification based on the discriminative pre-training model and prompt-tuning

DIGITAL HEALTH 2023年 9卷

作者： Wang, Yu Wang, Yuan Peng, Zhenwan Zhang, Feifan Zhou, Luyao Yang, Fei Anhui Med Univ Sch Biomed Engn Hefei Peoples R China Chinese Acad Sci Hefei Inst Phys Sci Hefei Peoples R China

Medical text classification, as a fundamental medical natural language processing task, aims to identify the categories to which a short medical text belongs. Current research has focused on performing the medical text classification task using a pre-training language model through fine-tuning. However, this paradigm introduces additional parameters when training extra classifiers. Recent studies have shown that the "prompt-tuning" paradigm induces better performance in many natural language processing tasks because it bridges the gap between pre-training goals and downstream tasks. The main idea of prompt-tuning is to transform binary or multi-classification tasks into mask prediction tasks by fully exploiting the features learned by pre-training language models. This study explores, for the first time, how to classify medical texts using a discriminative pre-training language model called ERNIE-Health through prompt-tuning. Specifically, we attempt to perform prompt-tuning based on the multi-token selection task, which is a pre-training task of ERNIE-Health. The raw text is wrapped into a new sequence with a template in which the category label is replaced by a [UNK] token. The model is then trained to calculate the probability distribution of the candidate categories. Our method is tested on the KUAKE-Question Intention Classification and CHiP-Clinical Trial Criterion datasets and obtains the accuracy values of 0.866 and 0.861. In addition, the loss values of our model decrease faster throughout the training period compared to the fine-tuning. The experimental results provide valuable insights to the community and suggest that prompt-tuning can be a promising approach to improve the performance of pre-training models in domain-specific tasks.

关键词： Text classification pre-training language model prompt-tuning ERNIE-Health bidirectional encoder representations from transformers

来源：评论

学校读者我要写书评

暂无评论

Identifying Potential Lyme Disease Cases Using Self-Reported Worldwide Tweets: Deep Learning Modeling Approach Enhanced With Sentimental Words Through Emojis

引用

JOURNAL OF MEDICAL INTERNET RESEARCH 2023年第1期25卷 e47014页

作者： Laison, Elda Kokoe Elolo Ibrahim, Mohamed Hamza Boligarla, Srikanth Li, Jiaxin Mahadevan, Raja Ng, Austen Muthuramalingam, Venkataraman Lee, Wee Yi Yin, Yijun Nasri, Bouchra R. Univ Montreal Ecole St Publ Univ Montreal Dept Med Sociale & Prevent Montreal PQ Canada Zagazig Univ Fac Sci Dept Math Zagazig Egypt Harvard Univ Harvard Extens Sch Cambridge MA 02138 USA Univ Montreal Sch Publ Hlth Dept Social & Prevent Med 7101 Pk Ave Montreal PQ H3N 1X9 Canada

Background: Lyme disease is among the most reported tick-borne diseases worldwide, making it a major ongoing public health concern. An effective Lyme disease case reporting system depends on timely diagnosis and reporting by health care professionals, and accurate laboratory testing and interpretation for clinical diagnosis validation. A lack of these can lead to delayed diagnosis and treatment, which can exacerbate the severity of Lyme disease symptoms. Therefore, there is a need to improve the monitoring of Lyme disease by using other data sources, such as web-based data. Objective: We analyzed global Twitter data to understand its potential and limitations as a tool for Lyme disease surveillance. We propose a transformer-based classification system to identify potential Lyme disease cases using self-reported tweets. Methods: Our initial sample included 20,000 tweets collected worldwide from a database of over 1.3 million Lyme disease tweets. After preprocessing and geolocating tweets, tweets in a subset of the initial sample were manually labeled as potential Lyme disease cases or non-Lyme disease cases using carefully selected keywords. Emojis were converted to sentiment words, which were then replaced in the tweets. This labeled tweet set was used for the training, validation, and performance testing of DistilBERT (distilled version of BERT [bidirectional encoder representations from transformers]), ALBERT (A Lite BERT), and BERTweet (BERT for English Tweets) classifiers. Results: The empirical results showed that BERTweet was the best classifier among all evaluated models (average F1-score of 89.3%, classification accuracy of 90.0%, and precision of 97.1%). However, for recall, term frequency-inverse document frequency and k-nearest neighbors performed better (93.2% and 82.6%, respectively). On using emojis to enrich the tweet embeddings, BERTweet had an increased recall (8% increase), DistilBERT had an increased F1-score of 93.8% (4% increase) and classificat

关键词： Lyme disease Twitter BERT bidirectional encoder representations from transformers emojis machine learning natural language processing

来源：评论

学校读者我要写书评

暂无评论

The Most Effective Interventions for Classification ModelDevelopment to Predict Chat Outcomes Based on theConversation Content in Online Suicide Prevention Chats:MachineLearning Approach

引用

JMIR MENTAL HEALTH 2024年 11卷 e57362页

作者： Salmi, Salim Merel, Saskia Gilisse, Renske van der Mei, Rob Bhulai, Sandjai 113 Suicide Prevent Res Dept Paasheuvelweg 25 NL-1105 BP Amsterdam Netherlands Ctr Wiskunde & Informat Dept Stochast Amsterdam Netherlands Vrije Univ Amsterdam Dept Math Amsterdam Netherlands

Background: For the provision of optimal care in a suicide prevention helpline, it is important to know what contributes topositive or negative effects on help seekers. Helplines can often be contacted through text-based chat services, which producelarge amounts of text data for use in large-scale analysis. Objective: We trained a machine learning classification model to predict chat outcomes based on the content of the chatconversations in suicide helplines and identified the counsellor utterances that had the most impact on its outputs. Methods: from August 2021 until January 2023, help seekers (N=6903) scored themselves on factors known to be associatedwith suicidality (eg, hopelessness, feeling entrapped, will to live) before and after a chat conversation with the suicide preventionhelpline in the Netherlands (113 Suicide Prevention). Machine learning text analysis was used to predict help seeker scores onthese factors. Using 2 approaches for interpreting machine learning models, we identified text messages from helpers in a chatthat contributed the most to the prediction of the model. Results: According to the machine learning model, helpers'positive affirmations and expressing involvement contributed toimproved scores of the help seekers. Use of macros and ending the chat prematurely due to the help seeker being in an unsafesituation had negative effects on help seekers. Conclusions: This study reveals insights for improving helpline chats, emphasizing the value of an evocative style with questions,positive affirmations, and practical advice. It also underscores the potential of machine learning in helpline chat analysi

关键词： suicide suicidality suicide prevention helpline suicide helpline classification interpretable AI explainable AI conversations BERT bidirectional encoder representations from transformers machine learning artificial intelligence large language models LLM natural language processing

来源：评论

学校读者我要写书评

暂无评论

Large Language Models for Mental Health Applications:Systematic Review

引用

JMIR MENTAL HEALTH 2024年 11卷 e57400页

作者： Guo, Zhijun Lai, Alvina Thygesen, Johan H. Farrington, Joseph Keen, Thomas Li, Kezhi UCL Inst Hlth Informat 222 Euston Rd London NW1 2DA England UCL Great Ormond St Inst Child Hlth London England

Background: Large language models (LLMs) are advanced artificial neural networks trained on extensive datasets to accurately understand and generate natural language. While they have received much attention and demonstrated potential in digital health, their application in mental health, particularly in clinical settings, has generated considerable debate. Objective: This systematic review aims to critically assess the use of LLMs in mental health, specifically focusing on theirapplicability and efficacy in early screening, digital interventions, and clinical settings. By systematically collating and assessingthe evidence from current studies, our work analyzes models, methodologies, data sources, and outcomes, thereby highlightingthe potential of LLMs in mental health, the challenges they present, and the prospects for their clinical use. Methods: Adhering to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, thisreview searched 5 open-access databases: MEDLINE (accessed by PubMed), IEEE Xplore, Scopus, JMIR, and ACM DigitalLibrary. Keywords used were (mental healthOR mental illnessOR mental disorderOR psychiatry) AND (large languagemodels). This study included articles published between January 1, 2017, and April 30, 2024, and excluded articles published inlanguages other than English. Results: In total, 40 articles were evaluated, including 15 (38%) articles on mental health conditions and suicidal ideationdetection through text analysis, 7 (18%) on the use of LLMs as mental health conversational agents, and 18 (45%) on otherapplications and evaluations of LLMs in mental health. LLMs show good effectiveness in detecting mental health issues andproviding accessible, destigmatized eHealth services. However, assessments also indicate that the current risks associated withclinical use might surpass their benefits. These risks include inconsistencies in generated text;the production of hallucinations;and the absence of a compre

关键词： large language models mental health digital health care ChatGPT bidirectional encoder representations from transformers BERT

来源：评论

学校读者我要写书评

暂无评论

A Combination of Enhanced WordNet and BERT for Semantic Textual Similarity

A Combination of Enhanced WordNet and BERT for Semantic Text...

引用

2021年第二届控制、机器人和智能系统国际会议

作者： Shruthi Srinarasi Reshma Ram Sanjay Raghavendra Annapurna P Patil S Rajarajeswari Manjunath Belgod Rituraj Kabra Abhishek Singh Ramaiah Institute of Technology Samsung Research and Development Institute India-Bangalore

The task of measuring sentence similarity deals with computing the likeness between a pair of sentences by adopting Natural Language Processing techniques(Euclidean distance,Jaccard distance,Manhattan distance,etc.) as well as embedding techniques(word2vec,GloVe,Flair,etc.).For the purpose of determining sentence similarity,this paper proposes a novel,ensemble learning approach which uses the WordNet corpus and the bidirectional encoder representations from transformers(BERT) in order to consider the context of words in sentences while computing the similarity *** accuracy of the proposed model is computed by calculating the Pearson and Spearman scores for the sentence pairs from the Sentences Involving Compositional Knowledge(SICK) *** analyzing the results,the proposed approach is observed to outperform existing state-of-the-art semantic textual similarity models since it returns the highest correlation ***,this paper also introduces a possible machine learning approach for the same and evaluates its scope and drawbacks.

关键词： Natural Language Processing Sentence similarity WordNet bidirectional encoder representations from transformers Semantic textual similarity

来源：评论

学校读者我要写书评

暂无评论

Enhanced Unsupervised Data Augmentation for Emergency Events Detection and Classification

Enhanced Unsupervised Data Augmentation for Emergency Events...

引用

第33届中国控制与决策会议

作者： Xiaomeng Liu Fei Long Kun Huang Qiang Ling State Key Laboratory of Media Convergence Production Technology and Systems Xinhua News Agency Department of Automation University of Science and Technology of China

We propose to jointly detect and classify emergency events using a multi-class text classifier,which is a typical deep learning architecture with transformer modules and particularly employs bidirectional encoder representations from transformers(BERT).Deep learning requires a large number of labeled data to ***,deep learning often implements the semi-supervised learning(SSL) method,which is able to use massive unlabeled data to improve performance of supervised deep *** an effective SSL variant,unsupervised data augmentation(UDA) focuses on data augmentation techniques to improve the performance of deep *** present an enhanced version of UDA(EUDA) by mixing more data augmentation strategies and using a problem related *** EUDA targets at emergency event detection and *** that emergency events always have time and location elements,text can be filtered based on this semantic *** propose to use semantic feature aided enhanced unsupervised data augmentation to solve the concerned *** studies on the dataset prepared for the task validates that the proposed EUDA can achieve significantly better performance than supervised learning with a limited size of labeled *** are also carried out on a text classification task,which confirms that EUDA improves performance for BERT neural network.

关键词： Semi-supervised Learning bidirectional encoder representations from transformers Unsupervised Data Augmentation

来源：评论

学校读者我要写书评

暂无评论

Short human eccDNAs are predictable from sequences

引用

BRIEFINGS IN BIOINFORMATICS 2023年第3期24卷 bbad147-bbad147页

作者： Chang, Kai-Li Chen, Jia-Hong Lin, Tzu-Chieh Leu, Jun-Yi Kao, Cheng-Fu Wong, Jin Yung Tsai, Huai-Kuang Acad Sin Inst Informat Sci Taipei 115 Taiwan Acad Sin Inst Informat Sci Taipei Taiwan Natl Taiwan Univ Grad Inst Elect Engn Taipei Taiwan Acad Sin Inst Mol Biol Taipei Taiwan Acad Sin Inst Cellular & Organism Biol Taipei Taiwan

Background Ubiquitous presence of short extrachromosomal circular DNAs (eccDNAs) in eukaryotic cells has perplexed generations of biologists. Their widespread origins in the genome lacking apparent specificity led some studies to conclude their formation as random or near-random. Despite this, the search for specific formation of short eccDNA continues with a recent surge of interest in biomarker development. Results To shed new light on the conflicting views on short eccDNAs' randomness, here we present DeepCircle, a bioinformatics framework incorporating convolution- and attention-based neural networks to assess their predictability. Short human eccDNAs from different datasets indeed have low similarity in genomic locations, but DeepCircle successfully learned shared DNA sequence features to make accurate cross-datasets predictions (accuracy: convolution-based models: 79.65 +/- 4.7%, attention-based models: 83.31 +/- 4.18%). Conclusions The excellent performance of our models shows that the intrinsic predictability of eccDNAs is encoded in the sequences across tissue origins. Our work demonstrates how the perceived lack of specificity in genomics data can be re-assessed by deep learning models to uncover unexpected similarity.

关键词： bidirectional encoder representations from transformers convolutional neural network deep learning extrachromosomal circular DNA

来源：评论

学校读者我要写书评

暂无评论

Comparison of Machine Learning Algorithms for Predicting Hospital Readmissions and Worsening Heart Failure Events in Patients With Heart Failure With Reduced Ejection Fraction: Modeling Study

引用

JMIR FORMATIVE RESEARCH 2023年 7卷 e41775页

作者： Ru, Boshu Tan, Xi Liu, Yu Kannapur, Kartik Ramanan, Dheepan Kessler, Garin Lautsch, Dominik Fonarow, Gregg Merck & Co Inc Rahway NJ USA Amazon Web Serv Inc Seattle WA USA Georgetown Univ Sch Continuing Studies Washington DC USA Univ Calif Los Angeles Ahmanson UCLA Cardiomyopathy Ctr Los Angeles CA USA Univ Calif Los Angeles Ahmanson UCLA Cardiomyopathy Ctr 10833 LeConte Ave Los Angeles CA 90095 USA

Background: Heart failure (HF) is highly prevalent in the United States. Approximately one-third to one-half of HF cases are categorized as HF with reduced ejection fraction (HFrEF). Patients with HFrEF are at risk of worsening HF, have a high risk of adverse outcomes, and experience higher health care use and costs. Therefore, it is crucial to identify patients with HFrEF who are at high risk of subsequent events after HF ***: Machine learning (ML) has been used to predict HF-related outcomes. The objective of this study was to compare different ML prediction models and feature construction methods to predict 30-, 90-, and 365-day hospital readmissions and worsening HF events (WHFEs).Methods: We used the Veradigm PINNACLE outpatient registry linked to Symphony Health's Integrated Dataverse data from July 1, 2013, to September 30, 2017. Adults with a confirmed diagnosis of HFrEF and HF-related hospitalization were included. WHFEs were defined as HF-related hospitalizations or outpatient intravenous diuretic use within 1 year of the first HF hospitalization. We used different approaches to construct ML features from clinical codes, including frequencies of clinical classification software (CCS) categories, bidirectional encoder representations from transformers (BERT) trained with CCS sequences (BERT + CCS), BERT trained on raw clinical codes (BERT + raw), and prespecified features based on clinical knowledge. A multilayer perceptron neural network, extreme gradient boosting (XGBoost), random forest, and logistic regression prediction models were applied and ***: A total of 30,687 adult patients with HFrEF were included in the analysis;11.41% (3184/27,917) of adults experienced a hospital readmission within 30 days of their first HF hospitalization, and nearly half (9231/21,562, 42.81%) of the patients experienced at least 1 WHFE within 1 year after HF hospitalization. The prediction models and feature combinations with the best are

关键词： deep learning machine learning hospital readmission heart failure heart failure with reduced ejection fraction worsening heart failure event bidirectional encoder representations from transformers BERT clinical registry medical claims real-world data

来源：评论

学校读者我要写书评

暂无评论

Natural language processing versus rule-based text analysis: Comparing BERT score and readability indices to predict crowdfunding outcomes

引用

Journal of Business Venturing Insights 2021年 16卷

作者： Chan, C.S. Richard Pethe, Charuta Skiena, Steven College of Business Stony Brook University Stony Brook 11794 NY United States Computer Science Stony Brook University Stony Brook 11794 NY United States AI Institute Computer Science Stony Brook University Stony Brook 11794 NY United States

We explore how natural language processing can be applied to predict crowdfunding outcomes. Using the bidirectional encoder representations from transformers (BERT) technique, we find that crowdfunding projects that use a story section description with a higher average BERT score (indicating a lower quality of writing) tend to raise more funding than those with lower average BERT scores. In contrast, risk descriptions that have higher BERT scores tend to receive less funding and attract fewer backers. These relationships remain consistent after controlling for various traditional readability indices, highlighting the potential benefits of incorporating natural language processing techniques in entrepreneurship research. © 2021

关键词： bidirectional encoder representations from transformers Crowdfunding outcomes Natural language processing Readability

来源：评论

学校读者我要写书评

暂无评论

Pretrained Transformer Language Models Versus Pretrained Word Embeddings for the Detection of Accurate Health Information on Arabic Social Media: Comparative Study

引用

JMIR FORMATIVE RESEARCH 2022年第6期6卷 e34834页

作者： Albalawi, Yahya Nikolov, Nikola S. Buckley, Jim Univ Limerick Dept Comp Sci & Informat Syst Tierney Bldg Limerick V94 T9PX Ireland Univ Taibah Coll Arts & Sci Dept Comp & Informat Sci Al Ula Saudi Arabia Univ Limerick Irish Software Res Ctr Limerick Ireland

Background: In recent years, social media has become a major channel for health-related information in Saudi Arabia. Prior health informatics studies have suggested that a large proportion of health-related posts on social media are inaccurate. Given the subject matter and the scale of dissemination of such information, it is important to be able to automatically discriminate between accurate and inaccurate health-related posts in Arabic. Objective: The first aim of this study is to generate a data set of generic health-related tweets in Arabic, labeled as either accurate or inaccurate health information. The second aim is to leverage this data set to train a state-of-the-art deep learning model for detecting the accuracy of health-related tweets in Arabic. In particular, this study aims to train and compare the performance of multiple deep learning models that use pretrained word embeddings and transformer language models. Methods: We used 900 health-related tweets from a previously published data set extracted between July 15, 2019, and August 31, 2019. Furthermore, we applied a pretrained model to extract an additional 900 health-related tweets from a second data set collected specifically for this study between March 1, 2019, and April 15, 2019. The 1800 tweets were labeled by 2 physicians as accurate, inaccurate, or unsure. The physicians agreed on 43.3% (779/1800) of tweets, which were thus labeled as accurate or inaccurate. A total of 9 variations of the pretrained transformer language models were then trained and validated on 79.9% (623/779 tweets) of the data set and tested on 20% (156/779 tweets) of the data set. For comparison, we also trained a bidirectional long short-term memory model with 7 different pretrained word embeddings as the input layer on the same data set. The models were compared in terms of their accuracy, precision, recall, F-1 score, and macroaverage of the F-1 score. Results: We constructed a data set of labeled tweets, 38% (296/779) o

关键词： social media machine learning pretrained language models bidirectional encoder representations from transformers BERT deep learning health information infodemiology tweets language model health informatics misinformation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：