TCMLLM-PR:evaluation of large language models for prescription recommendation in traditional Chinese medicine

作     者：TIAN Haoyu YANG Kuo DONG Xin ZHAO Chenxi YE Mingwei WANG Hongyan LIU Yiming HU Minjie ZHU Qiang YU Jian ZHANG Lei ZHOU Xuezhong 田昊宇;杨扩;董鑫;赵辰羲;叶明蔚;王鸿燕;刘一铭;胡敏杰;诸强;于剑;张磊;周雪忠

作者机构：Beijing Key Lab of Traffic Data Analysis and MiningSchool of Computer Science&TechnologyBeijing Jiaotong UniversityBeijing 100044China National Data Center of Traditional Chinese MedicineChina Academy of Chinese Medical SciencesBeijing 100700China 

出 版 物：《Digital Chinese Medicine》 (数字中医药(英文))

年 卷 期：2024年第7卷第4期

页      面：343-355页

核心收录：

学科分类：1002[医学-临床医学] 100214[医学-肿瘤学] 10[医学] 

基　　金：National Key Research and Development Program(2023YFC3502604) National Natural Science Foundation of China(U23B2062 and 82374302) 

主　　题：Large language models Instruction-tuning Prescription recommendation Traditional Chinese medicine(TCM) Assisted decision-making 

摘      要：Objective To develop and evaluate a fine-tuned large language model(LLM)for traditional Chinese medicine(TCM)prescription recommendation named *** First,we constructed an instruction-tuning dataset containing 68654 samples(ap-proximately 10 million tokens)by integrating data from eight sources,including four TCM textbooks,Pharmacopoeia of the People’s Republic of China 2020(CHP),Chinese Medicine Clinical Cases(CMCC),and hospital clinical records covering lung disease,liver disease,stroke,diabetes,and splenic-stomach ***,we trained TCMLLM-PR using Chat-GLM-6B with P-Tuning v2 *** evaluation consisted of three aspects:(i)compari-son with traditional prescription recommendation models(PTM,TCMPR,and PresRecST);(ii)comparison with TCM-specific LLMs(ShenNong,Huatuo,and HuatuoGPT)and general-domain ChatGPT;(iii)assessment of model migration capability across different disease *** employed precision,recall,and F1 score as evaluation *** The experiments showed that TCMLLM-PR significantly outperformed baseline models on TCM textbooks and CHP datasets,with F1@10 improvements of 31.80%and 59.48%,*** cross-dataset validation,the model performed best when migrating from TCM textbooks to liver disease dataset,achieving an F1@10 of *** of real-world cases demonstrated that TCMLLM-PR s prescription recommendations most closely matched actual doctors’*** This study integrated LLMs into TCM prescription recommendations,leverag-ing a tailored instruction-tuning dataset and developing *** study will pub-licly release the best model parameters of TCMLLM-PR to promote the development of the decision-making process in TCM practices(https://***/2020MEAI/TCMLLM).