Are Clinical T5 Models Better for Clinical Text?

作     者：Li, Yahan Harrigian, Keith Zirikly, Ayah Dredze, Mark 

作者机构：Department of Computer Science University of Southern California United States Department of Computer Science Johns Hopkins University United States Center for Language and Speech Processing Whiting School of Engineering Johns Hopkins University United States Malone Center for Engineering in Healthcare Johns Hopkins University United States 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2024年

核心收录：

摘      要：Large language models with a transformerbased encoder/decoder architecture, such as T5 (Raffel et al., 2023), have become standard platforms for supervised tasks. To bring these technologies to the clinical domain, recent work has trained new (Lehman et al., 2023) or adapted existing (Lu et al., 2022) models to clinical data. However, the evaluation of these clinical T5 models and comparison to other models has been limited. Are the clinical T5 models better choices than FLAN-tuned (Chung et al., 2022a) generic T5 models? Do they generalize better to new clinical domains that differ from the training sets? We comprehensively evaluate these models across several clinical tasks and domains. We find that clinical T5 models provide marginal improvements over existing models, and perform worse when evaluated on different domains. Our results inform future choices in developing clinical LLMs. © 2024, CC BY-NC-ND.