检索结果-内蒙古大学图书馆

Large language models fall short in classifying learners’ open-ended responses

Research Methods in Applied Linguistics 2025年第2期4卷

作者： Mizumoto, Atsushi Teng, Mark Feng Kansai University Japan Macao Polytechnic University Macau SAR China

Generative Artificial Intelligence (GenAI), based on large language models (LLMs), excels in various language comprehension tasks and is increasingly utilized in applied linguistics research. This study examines the accuracy and methodological implications of using LLMs to classify open-ended responses from learners. We surveyed 143 Japanese university students studying English as a foreign language (EFL) about their essay-writing process. Two human evaluators independently classified the students’ responses based on self-regulated learning processes: planning, monitoring, and evaluation. At the same time, several LLMs performed the same classification task, and their results were compared with those of the human evaluators using Cohen's kappa coefficient. We established κ ≥ 0.8 as the threshold for strong agreement based on rigorous methodological standards. Our findings revealed that even the best-performing model (DeepSeek-V3) achieved only moderate agreement (κ = 0.68), while other models demonstrated fair-to-moderate agreement (κ = 0.37–0.61). Surprisingly, open-source models outperformed several commercial counterparts. These results highlight the necessity of expert oversight when integrating GenAI as a support tool in qualitative data analysis. The paper concludes by discussing the methodological implications for using LLMs in qualitative research and proposing specific prompt engineering strategies to enhance their reliability in applied linguistics. © 2025 The Authors

关键词： coding and classification Generative AI Large language models (LLM) Qualitative analysis Research methods

来源：评论

学校读者我要写书评

暂无评论

A Terminology Server for Medical Language and Medical Information Systems

引用

Methods of Information in Medicine 2018年第1/02期34卷 147-157页

作者： A. L. Rector W. D. Solomon W. A. Nowlan T. W. Rush P. E. Zanstra W. M. A. Claassen

GALEN is developing a Terminology Server to support the development and integration of clinical systems through a range of key

关键词： Terminology Server Natural Language Processing coding and classification Electronic Medical Records Knowledge Representation

来源：评论

学校读者我要写书评

暂无评论

Detection of Adverse Events Through Hospital Administrative Data 5th

Detection of Adverse Events Through Hospital Administrative ...

引用

World Conference on Information Systems and Technologies (WorldCIST)

作者： Marques, Bernardo Sousa-Pinto, Bernardo Silva-Costa, Tiago Lopes, Fernando Freitas, Alberto Univ Porto Fac Med Dept Community Med Informat & Hlth Decis Sci MEDC Oporto Portugal CINTESIS Ctr Hlth Technol & Serv Res Oporto Portugal

ISBN: (纸本)9783319565385;9783319565378

This study aims to estimate and describe the incidence of adverse events (AE) registered in Portuguese public hospitals and consequently to determine the feasibility of using hospital administrative data as a tool for AE surveillance. A retrospective observational study using hospital administrative data was performed to detect the incidence of AE based on a selection of ICD-9-CM codes (diagnoses and external causes). All episodes in public hospitals in the period 2000-2010 were included. AE were divided in three main categories: complications of surgical or medical procedures, misadventures of surgical and medical care, and adverse drug events (ADE). The ADE subgroup was further subdivided in: poisoning, late effect, and adverse drug reaction. Over the studied period, the algorithm was able to detect 543,242 episodes with AE events (3.7% of all episodes), with an in-hospital mortality rate of 6.3%, and a median length-of-stay of 8 days. In a scenario of underreporting of AE, this administrative data approach in an important complement to the other existing surveillance techniques.

关键词： Administrative data coding and classification ICD-9-CM Adverse events Adverse drug events Data quality

来源：评论

学校读者我要写书评

暂无评论

Exploring coronial determination of intent for poisoning-related deaths in Australia, 2001-2013

引用

BMC PUBLIC HEALTH 2017年第1期18卷 83-83页

作者： Churruca, Kate Mitchell, Rebecca Macquarie Univ Australian Inst Hlth Innovat Level 675 Talavera Rd Sydney NSW 2109 Australia

Background: In countries like the United States and the United Kingdom, systematic variation in the classification of intent in pharmaceutical poisoning deaths have been identified between jurisdictions. This study aimed to explore whether the coronial determination of intent (unintentional, intentional, undetermined) for pharmaceutical-related poisoning deaths may have affected death rates over time and by jurisdiction in Australia. Methods: A retrospective examination of mortality records in the National Coronial Information System (NCIS) during 1 January 2001 to 31 December 2013 was conducted. The NCIS is a national internet-based data storage and retrieval system for deaths that were notified to a coroner. Pharmaceutical deaths due to unintentional, intentional or undetermined intent were identified using the NCIS classification. Proportions of the different intent classifications and the mortality rates by intent over time were compared between jurisdictions. Results: There were 17,895 pharmaceutical-related poisoning deaths in Australia between 2001 and 2013 that had closed cases in the NCIS. Proportions of deaths classified as unintentional (48.3-66.3%), intentional (24.7-35.9%) and undetermined (6.7-24.7%) varied significantly among Australian jurisdictions. There were significant increases in the rate of classification of unintentional poisoning for some states, and significant increases in intentional poisoning classification in Western Australia, and decreases in New South Wales and Victoria. There was no significant change in classification of undetermined intent. Conclusions: Significant variation in classifications of intent, both between state jurisdictions and over time, may be the result of regional differences in demographics and increases in prescription drug misuse. However, the inconsistent use of 'undetermined' intent between state jurisdictions suggests coroners may experience varying difficulty in retrospectively ruling on intent in the equiv

关键词： Coroners systems Suicide Drug and alcohol policy Medico-legal databases coding and classification

来源：评论

学校读者我要写书评

暂无评论

Comorbidity coding trends in hospital administrative databases

Comorbidity coding trends in hospital administrative databas...

引用

World Conference on Information Systems and Technologies (WorldCIST)

作者： Freitas, Alberto Lema, Isabel da Costa-Pereira, Altamiro Univ Porto Dept Hlth Informat & Decis Sci Fac Med Oporto Portugal CINTESIS Ctr Hlth Technol & Serv Res Oporto Portugal

ISBN: (纸本)9783319313078;9783319313061

Comorbidities are related to an increase in hospital costs, hospital mortality and length-of-stay. The analysis of inpatient comorbidities is therefore important for hospital management, for epidemiological studies, and for health services research and planning. In this paper we study the annual evolution of coded comorbidities in administrative databases. We used data from Portuguese hospitals over a period of eleven years (2000-2010). We identified comorbidities in 7,034,213 inpatient episodes using both the Elixhauser and the Charslon/Deyo comorbidity methods. Our results clearly evidenced a positive association between the number of secondary diagnosis and coded comorbidities. We argue that the increased number of comorbidities over time is mostly related to an increase in the quality of coded data. Data analysts, researchers and decision makers should always be aware of possible data quality bias, such as completeness, when using administrative databases.

关键词： ICD-9-CM comorbidity coding and classification administrative data data quality

来源：评论

学校读者我要写书评

暂无评论

Automatic object-oriented coding facility for product life cycle management of discrete products

引用

INTERNATIONAL JOURNAL OF COMPUTER INTEGRATED MANUFACTURING 2014年第1期27卷 60-84页

作者： Khan, Wasim A. Hussain, Amir GIK Inst Engn Sci & Technol Topi 23640 Swabi Pakistan Inst Business Adm Karachi 74400 Pakistan

Discrete products are produced in enormous quantity and diversity all over the world. A global collaborative manufacturing environment requires the minimisation of environmental impact that these products shall ultimately pose. The implementation of an efficient product life cycle management system that encapsulates complete historical information about the product shall prove useful. This information may pertain to the design methodology, material type, details regarding manufacturing methods, use of coating or paints and exposure to radiation, chemicals of varying pH and electroweak forces during production or use. It is essential to have access to this information when the product is in use or in the disassembly stage at the end of the useful life of a product to accomplish the proper reuse or disposal of each component. A coding system that embodies the product life cycle stages for discrete products is envisaged. The proposed coding system stores parameters that are related to the design, manufacture, usage and disposal of a product. A computer programme is developed to establish a product life cycle code (PLCC) in the form of a 256-bit binary number or 64-bit hexadecimal number. The code can be allocated to a component or product at its inception in an interactive web session and stored in a repository for record keeping. The cradle-to-grave binary or hexadecimal code becomes an essential part of each component and product and facilitates the usage, disassembly and disposal of each component. A case study related to the generation of PLCC for discrete products is included towards the end of the article.

关键词： product life cycle code coding and classification system design discrete manufacturing object-oriented programming object-oriented databases

来源：评论

学校读者我要写书评

暂无评论

From a time standard for medical informatics to a controlled language for health

引用

INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS 1998年第1-3期48卷 85-101页

作者： Ceusters, W Steurs, F Zanstra, P Van der Haring, E Rogers, J Off Line Engn NV B-9520 Zonnegem Belgium Katholieke Vlaamse Hogesch Dept Informat B-2000 Antwerp Belgium State Univ Ghent Hosp Dept Med Informat B-9000 Ghent Belgium Univ Nijmegen Dept Med Informat & Epidemiol NL-6500 HB Nijmegen Netherlands Univ Manchester Dept Comp Sci Med Informat Grp Manchester M13 9PL Lancs England

CEN ENV 12381 is a European Prestandard focusing on formal representation and explicit reference of temporal information in healthcare informatics and telematics. One of its merits is not just the possibility to represent natural language expressions containing time-related information in a structured way, but also to give some mechanisms on how clinical language itself can be used to convey meaning unambiguously. As such, CEN ENV 12381 introduces the notion of 'controlled language use' in the domain of healthcare. In this paper the principles behind controlled language design and use are explained. Through a detailed study of the inconsistencies and ambiguities that arise when interpreting Snomed procedure terms in the framework of the Galen-In-Use project, it is shown that most of them can be explained as a violation of sound term-formation principles. A proposal is made to develop a controlled language for health and to use it in subsequent versions of coding and classification systems. It is expected that such an endeavour will lead to a more effective application of linguistic engineering in areas such as automatic knowledge acquisition, automatic translation, and terminology validation in the domain of healthcare informatics. (C) 1998 Elsevier Science Ireland Ltd, All rights reserved.

关键词： controlled language natural language processing coding and classification formal terminological systems

来源：评论

学校读者我要写书评

暂无评论

From a time standard for medical informatics to a controlled language for health

From a time standard for medical informatics to a controlled...

引用

Meeting on Standardisation in Health Informatics - Towards Global Consensus and Cooperation

关键词： controlled language natural language processing coding and classification formal terminological systems

来源：评论

学校读者我要写书评

暂无评论

FORCOD - A coding and classification-SYSTEM FOR FORMED PARTS

引用

JOURNAL OF MANUFACTURING SYSTEMS 1991年第3期10卷 223-232页

作者： JUNG, JY AHLUWALIA, RS West Virginia University Morgantown WV USA

Part coding and classification systems are required to apply the group technology philosophy. Thus far, most coding and classification systems are for metal removal processes. This paper proposes a coding and classification scheme for the metal forming process. coding details based on part geometry, tolerance level, surface finish, material, heat treatment, and defect level are defined. As an example, based on the proposed coding scheme, a multi-objective cluster analysis problem is formulated and solved for bosses and connecting rods, which are typical examples of forged parts.

关键词： GROUP TECHNOLOGY coding and classification CLUSTERING METAL FORMING OPTIMIZATION

来源：评论

学校读者我要写书评

暂无评论

CONVERTING THE REPRESENTATION OF MEDICAL DATA - CRITERIA TO CODE THE UNDERLYING CAUSE OF DEATH

引用

METHODS OF INFORMATION IN MEDICINE 1990年第3期29卷 220-235页

作者： PISANELLI, DM ROSSIMORI, A Istituto di Tecnologie Biomediche CNR Rome Italy.

This study deals with a set of coding directives that were conceived for trained coding clerks and rely upon knowledge of their cultural background. These directives were formalized and adapted for computer use and in this form must rely upon a background of explicit medical knowledge. Medical data on death certificates are an invaluable source of information regarding prevention of major causes of death. These causes are coded and tabulated worldwide by means of the International classification of Diseases (ICD). The ICD manual issues directives to achieve uniformity of coding throughout the world. The coder is required to trace back the flow of events which caused death and to single out the most significant concept from the statistical point of view. After emphasizing the problems encountered in the formalization, the methodological contribution of this work to the identification of a modular architecture for a system which represents and “reshapes” knowledge from medical documents is presented. Therefore we focus on the features of the two kinds of knowledge that must be supplied to a knowledge-based system, in order to enable it to perform semantic conversions on given medical data, namely: i) generic guidelines; ii) detailed medical knowledge.

关键词： Semantic Standards coding and classification ICD 9 Decision Support Systems Knowledge Bases Causes of Death Death Cer tificates

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：