检索结果-内蒙古大学图书馆

Reducing the Misclassification Costs of Cognitive Diagnosis computerized adaptive testing: Item Selection With Minimum Expected Risk

引用

APPLIED PSYCHOLOGICAL MEASUREMENT 2022年第3期46卷 185-199页

作者： Hsu, Chia-Ling Wang, Wen-Chung Educ Univ Hong Kong Hong Kong Peoples R China Hong Kong Examinat & Assessment Author Hong Kong Peoples R China

Cognitive diagnosis computerized adaptive testing (CD-CAT) aims to identify each examinee's strengths and weaknesses on latent attributes for appropriate classification into an attribute profile. As the cost of a CD-CAT misclassification differs across user needs (e.g., remedial program vs. scholarship eligibilities), item selection can incorporate such costs to improve measurement efficiency. This study proposes such a method, minimum expected risk (MER), based on Bayesian decision theory. According to simulations, using MER to identify examinees with no mastery (MER-U0) or full mastery (MER-U1) showed greater classification accuracy and efficiency than other methods for these attribute profiles, especially for shorter tests or low quality item banks. For other attribute profiles, regardless of item quality or termination criterion, MER methods, modified posterior-weighted Kullback-Leibler information (MPWKL), posterior-weighted CDM discrimination index (PWCDI), and Shannon entropy (SHE) performed similarly and outperformed posterior-weighted attribute-level CDM discrimination index (PWACDI) in classification accuracy and test efficiency, especially on short tests. MER with a zero-one loss function, MER-U0, MER-U1, and PWACDI utilized item banks more effectively than the other methods. Overall, these results show the feasibility of using MER in CD-CAT to increase the accuracy for specific attribute profiles to address different user needs.

关键词： cognitive diagnosis computerized adaptive testing item selection bayesian decision theory minimum expected cost

来源：评论

学校读者我要写书评

暂无评论

To Implement computerized adaptive testing by Automatically Adjusting Item Difficulty Index on adaptive English Learning Platform

引用

JOURNAL OF INTERNET TECHNOLOGY 2021年第7期22卷 1599-1608页

作者： Cheng, Shu-Chen Cheng, Yu-Ping Huang, Yueh-Min Southern Taiwan Univ Sci & Technol Dept Comp Sci & Informat Engn Tainan Taiwan Natl Cheng Kung Univ Dept Engn Sci Tainan Taiwan

In previous language teaching studies, there were still some issues that have rarely been discussed or been of concern. One of the issues is the limitation to provide the appropriate test items according to the degree of the students' capability in computerized adaptive testing. However, the difficulty level of a test item relies on expert judgment, which it is too time-consuming and expert-dependent. In order to effectively achieve automatic adjustment of the difficulty levels of test items to implement computerized adaptive testing, an attempt is made to address this issue by developing an adaptive English learning platform. This platform was used to analyze the difficulty levels of the items along with 162 individual ability values from the massive data of the 53,191 item answer records. The research results showed that analyzing the records using big data analysis and the item response theory enables the platform to automatically determine and adjust the difficulty levels of the items and to provide the fitting test items that match learner capabilities in computerized adaptive testing. Therefore, teachers do not have to manually adjust the difficulty levels of the test items. In addition, students can use computerized adaptive tests to improve their learning performance.

关键词： Item difficulty Index Item response theory computerized adaptive testing

来源：评论

学校读者我要写书评

暂无评论

Diffusion-Inspired Cold Start with Sufficient Prior in computerized adaptive testing 25

Diffusion-Inspired Cold Start with Sufficient Prior in Compu...

引用

Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1

作者： Haiping Ma Aoqing Xia Changqian Wang Hai Wang Xingyi Zhang Information Materials and Intelligent Sensing Laboratory of Anhui Province the Institutes of Physical Science and Information Technology Anhui University Hefei Anhui China The Institutes of Physical Science and Information Technology Anhui University Hefei Anhui China The Institutes of Physical Science and Information Technology Anhui University hefei anhui China School of Computer Science and Technology Anhui University Hefei Anhui China

ISBN: (纸本)9798400712456

computerized adaptive testing (CAT) aims to select the most appropriate questions based on the examinee's ability and is widely used in online education. However, existing CAT systems often lack initial understanding of the examinee's ability, requiring random probing questions. This can lead to poorly matched questions, extending the test duration and negatively impacting the examinee's mindset, a phenomenon referred to as the Cold Start with Insufficient Prior (CSIP) task. This issue occurs because CAT systems do not effectively utilize the abundant prior information about the examinee available from other courses on online platforms. These response records, due to the commonality of cognitive states across different knowledge domains, can provide valuable prior information for the target domain. However, no prior work has explored solutions for the CSIP task. In response to this gap, we propose Diffusion Cognitive States TransfeR Framework (DCSR), a novel domain transfer framework based on Diffusion Models (DMs) to address the CSIP task. Specifically, we construct a cognitive state transition bridge between domains, guided by the common cognitive states of examinees, encouraging the model to reconstruct the initial ability state in the target domain. To enrich the expressive power of the generated data, we analyze the causal relationships in the generation process from a causal perspective. Redundant and extraneous cognitive states can lead to limited transfer and negative transfer effects. Therefore, we designed three decoupling strategies to control confounding variables, thereby blocking backdoor paths that hinder causal discovery. Given that excessive uncertainty can affect the applicability of generated results to the CAT system, we propose consistency constraint and task-oriented constraint to control the randomness of the generated results and their relevance to the CAT task, respectively. Our DCSR can seamlessly apply the generated initial ability states

关键词： computerized adaptive testing

来源：评论

学校读者我要写书评

暂无评论

Comprehensive Empirical Analysis of Stop Criteria in computerized adaptive testing 13

Comprehensive Empirical Analysis of Stop Criteria in Compute...

引用

13th International Conference on Computer Supported Education (CSEDU)

作者： Gilavert, Patricia Freire, Valdinei Univ Sao Paulo Sch Arts Sci & Humanities Sao Paulo Brazil

ISBN: (纸本)9789897585029

computerized adaptive testing is an assessment approach that selects questions one after another while conditioning each selection on the previous questions and answers. CAT is evaluated mainly for its precision, the correctness of estimation of the examinee trait, and efficiency, the test length. The precision-efficiency trade-off depends mostly on two CAT components: an item selection criterion and a stop criterion. While much research is dedicated to the first, stop criteria lack relevant research. We contribute with a comprehensive evaluation of stop criteria. First, we test a variety of seven stop-criteria for different setups of item banks and estimation mechanism. Second, we contribute with a precision-efficiency trade-off method to evaluate stop criteria. Finally, we contribute with an experiment considering simulations over a myriad of synthetic item banks. We conclude in favor of the Fixed-Length criterion, as long it can be tuned to the item bank at hand;the Fixed-Length criterion shows a competitive precision-efficiency trade-off curve in every scenario while presenting zero variance in test length. We also highlight that estimation mechanism and item-bank distribution have a influence over the performance of stop criteria.

关键词： computerized adaptive testing Stop Criteria Combine Stop Criteria Threshold to Stop

来源：评论

学校读者我要写书评

暂无评论

computerized adaptive testing and short form development for child and adolescent oral health patient-reported outcomes measurement

引用

CLINICAL AND EXPERIMENTAL DENTAL RESEARCH 2020年第1期6卷 124-133页

作者： Shen, Jie Hays, Ron D. Wang, Yan Marcus, Marvin Maida, Carl A. Xiong, Di Lee, Steve Y. Spolsky, Vladimir W. Coulter, Ian D. Crall, James J. Liu, Honghu UCLA Sch Dent Publ Hlth & Community Dent Los Angeles CA 90095 USA UCLA David Geffen Sch Med Dept Med Los Angeles CA 90095 USA UCLA Dept Biostat Los Angeles CA USA

Objectives To develop computerized adaptive testing (CAT) and short forms of self-report oral health measures that are predictive of both the children's oral health status index (COHSI) and the children's oral health referral recommendation (COHRR) scales, for children and adolescents, ages 8-17. Material and methods Using final item calibration parameters (discrimination and difficulty parameters) from the item response theory analysis, we performed post hoc CAT simulation. Items most frequently administered in the simulation were incorporated for possible inclusion in final oral health assessment toolkits, to select the best performing eight items for COHSI and COHRR. Results Two previously identified unidimensional sets of self-report items consisting of 19 items for the COHSI and 22 items for the COHRR were administered through CAT resulting in eight-item short forms for both the COHSI and COHRR. Correlations between the simulated CAT scores and the full item bank representing the latent trait are r = .94 for COHSI and r = .96 for COHRR, respectively, which demonstrated high reliability of the CAT and short form. Conclusions Using established rigorous measurement development standards, the CAT and corresponding eight-item short form items for COHSI and COHRR were developed to assess the oral health status of children and adolescents, ages 8-17. These measures demonstrated good psychometric properties and can have clinical utility in oral health screening and evaluation and clinical referral recommendations.

关键词： computerized adaptive testing oral health PROMIS short form

来源：评论

学校读者我要写书评

暂无评论

Improving Efficiency of Patient-Reported Outcome Collection: Application of computerized adaptive testing to DASH and QuickDASH Outcome Scores

引用

JOURNAL OF HAND SURGERY-AMERICAN VOLUME 2021年第4期46卷 278-286页

作者： Kane, Liam T. Abboud, Joseph A. Plummer, Otho R. Beredjiklian, Pedro T. Rothman Orthopaed Inst 925 Chestnut St Philadelphia PA 19107 USA Universal Res Solut Columbia MO USA

Purpose Patient-reported outcome measures assess health status and treatment outcomes in orthopedic care, but they may burden patients with lengthy questionnaires. Predictive models using machine learning, known as computerized adaptive testing (CAT), offer a potential solution. This study evaluates the ability of CAT to improve efficiency of the 30-item Disabilities of the Arm, Shoulder, and Hand (DASH) and 11-item QuickDASH questionnaires. Methods A total of 2,860 DASH and 27,355 QuickDASH respondents were included in the analysis. The CAT system was retrospectively applied to each set of patient responses stored on the instrument to calculate a CAT-specific score for all DASH and QuickDASH entries. The accuracy of the CAT scores, viewed in the context of the minimal clinically important difference for both patient-reported outcome measures (DASH, 12;QuickDASH, 9), was determined through descriptive statistics, Pearson correlation coefficient, intraclass correlation coefficient, and distribution of scores and score differences. Results The CAT model required an average of 15.3 questions to be answered for the DASH and 5.8 questions for the QuickDASH, representing a 49% and 47% decrease in question burden, respectively. Mean CAT score was the same for DASH and 0.1 points lower for QuickDASH with similar SDs (DASH, 12.9 +/- 19.8 vs 12.9 +/- 19.9;QuickDASH, 32.7 +/- 24.7 vs 32.6 +/- 24.6). Pearson coefficients (DASH, 0.99;QuickDASH, 0.98) and intraclass correlation coefficients (DASH, 1.0;QuickDASH, 0.98) indicated strong agreement between scores. The difference between the CAT and full score was less than the minimal clinically important difference in 99% of cases for DASH and approximately 95% of cases for QuickDASH. Conclusions The application of CAT to DASH and QuickDASH surveys demonstrated an ability to lessen the response burden with negligible effect on score integrity. Clinical relevance In the case of DASH and QuickDASH, CAT is an appropriate alternative to

关键词： computerized adaptive testing DASH patient-reported outcomes Artificial intelligence

来源：评论

学校读者我要写书评

暂无评论

Remote Symptom Monitoring With Ecological Momentary computerized adaptive testing: Pilot Cohort Study of a Platform for Frequent, Low-Burden, and Personalized Patient-Reported Outcome Measures

引用

JOURNAL OF MEDICAL INTERNET RESEARCH 2023年第1期25卷 e47179页

作者： Harrison, Conrad Trickett, Ryan Wormald, Justin Dobbs, Thomas Lis, Przemyslaw Popov, Vesselin Beard, David J. Rodrigues, Jeremy Univ Oxford Nuffield Dept Orthopaed Rheumatol & Musculoskelet Surg Intervent Trials Unit Oxford England Cardiff & Vale Univ Hlth Board Cardiff Wales Swansea Univ Welsh Ctr Burns & Plast Surg Swansea Wales Univ Cambridge Psychometr Ctr Cambridge England Warwick Med Sch Clin Trials Unit Coventry England Stoke Mandeville Hosp Dept Plast Surg Aylesbury England Univ Oxford Botnar Res Ctr Nuffield Dept Orthopaed Rheumatol & Musculoskeleta Surg Intervent Trials Unit Old Rd Oxford OX3 7LD England

Background: Remote patient-reported outcome measure (PROM) data capture can provide useful insights into research and clinical practice and deeper insights can be gained by administering assessments more frequently, for example, in ecological momentary assessment. However, frequent data collection can be limited by the burden of multiple, lengthy questionnaires. This burden can be reduced with computerized adaptive testing (CAT) algorithms that select only the most relevant items from a PROM for an individual respondent. In this paper, we propose "ecological momentary computerized adaptive testing" (EMCAT): the use of CAT algorithms to reduce PROM response burden and facilitate high-frequency data capture via a smartphone app. We develop and pilot a smartphone app for performing EMCAT using a popular hand surgery PROM. Objective: The aim of this study is to determine the feasibility of EMCAT as a system for remote PROM administration. Methods: We built the EMCAT web app using Concerto, an open-source CAT platform maintained by the Psychometrics Centre, University of Cambridge, and hosted it on an Amazon Web Service cloud server. The platform is compatible with any questionnaire that has been parameterized with item response theory or Rasch measurement theory. For this study, the PROM we chose was the patient evaluation measure, which is commonly used in hand surgery. CAT algorithms were built using item response theory models derived from UK Hand Registry data. In the pilot study, we enrolled 40 patients with hand trauma or thumb-base arthritis, across 2 sites, between July 13, 2022, and September 14, 2022. We monitored their symptoms with the patient evaluation measure, via EMCAT, over a 12-week period. Patients were assessed thrice weekly, once daily, or thrice daily. We additionally administered full-length PROM assessments at 0, 6, and 12 weeks, and the User Engagement Scale at 12 weeks. Results: The use of EMCAT significantly reduced the length of the PROM (med

关键词： patient-reported outcome measures ecological momentary assessment computerized adaptive testing EMCAT symptom monitoring monitoring assessment smartphone app trauma arthritis usability mobile phone

来源：评论

学校读者我要写书评

暂无评论

Development and Psychometric Evaluation of an Item Bank for computerized adaptive testing of the EORTC Insomnia Dimension in Cancer Patients (EORTC CAT-SL)

引用

APPLIED RESEARCH IN QUALITY OF LIFE 2021年第2期16卷 827-844页

作者： Dirven, Linda Petersen, Morten Aa Aaronson, Neil K. Chie, Wei-Chu Conroy, Thierry Costantini, Anna Hammerlid, Eva Velikova, Galina Verdonck-de Leeuw, Irma M. Young, Teresa Groenvold, Mogens Leiden Univ Dept Neurol Med Ctr POB 9600 NL-2300 RC Leiden Netherlands Haaglanden Med Ctr Dept Neurol The Hague Netherlands Univ Copenhagen Bispebjerg Hosp Dept Palliat Med Res Unit Copenhagen Denmark Netherlands Canc Inst Div Psychosocial Res & Epidemiol Amsterdam Netherlands Natl Taiwan Univ Coll Publ Hlth Inst Epidemiol & Prevent Med Dept Family Med Taipei Taiwan Inst Cancerol Lorraine Med Oncol Dept Vandoeuvre Les Nancy France Sapienza Univ St Andrea Hosp Fac Med & Psychol Psychooncol Unit Rome Italy Gothenburg Univ Sahlgrenska Univ Hosp Dept Otolaryngol Head & Neck Surg Gothenburg Sweden Univ Leeds Leeds Inst Canc & Pathol Fac Med & Hlth Leeds W Yorkshire England Univ Amsterdam Med Ctr Locat VU Univ Canc Ctr AmsterdamDept Otolaryngol Lead & Neck S Amsterdam Netherlands Vrije Univ Amsterdam Publ Hlth Res Inst Dept Clin Neuro & Dev Psychol Amsterdam Netherlands East & North Hertfordshire NHS Trust Incorporatin Northwood Middx England Univ Copenhagen Dept Publ Hlth Copenhagen Denmark

To further advance assessment of patient-reported outcomes, the European Organisation of Research and Treatment of Cancer (EORTC) Quality of Life Group has developed computerized adaptive test (CAT) versions of all EORTC Quality of Life Core Questionnaire (QLQ-C30) scales/items. The aim of this study was to develop and evaluate an item bank for CAT measurement of insomnia (CAT-SL). In line with the EORTC guidelines, the developmental process comprised four phases: (I) defining the concept insomnia and literature search, (II) selection and formulation of new items, (III) pre-testing and (IV) field-testing, including psychometric analyses of the final item bank. In phase I, the literature search identified 155 items that were compatible with our conceptualisation of insomnia, including both quantity and quality of sleep. In phase II, following a multistep-approach, this number was reduced to 15 candidate items. Pre-testing of these items in cancer patients (phase III) resulted in an item list of 14 items, which were field-tested among 1094 patients in phase IV. Psychometric evaluations showed that eight items could be retained in a unidimensional model. The final item bank yielded greater measurement precision than the original QLQ-C30 insomnia item. It was estimated that administering two or more items from the insomnia item bank with CAT results in a saving in sample size between approximately 15-25%. The 8-item EORTC CAT-SL item bank facilitates precise and efficient measurement of insomnia as part of the EORTC CAT system of health-related quality life assessment in both clinical research and practice.

关键词： computerized adaptive testing Cancer Insomnia Sleeplessness Item bank Health-related quality of life EORTC QLQ-C30

来源：评论

学校读者我要写书评

暂无评论

The Block Item Pocket Method for Reviewable Multidimensional computerized adaptive testing

引用

APPLIED PSYCHOLOGICAL MEASUREMENT 2021年第1期45卷 22-36页

作者： Lin, Zhe Chen, Ping Xin, Tao Beijing Normal Univ Sch Psychol Beijing Peoples R China Beijing Normal Univ Collaborat Innovat Ctr Assessment Basic Educ Qual 19 Xin Jie Kou Wai St Beijing 100875 Peoples R China

Most computerized adaptive testing (CAT) programs do not allow item review due to a decrease in estimation precision and aberrant manipulation strategies. In this article, a block item pocket (BIP) method that combines the item pocket method with the successive block method to realize reviewable CAT was proposed. A worst-case but still reasonable answering strategy and the Wainer-like manipulation strategy were simulated to evaluate the estimation precision of reviewable unidimensional computerized adaptive testing (UCAT) and multidimensional computerized adaptive testing (MCAT) under a series of BIP settings. For both UCAT and MCAT, it was found that the estimation precision of the BIP method improved as the number of blocks increased or the item pocket size decreased under the reasonable strategy. The BIP method was more effective in handling the Wainer-like strategy. With the help of block design, the BIP method can still maintain acceptable estimation precision under slightly large total IP size conditions. These results suggested that the BIP method was a reliable solution for both reviewable UCAT and MCAT.

关键词： computerized adaptive testing reviewable CAT item review block item pocket

来源：评论

学校读者我要写书评

暂无评论

The Effects of Item Exposure Control on Measurement Precision of Vocabulary Size Estimates in computerized adaptive testing

引用

ENGLISH TEACHING AND LEARNING 2021年第2期45卷 217-236页

作者： Tseng, Wen-Ta Natl Taiwan Univ Sci & Technol Dept Appl Foreign Languages Taipei Taiwan

Research has shown the viability of computerized adaptive testing (CAT) as a valid and efficient approach to measuring the English vocabulary knowledge. However, the CAT theta of vocabulary knowledge obtained may become inflated and over-estimated when a certain number of vocabulary items are over-exposed. Hence, due to the potential risk of over-exposure of test items, the aim of the study sets out to examine the effects of different levels of item exposure control upon the precision of vocabulary size estimate as well as precision of CAT diagnostic classification rate (pass/fail) regarding a predetermined vocabulary size threshold. In total, 200 Taiwanese junior high school students were recruited to take part in the study. The results of the study showed that both the mean theta values and the mean standard errors among the three CAT exposure control conditions did not statistically differ from one another. Nevertheless, the results of the study further revealed that a strict exposure rate posed a great challenge to stably and precisely reach a statistical decision upon granting pass/fail for the predetermined vocabulary size threshold. Implications for CAT-based vocabulary assessment were further discussed.

关键词： Vocabulary size Exposure control computerized adaptive testing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：