检索结果-内蒙古大学图书馆

ROAR-CAT: Rapid Online Assessment of Reading ability with computerized adaptive testing

BEHAVIOR RESEARCH METHODS 2025年第1期57卷 1-17页

作者： Ma, Wanjing Anya Richie-Halford, Adam Burkhardt, Amy K. Kanopka, Klint Chou, Clementine Domingue, Benjamin W. Yeatman, Jason D. Stanford Univ Grad Sch Educ 520 Galvez Mall Stanford CA 94305 USA Stanford Univ Sch Med Div Dev & Behav Pediat Stanford CA USA Stanford Univ Dept Psychol Stanford CA USA NYU New York NY USA Cambium Assessment Washington DC USA

The Rapid Online Assessment of Reading (ROAR) is a web-based lexical decision task that measures single-word reading abilities in children and adults without a proctor. Here we study whether item response theory (IRT) and computerized adaptive testing (CAT) can be used to create a more efficient online measure of word recognition. To construct an item bank, we first analyzed data taken from four groups of students (N = 1960) who differed in age, socioeconomic status, and language-based learning disabilities. The majority of item parameters were highly consistent across groups (r = .78-.94), and six items that functioned differently across groups were removed. Next, we implemented a JavaScript CAT algorithm and conducted a validation experiment with 485 students in grades 1-8 who were randomly assigned to complete trials of all items in the item bank in either (a) a random order or (b) a CAT order. We found that, to achieve reliability of 0.9, CAT improved test efficiency by 40%: 75 CAT items produced the same standard error of measurement as 125 items in a random order. Subsequent validation in 32 public school classrooms showed that an approximately 3-min ROAR-CAT can achieve high correlations (r = .89 for first grade, r = .73 for second grade) with alternative 5-15-min individually proctored oral reading assessments. Our findings suggest that ROAR-CAT is a promising tool for efficiently and accurately measuring single-word reading ability. Furthermore, our development process serves as a model for creating adaptive online assessments that bridge research and practice.

关键词： Dyslexia Screening tools Psychometrics computerized adaptive testing Lexical decision task Word recognition

来源：评论

学校读者我要写书评

暂无评论

Measuring glaucoma quality of life in an Asian population using item banking: psychometric evaluation and computerized adaptive testing simulations

引用

QUALITY OF LIFE RESEARCH 2023年第9期32卷 2667-2679页

作者： Fenwick, Eva K. Lim, Belicia Man, Ryan E. K. Baskaran, Mani Nongpiur, Monisha Sng, Chelvin C. A. Iyer, Jayant Venkatramani Husain, Rahat Perera, Shamira Wong, Tina Low, Jin Rong Loe, Bao Sheng Huang, Olivia Shimin Lun, Katherine Aung, Tin Lamoureux, Ecosse L. Academia Singapore Eye Res Inst Singapore Natl Eye Ctr 20 Coll RdLevel 6 Singapore 169856 Singapore Natl Univ Singapore Duke NUS Med Sch Singapore Singapore Sankara Nethralaya Med & Vis Res Fdn Chennai India Natl Univ Hlth Syst Singapore Singapore Univ Cambridge Sch Psychol Cambridge England Natl Univ Singapore Dept Ophthalmol Singapore Singapore Univ Melbourne Melbourne Australia

PurposeTo assess the psychometric properties of glaucoma-specific health-related quality of life (HRQoL) item banks (IBs), and explore their efficiency using computerized adaptive testing (CAT) *** this cross-sectional, clinical study, 300 Asian glaucoma patients answered 221 items within seven IBs: Ocular Comfort Symptoms (OS);Activity Limitation (AL);Lighting (LT);Mobility (MB);Glaucoma Management (GM);Psychosocial (PSY);and Work (WK). Rasch analysis was conducted to assess each IB's psychometric properties (e.g., item "fit" to the construct;unidimensionality) and a set of analytic performance criteria guiding decision making relating to retaining or dropping domains and items was employed. CAT simulations determined the mean number of items for 'high' and 'moderate' measurement precision (stopping rule: SEM 0.3 and 0.387, respectively).ResultsParticipants' mean age was 67.2 +/- 9.2 years (62% male;87% Chinese). LT, MB, and GM displayed good psychometric properties overall. To optimize AL's psychometric properties, 16 items were deleted due to poor "fit", high missing data, item bias, low discrimination and/or a low clinical/patient importance rating. To resolve multidimensionality in PSY, we rehomed 16 items into a "Concern (CN)" domain. PSY and CN required further amendment, including collapsing of response categories, and removal of poorly functioning items (N = 7). Due to poor measurement precision, low applicability and high ceiling effect, low test information indices, and low item separation index the WK IB was not considered further. In CAT simulations on the final seven IBs (n = 182 items total), an average of 12.1 and 15.7 items per IB were required for moderate and high precision measurement, *** reengineering our seven IBs, they displayed robust psychometric properties and good efficiency in CAT simulations. Once finalized, GlauCAT (TM)-Asian may enable comprehensive assessment of the HRQoL impact of glaucoma

关键词： Glaucoma Quality of life Item banks Rasch analysis computerized adaptive testing

来源：评论

学校读者我要写书评

暂无评论

Generalizing computerized adaptive testing for problematic mobile phone use from Chinese adults to adolescents

引用

CURRENT PSYCHOLOGY 2024年第16期43卷 14148-14158页

作者： Lei, Guo Xiaorui, Liu Tour, Liu Southwest Univ Fac Psychol Chongqing Peoples R China Southwest Univ Branch Collaborat Innovat Ctr Assessment Basic Educ Qual Chongqing Peoples R China Tianjin Normal Univ Acad Psychol & Behav Key Res Base Humanities & Social Sci Minist Educ Tianjin Peoples R China Tianjin Normal Univ Fac Psychol Tianjin Peoples R China Tianjin Social Sci Lab Students Mental Dev & Learn Tianjin Peoples R China

The number of mobile phone users worldwide has increased in recent years. As people spend more time on their phones, negative effects such as problematic mobile phone use (PMPU) have become more pronounced. Many researchers have dedicated their efforts to developing the questionnaires and revising the tools to more accurately evaluate PMPU. Previous studies had demonstrated that CAT-PMPU for adults could significantly enhance measurement accuracy and efficiency. However, most of the items in this scenario was developed for adults, and there were notable differences between adults and adolescents, making some items potentially unsuitable for the latter. Thus, this study aimed to generalize the CAT-PMPU adult version to make it suitable for both adult and adolescent populations. A total of 740 Chinese adolescents and 980 Chinese adults participated in this study, completing online or paper-pencil questionnaires. Empirical data was then used to simulate CAT-PMPU, and the measurement efficiency, accuracy and reliability were compared between adults and adolescents in different stopping rules. The results showed that generalized CAT-PMPU in this study had promising measurement efficiency and accuracy, which was consistent with adult version. In conclusion, the CAT-PMPU developed in this study not only exhibited satisfactory test reliability but also emerged as a novel technical support for evaluation of PMPU in both adolescent and adult populations, which demonstrated the potential applicability in practice.

关键词： Problematic mobile phone use computerized adaptive testing Item banking Item response theory Cross-sample study

来源：评论

学校读者我要写书评

暂无评论

Accounting for item calibration error in computerized adaptive testing

引用

BEHAVIOR RESEARCH METHODS 2025年第5期57卷 1-12页

作者： Fink, Aron Koenig, Christoph Frey, Andreas Goethe Univ Frankfurt Theodor W Adorno Pl 6 D-60323 Frankfurt Germany

In computerized adaptive testing (CAT), item parameter estimates derived from calibration studies are considered to be known and are used as fixed values for adaptive item selection and ability estimation. This is not completely accurate because these item parameter estimates contain a certain degree of error. If this error is random, the typical CAT procedure leads to standard errors of the final ability estimates that are too small. If the calibration error is large, it has been shown that the accuracy of the ability estimates is negatively affected due to the capitalization on chance problem, especially for extreme ability levels. In order to find a solution for this fundamental problem of CAT, we conducted a Monte Carlo simulation study to examine three approaches that can be used to consider the uncertainty of item parameter estimates in CAT. The first two approaches used a measurement error modeling approach in which item parameters were treated as covariates that contained errors. The third approach was fully Bayesian. Each of the approaches was compared with regard to the quality of the resulting ability estimates. The results indicate that each of the three approaches is capable of reducing bias and the mean squared error (MSE) of the ability estimates, especially for high item calibration errors. The Bayesian approach clearly outperformed the other approaches. We recommend the Bayesian approach, especially for application areas in which the recruitment of a large calibration sample is infeasible.

关键词： computerized adaptive testing Item response theory Bayesian techniques Calibration error

来源：评论

学校读者我要写书评

暂无评论

A novel computerized adaptive testing framework with decoupled learning selector

引用

COMPLEX & INTELLIGENT SYSTEMS 2023年第5期9卷 5555-5566页

作者： Ma, Haiping Zeng, Yi Yang, Shangshang Qin, Chuan Zhang, Xingyi Zhang, Limiao Anhui Univ Inst Phys Sci & Informat Technol Hefei 230601 Anhui Peoples R China Hefei Comprehens Natl Sci Ctr Inst Artificial Intelligence Hefei 230088 Anhui Peoples R China Anhui Univ Sch Comp Sci & Technol Hefei 230601 Anhui Peoples R China Baidu Inc Beijing Baidu Talent Intelligence Ctr Beijing 10085 Peoples R China Anhui Univ Informat Mat & Intelligent Sensing Lab Anhui Prov Hefei 230601 Anhui Peoples R China

computerized adaptive testing (CAT) targets to accurately assess the student's proficiency in the required subject/area. The key issue is how to design a question selector that adaptively selects the best-suited questions for each student based on previous performance step by step. Most existing question selectors execute via greedy metric functions (e.g., question information and uncertainty), which can not effectively capture data characteristics. There also exist learning-based question selectors that redefine the CAT problem as a bilevel optimization problem, where the parameter learning of the question selector and the student proficiency estimation model are coupled, which is not flexible enough. To this end, in this paper, we propose a novel CAT framework with Decoupled Learning selector (DL-CAT). Specifically, we first use the currently estimated student ability and question characteristics as input and design a deep learning-based question selector to predict question selection scores. Then, to address the issue that there is no ground truth to measure the quality of the selected question, an approximate ground-truth and a pairwise rank loss function are specially designed to update the parameters of the question selector independently. Extensive experiments on two real datasets demonstrate that our proposed DL-CAT has certain advantages in effectiveness and significant advantages in efficiency.

关键词： computerized adaptive testing Question selector Student assessment Learning selector

来源：评论

学校读者我要写书评

暂无评论

The General Psychopathology 'p' Factor in Adolescence: Multi-Informant Assessment and computerized adaptive testing

引用

RESEARCH ON CHILD AND ADOLESCENT PSYCHOPATHOLOGY 2024年第11期52卷 1753-1764页

作者： Jones, Jason D. Boyd, Rhonda C. Sandro, Akira Di Calkins, Monica E. De Los Reyes, Andres Barzilay, Ran Young, Jami F. Benton, Tami D. Gur, Ruben C. Moore, Tyler M. Gur, Raquel E. Childrens Hosp Philadelphia Roberts Ctr Pediat Res Dept Child & Adolescent Psychiat & Behav Sci 2716 South St Philadelphia PA 19146 USA Univ Penn Perelman Sch Med Dept Psychiat Philadelphia PA 19104 USA Univ Maryland Dept Psychol College Pk MD USA

Accumulating evidence supports the presence of a general psychopathology dimension, the p factor ('p'). Despite growing interest in the p factor, questions remain about how p is assessed. Although multi-informant assessment of psychopathology is commonplace in clinical research and practice with children and adolescents, almost no research has taken a multi-informant approach to studying youth p or has examined the degree of concordance between parent and youth reports. Further, estimating p requires assessment of a large number of symptoms, resulting in high reporter burden that may not be feasible in many clinical and research settings. In the present study, we used bifactor multidimensional item response theory models to estimate parent- and adolescent-reported p in a large community sample of youth (11-17 years) and parents (N = 5,060 dyads). We examined agreement between parent and youth p scores and associations with assessor-rated youth global functioning. We also applied computerized adaptive testing (CAT) simulations to parent and youth reports to determine whether adaptive testing substantially alters agreement on p or associations with youth global functioning. Parent-youth agreement on p was moderate (r =.44) and both reports were negatively associated with youth global functioning. Notably, 7 out of 10 of the highest loading items were common across reporters. CAT reduced the average number of items administered by 57%. Agreement between CAT-derived p scores was similar to the full form (r =.40) and CAT scores were negatively correlated with youth functioning. These novel results highlight the promise and potential clinical utility of a multi-informant p factor approach.

关键词： Psychopathology P Factor Multi-informant Discrepancies computerized adaptive testing

来源：评论

学校读者我要写书评

暂无评论

Differential Performance of computerized adaptive testing in Students With and Without Disabilities - A Simulation Study

引用

JOURNAL OF SPECIAL EDUCATION TECHNOLOGY 2024年第4期39卷 481-490页

作者： Ebenbeck, Nikola Gebhardt, Markus Univ Munich Munich Germany Univ Munich Leopoldstr 13 D-80802 Munich Germany

Technologies that enable individualization for students have significant potential in special education. computerized adaptive testing (CAT) refers to digital assessments that automatically adjust their difficulty level based on students' abilities, allowing for personalized, efficient, and accurate measurement. This article examines whether CAT performs differently for students with and without special educational needs (SEN). Two simulation studies were conducted using a sample of 709 third-grade students from general and special schools in Germany, who took a reading test. The results indicate that students with SEN were assessed with fewer items, reduced bias, and higher accuracy compared to students without SEN. However, measurement accuracy decreased, and test length increased for students whose abilities deviated more than two SD from the norm. We discuss potential adaptations of CAT for students with SEN in the classroom, as well as the integration of CAT with AI-supported feedback and tailored exercises within a digital learning environment.

关键词： computerized adaptive testing special educational needs individualization assessment

来源：评论

学校读者我要写书评

暂无评论

Utilizing Real-Time Test Data to Solve Attenuation Paradox in computerized adaptive testing to Enhance Optimal Design

引用

Journal of Educational and Behavioral Statistics 2024年第4期49卷 630-657页

作者： Chen, Jyun-Hong Chao, Hsiu-Yi National Cheng Kung University Taiwan Soochow University China

To solve the attenuation paradox in computerized adaptive testing (CAT), this study proposes an item selection method, the integer programming approach based on real-time test data (IPRD), to improve test efficiency. The IPRD method turns information regarding the ability distribution of the population from real-time test data into feasible test constraints to reversely assembled shadow tests for item selection to prevent the attenuation paradox by integer programming. A simulation study was conducted to thoroughly investigate IPRD performance. The results indicate that the IPRD method can efficiently improve CAT performance in terms of the precision of trait estimation and satisfaction of all required test constraints, especially for conditions with stringent exposure control. © 2023 The Author(s).

关键词： attenuation paradox computerized adaptive testing integer programming optimal design shadow test

来源：评论

学校读者我要写书评

暂无评论

Novel item selection strategies for cognitive diagnostic computerized adaptive testing: A heuristic search framework

引用

BEHAVIOR RESEARCH METHODS 2024年第4期56卷 2859-2885页

作者： Cao, Xi Lin, Ying Liu, Dong Zheng, Fudan Duh, Henry Been-Lirn La Trobe Univ Dept Comp Sci & Informat Technol Melbourne Australia Sun Yat Sen Univ Dept Psychol Guangzhou Peoples R China Henan Normal Univ Coll Comp & Informat Engn Xinxiang Peoples R China Key Lab Artificial Intelligence & Personalized Lea Xinxiang Peoples R China Guangzhou City Univ Technol Sch Comp Engn Guangzhou 510800 Peoples R China Hong Kong Polytech Univ PolyU NVIDIA Joint Res Ctr Hong Kong Peoples R China

The computerized adaptive form of cognitive diagnostic testing, CD-CAT, has gained increasing attention in the domain of personalized measurements for its ability to categorize individual mastery status of fine-grained attributes more accurately and efficiently through administering items tailored to one's ability progressively. How to select the next item based on previous response(s) is crucial for the success of CD-CAT. Previous item selection strategies for CD-CAT have often followed a greedy or semi-greedy approach, which makes it difficult to strike a balance between diagnostic performance and item bank utilization. To address this issue, this study takes a graph perspective and transforms the item selection problem in CD-CAT into a path-searching problem, in which paths refer to possible test construction and nodes refer to individual items. A heuristic function is defined to predict the prospect of a path, indicating how well the corresponding test can diagnose the current examinee. Two search mechanisms with different biases towards item exposure control are proposed to approximate the optimal path with the best prospect. The first unused item on the resulting path is selected as the next item. The above components compose a novel CD-CAT item selection framework based on heuristic search. Simulation studies are conducted under a variety of conditions regarding bank designs, bank-quality conditions, and testing scenarios. The results are compared with different types of classic item selection strategies in CD-CAT, showing that the proposed framework can enhance bank utilization at a smaller cost of diagnostic performance.

关键词： Cognitive diagnostic testing computerized adaptive testing Heuristic search

来源：评论

学校读者我要写书评

暂无评论

Improving the Item Selection Process with Reinforcement Learning in computerized adaptive testing 24th

Improving the Item Selection Process with Reinforcement Lear...

引用

24th International Conference on Artificial Intelligence in Education (AIED)

作者： Pian, Yang Chen, Penghe Lu, Yu Song, Guangchen Chen, Pengtao Beijing Normal Univ Fac Educ Sch Educ Technol Beijing 100875 Peoples R China Beijing Normal Univ Adv Innovat Ctr Future Educ Beijing 100875 Peoples R China China Met Bur Geophys Explorat Acad Beijing 071051 Peoples R China

ISBN: (纸本)9783031363351;9783031363368

Item selection is the key process for computerized adaptive testing (CAT) to effectively assess examinees' knowledge states. Existing item selection algorithms mainly rely on information metrics, suffering two issues: one is that the implicit cognitive information like relations between testing items as well as knowledge components cannot be captured by the information-based methods, and the other one is that the information-based algorithms computes item's suitableness depending on examinees' knowledge states which are estimated and imprecise inherently. To address these two issues, this work proposes to employ reinforcement learning technology to learn the item selection algorithm automatically in a data-driven manner. It is also able to properly capture the implicit cognitive relations between different testing items and avoid unnecessary item testing, and does not depend on examinees' estimated knowledge states at all.

关键词： Reinforcement learning computerized adaptive testing Item selection User simulator Knowledge space

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：