TY - JOUR
T1 - Computer Adaptive vs. Non-adaptive Medical Progress Testing
T2 - Feasibility, Test Performance, and Student Experiences
AU - VAN WIJK, Elise V.
AU - Donkers, Jeroen
AU - DE LAAT, Peter C.J.
AU - Meiboom, Ariadne A.
AU - Jacobs, Bram
AU - Ravesloot, Jan Hindrik
AU - Tio, René A.
AU - VAN DER VLEUTEN, Cees P.M.
AU - Langers, Alexandra M.J.
AU - Bremers, Andre J.A.
N1 - Publisher Copyright:
© 2024 The Author(s).
PY - 2024/7/26
Y1 - 2024/7/26
N2 - Background: Computerized adaptive testing tailors test items to students’ abilities by adapting difficulty level. This more efficient, and reliable assessment form may provide advantages over a conventional medical progress test (PT). Prior to our study, a direct comparison of students’ performance on a computer adaptive progress test (CA-PT) and a conventional PT, which is crucial for nationwide implementation of the CA-PT, was missing. Therefore, we assessed the correlation between CA-PT and conventional PT test performance and explored the feasibility and student experiences of CA-PT in a large medical cohort. Methods: In this cross-over study medical students (n = 1432) of three Dutch medical schools participated in both a conventional PT and CA-PT. They were stratified to start with either a conventional PT or CA-PT to determine test performance. Student motivation, engagement and experiences were assessed by questionnaires in students from seven Dutch medical schools. Parallel-forms reliability was assessed using the Pearson correlation coefficient. Results: A strong correlation was found (0.834) between conventional PT and CA-PT test performance. The CA-PT was administered without system performance issues and was completed in a median time of 83 minutes (67–102 minutes). Questionnaire response rate was 31.7% (526/1658). Despite a higher experienced difficulty, most students reported persistence, adequate task management and good focus during the CA-PT. Conclusions: CA-PT provides a reliable estimation of students’ ability level in less time than a conventional non-adaptive PT and is feasible in students throughout the entire medical curriculum. Despite the strong correlation between PT scores, students found the CA-PT more challenging.
AB - Background: Computerized adaptive testing tailors test items to students’ abilities by adapting difficulty level. This more efficient, and reliable assessment form may provide advantages over a conventional medical progress test (PT). Prior to our study, a direct comparison of students’ performance on a computer adaptive progress test (CA-PT) and a conventional PT, which is crucial for nationwide implementation of the CA-PT, was missing. Therefore, we assessed the correlation between CA-PT and conventional PT test performance and explored the feasibility and student experiences of CA-PT in a large medical cohort. Methods: In this cross-over study medical students (n = 1432) of three Dutch medical schools participated in both a conventional PT and CA-PT. They were stratified to start with either a conventional PT or CA-PT to determine test performance. Student motivation, engagement and experiences were assessed by questionnaires in students from seven Dutch medical schools. Parallel-forms reliability was assessed using the Pearson correlation coefficient. Results: A strong correlation was found (0.834) between conventional PT and CA-PT test performance. The CA-PT was administered without system performance issues and was completed in a median time of 83 minutes (67–102 minutes). Questionnaire response rate was 31.7% (526/1658). Despite a higher experienced difficulty, most students reported persistence, adequate task management and good focus during the CA-PT. Conclusions: CA-PT provides a reliable estimation of students’ ability level in less time than a conventional non-adaptive PT and is feasible in students throughout the entire medical curriculum. Despite the strong correlation between PT scores, students found the CA-PT more challenging.
UR - https://www.scopus.com/pages/publications/85199936068
U2 - 10.5334/pme.1345
DO - 10.5334/pme.1345
M3 - Article
C2 - 39071727
AN - SCOPUS:85199936068
SN - 2212-2761
VL - 13
SP - 406
EP - 416
JO - Perspectives on Medical Education
JF - Perspectives on Medical Education
IS - 1
ER -