Reliability of residents' assessments of their postgraduate medical education learning environment: an observational study

Paul L. P. Brand*, H. Jeroen Rosingh, Maarten A. C. Meijssen, Ingrid M. Nijholt, Saskia Dunnwald, Jelle Prins, Johanna Schonrock-Adema

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

48 Downloads (Pure)

Abstract

Background: Even in anonymous evaluations of a postgraduate medical education (PGME) program, residents may be reluctant to provide an honest evaluation of their PGME program, because they fear embarrassment or repercussions from their supervisors if their anonymity as a respondent is endangered. This study was set up to test the hypothesis that current residents in a PGME program provide more positive evaluations of their PGME program than residents having completed it. We therefore compared PGME learning environment evaluations of current residents in the program to leaving residents having completed it.

Methods: This observational study used data gathered routinely in the quality cycle of PGME programs at two Dutch teaching hospitals to test our hypothesis. At both hospitals, all current PGME residents are requested to complete the Scan of Postgraduate Education Environment Domains (SPEED) annually. Residents leaving the hospital after completion of the PGME program are also asked to complete the SPEED after an exit interview with the hospital's independent residency coordinator. All SPEED evaluations are collected and analysed anonymously. We compared the residents' grades (on a continuous scale ranging from 0 (poor) to 10 (excellent)) on the three SPEED domains (content, atmosphere, and organization of the program) and their mean (overall department grade) between current and leaving residents.

Results: Mean (SD) overall SPEED department grades were 8.00 (0.52) for 287 current residents in 39 PGME programs and 8.07 (0.48) for 170 leaving residents in 39 programs. Neither the overall SPEED department grades (t test, p = 0.53, 95% CI for difference -0.16 to 0.31) nor the department SPEED domain grades (MANOVA, F(3, 62) = 0.79, p = 0.51) were significantly different between current and leaving residents.

Conclusions: Residents leaving the program did not provide more critical evaluations of their PGME learning environment than current residents in the program. This suggests that current residents' evaluations of their postgraduate learning environment were not affected by social desirability bias or fear of repercussions from faculty.

Original languageEnglish
Article number450
Number of pages7
JournalBMC Medical Education
Volume19
Issue number1
DOIs
Publication statusPublished - 3-Dec-2019

Keywords

  • Learning environment
  • Postgraduate medical education
  • Quality cycle
  • SPEED
  • adult
  • article
  • atmosphere
  • controlled study
  • fear
  • female
  • human
  • human experiment
  • interview
  • learning environment
  • male
  • multicenter study
  • multivariate analysis of variance
  • observational study
  • postgraduate education
  • reliability
  • resident
  • social desirability bias
  • teaching hospital

Cite this