Assessment of physician competency in patient education: Reliability and validity of a model-based instrument

Objective: Establish the inter-rater reliability and the concept, convergent and construct validity of an instrument for assessing the competency of physicians in patient education.

Methods: Three raters assessed the quality of patient education in 30 outpatient consultations with the CELI instrument. This instrument is based on a goal-directed model of patient education and assesses distinctive skills for patient education categorized in four subcompetencies. The inter-rater reliability was calculated. The concept validity was explored by factor analysis. The convergent validity was established by a comparison with two measures of patient-centred behaviour. The construct validity was explored by relating the subcompetencies with physician gender and patient satisfaction.

Results: The inter-rater reliability for the subcompetencies varied between 0.65 and 0.91. The factor analysis distinguished the four subcompetencies. All subcompetencies correlated with the measures of patient-centred behaviour. Female physicians performed better than male physicians on three subcompetencies. Positive correlations were found for three subcompetencies and patient satisfaction.

Conclusion: The CELI instrument appears to be a reliable and valid instrument. However, further research is needed to establish the generalizability and construct validity.

Practice implication: The CELI instrument is a useful tool for assessment and feedback in medical education since it assesses the performance of distinctive skills. (C) 2010 Elsevier Ireland Ltd. All rights reserved.