Scale construction and evaluation in practice: A review of factor analysis versus item response theory applications

Research output: Contribution to journalArticleAcademicpeer-review

157 Downloads (Pure)


In scale construction and evaluation, factor analysis (FA) and item response theory (IRT) are two methods frequently used to determine whether a set of items reliably measures a latent variable. In a review of 41 published studies we examined which methodology – FA or IRT – was used, and what researchers’ motivations were for applying either method. Characteristics of the studies were compared to gain more insight into the practice of scale analysis. Findings indicate that FA is applied far more often than IRT. Many times it is unclear whether the data justify the chosen method because model assumptions are neglected. We recommended that researchers (a) use substantive knowledge about the items to their advantage by more frequently employing confirmatory techniques, as well as adding item content and interpretability of factors to the criteria in model evaluation; and (b) investigate model assumptions and report corresponding findings. To this end, we recommend more collaboration between substantive researchers and statisticians/psychometricians.
Original languageEnglish
Pages (from-to)272-297
Number of pages26
JournalPsychological Test and Assessment Modeling
Issue number3
Publication statusPublished - 2010


  • factor analysis
  • scale evaluation
  • test construction
  • item response theory

Cite this