TY - JOUR

T1 - Bias-Variance Trade-Off in Continuous Test Norming

AU - Voncken, Lieke

AU - Albers, Casper J.

AU - Timmerman, Marieke E.

PY - 2021/12/1

Y1 - 2021/12/1

N2 - In continuous test norming, the test score distribution is estimated as a continuous function of predictor(s). A flexible approach for norm estimation is the use of generalized additive models for location, scale, and shape. It is unknown how sensitive their estimates are to model flexibility and sample size. Generally, a flexible model that fits at the population level has smaller bias than its restricted nonfitting version, yet it has larger sampling variability. We investigated how model flexibility relates to bias, variance, and total variability in estimates of normalizedzscores under empirically relevant conditions, involving the skew Studenttand normal distributions as population distributions. We considered both transversal and longitudinal assumption violations. We found that models with too strict distributional assumptions yield biased estimates, whereas too flexible models yield increased variance. The skew Studenttdistribution, unlike the Box-Cox Power Exponential distribution, appeared problematic to estimate for normally distributed data. Recommendations for empirical norming practice are provided.

AB - In continuous test norming, the test score distribution is estimated as a continuous function of predictor(s). A flexible approach for norm estimation is the use of generalized additive models for location, scale, and shape. It is unknown how sensitive their estimates are to model flexibility and sample size. Generally, a flexible model that fits at the population level has smaller bias than its restricted nonfitting version, yet it has larger sampling variability. We investigated how model flexibility relates to bias, variance, and total variability in estimates of normalizedzscores under empirically relevant conditions, involving the skew Studenttand normal distributions as population distributions. We considered both transversal and longitudinal assumption violations. We found that models with too strict distributional assumptions yield biased estimates, whereas too flexible models yield increased variance. The skew Studenttdistribution, unlike the Box-Cox Power Exponential distribution, appeared problematic to estimate for normally distributed data. Recommendations for empirical norming practice are provided.

KW - assumption violations

KW - GAMLSS

KW - model assumptions

KW - model flexibility

KW - skew Studenttdistribution

KW - standard linear regression model

KW - REGRESSION

KW - ABILITIES

KW - GROWTH

U2 - 10.1177/1073191120939155

DO - 10.1177/1073191120939155

M3 - Article

SN - 1073-1911

VL - 28

SP - 1932

EP - 1948

JO - Assessment

JF - Assessment

IS - 8

ER -