Factor analysis in educational settings : a simulation study comparing fit statistics across robust estimators.

In education and social science, data often arise from nested data structures, meaning that students are nested within teachers or schools. Traditional factor analytic approaches to measuring latent traits do not account for the nested structure of these data. The logic and potential issues of using multilevel confirmatory factor analysis were discussed. The ability of commonly used fit statistics to discriminate between a correctly specified model and models with omitted factor loading(s) were investigated with receiver-operating-characteristics (ROC) analyses. Combining ROC analyses with traditional methods of investigating fit statistic performance resulted in converging evidence for the utility of these common fit statistics. In general, these fit statistics performed poorly and should not be heavily relied upon for evidence of the factor structures specified. Recommendations were given for which commonly reported fit statistics to use, cut-off criteria to use for which estimators, and cautions about the use of the suggested cut-off criteria.