Stereotype threat research and the assumptions underlying analysis of covariance

Comments on an article by Paul Sackett, Chaitra Hardison and Michael Cullen entitled On Interpreting Stereotype Threat as Accounting for African American-White Differences on Cognitive Tests. Sackett, Hardison, and Cullen discussed the role of covariates in Steele and Aronson's seminal research on the effects of stereotype threat on scores of African American test takers. Besides highlighting some common misinterpretations that stem from the use of covariance-adjusted means in reporting Steele and Aronson's (Study 2) experimental results, Sackett et al. argued that these results indicate that Black-White testscore difference within the no-stereotype threat (i.e., nondiagnostic) condition actually reflects the test score difference on the SAT (i.e., the covariate). This implies that stereotype threat effects add to the often found Black-White test score gap instead of partly accounting for it (Sackett et al., 2004). Here the author comments on the use of analysis of covariance (ANCOVA) in stereotype threat (ST) experiments, because ST theory implies violations of the assumptions underlying ANCOVA. Such violations could result in incorrect Type I error rates and distortions in the adjustment of means. Because of this, ANCOVA appears inappropriate for analyzing (quasi-) experimental results of ST research. In addition, the interpretation proposed by Sackett et al. of Steele and Aronson's results may be due to distortions of mean adjustments caused by violations of model assumptions. While avoiding technical detail, the author provides the assumptions underlying ANCOVA and discuss why these assumptions do not sit well with several aspects of ST theory.