Designing Studies to Detect Impacts on Earnings
Abstract
This article reports empirical evidence to support the design of evaluations that estimate the impacts of programs that provide postsecondary credentials and/or job training on earnings. Statistical power analyses are strengthened by having accurate empirical estimates of the standard deviation of earnings, share of earnings variance explained by covariates (R2), and, for some designs, the intra-class correlation (ICC). We compute and report values of these inputs for a large sample of control group members from three large studies of such programs. Using our estimated properties of quarterly earnings, we calculate the minimum sample size needed to detect an earnings impact of reasonable magnitude. This calculation demonstrates that many recently published experimental program evaluations have samples sufficient to detect only very large earnings impacts.
Citation