Design Effects

Most large demographic survey are based on complex sampling designs which involve clustering of respondents, proportional and disproportionate stratification, and the use of post-stratification weights to adjust for nonresponse and coverage errors in the survey. These features of the survey designs need to be taken into account in the analysis of the data from the survey to yield unbiased estimates of effects and the stand errors and significance levels of these effects. Statistical software is available in most of the main statistical packages (e.g, SAS, Stata, Mplus) that allows the researcher to take into account the design for an increasing number of statistical procedures. Many large datasets (e.g., The American Community Survey and NHANES) now contain a set of 30 to 100 replicate weights to account for the design effects. 

By focusing on how to handle sampling design in surveys commonly used by demographers the PRI Demographic Methods Focus for 2009/2010 expects to increase the proficiency of the statistical programmers in advising and setting up the statistical procedures to correctly account for these design features in the analyses conducted for researchers. We also plan to provide training and resources for researcher and graduate students conducting their own analyses.