Split-sample design with parallel protocols to reduce cost and nonresponse bias in surveys

Abstract

Response rates in household surveys are declining, increasing the risk of nonresponse bias in survey estimates. Survey costs are increasing. As a result, design features such as higher monetary incentives are needed but often cannot be afforded. Two or more survey protocols could be implemented in parallel, where some have lower nonresponse while others have lower cost, as long as the data can be combined in a way that reflects the reduced potential for nonresponse bias under the more intensive protocol. We describe the main barrier to the use of such an approach-that traditional methods ignore the expected lower bias in one condition. The proposed approach includes random assignment of sample members to a data collection protocol and adjustment of survey estimates to the superior protocol, based on key survey variables-if differences in estimates are found. This represents a major departure from the current practice in constructing nonresponse adjustments and leverages the use of the same sampling design, survey instrument, and measurement procedures in each condition. An illustrative example is presented using data from a national survey. Methods to address both bias reduction and variance estimation are described. We end with limitations and suggestions for future research.