Impact
Creating a self-administered version of a key national energy survey
Since 1978, the U.S. Energy Information Administration (EIA) has conducted the Residential Energy Consumption Survey (RECS), a periodic study of home energy usage based on a sample of residences across the United States. Each iteration of RECS produces a wealth of data: for example, the 2015 survey showed that nearly one-third of U.S. households faced challenges paying energy bills or struggled to adequately heat or cool their homes, and that the number of TVs per household actually decreased compared to 2009 (from 2.6 to 2.3).
The 2015 RECS survey marked a major watershed: for the first time, with the help of RTI researchers, EIA supplemented in-person data acquisition with self-administered web and paper surveys. The fact that this transition was successful represents a huge cost savings potential for the government—an in-person survey costs about $400 per household, compared to a fraction of that for a self-administered survey—and also allows for the collection of energy usage data from a wider, more representative sample of U.S. residents.
Creating a Pilot Program for the Self-Administered RECS Survey
The switch from in-person to self-administered data acquisition in the 2015 RECS had an element of serendipity. Originally, our team collaborated with EIA for two functions: to perform traditional in-person interviews with the full RECS sample, and to conduct a pilot program for the self-administered survey, the results of which would presumably be applied to the next iteration of RECS.
Developing an effective, easy-to-complete, self-administered survey (especially one as complicated as RECS) requires a lot of nuance and attention to detail. In the first phase of our pilot test, conducted in five cities, we determined that the in-person RECS survey, which took over an hour to complete, should ideally be shortened to 30 minutes for the self-administered version, and we gleaned some important insights about the preference of households for web or paper versions of the survey. We also performed extensive cognitive research and compared estimates between self-administered, interviewer-administered, and external benchmarks to assure similarity in the information gathered.
We then rolled out the 30-minute self-administered survey decided on in the city pilot test for a national pilot test involving thousands of households. The goal of this larger test was twofold: to hit upon the right incentive structure to encourage households to complete the survey, and to figure out a way to “push” respondents toward the web rather than the paper version of the survey. Of the eight different protocols we tested, the most successful consisted of a $5 cash pre-incentive, a $10 cash incentive for completing the paper survey, and a $20 cash incentive for completing the web survey, a system we called “Choice Plus.”
Compared to the meta-issues involved in implementing the self-administered RECS survey, choosing and sequencing the actual questions was relatively straightforward. However, this still involved a fair amount of work on our part. Here is a typical question from the “air conditioning” section of the survey:
Which of the following best describes how your household controls your central air conditioning system most of the time?
--Set one temperature and leave it there most of the time
--Manually adjust the temperature at night or when no one is at home
--Program the thermostat to automatically adjust the temperature during the day and night at certain times
--Turn equipment on or off as needed
--Our household does not have control over the equipment
--Other (please specify):
A Benefit of Good Planning: The Switch to Self-Administered RECS
Our pilot testing for the self-administered RECS survey transpired at the same time as our in-person data acquisition efforts. But then, about halfway through the 2015 RECS data acquisition period, EIA made a momentous decision: a full switch from in-person to self-administered surveys. Our self-administered survey had proved to be more economical, and just as easy to complete, as the in-person version, and it also avoided the costs and difficulties of scheduling and completing in-person visits. We are confident that the protocols we helped develop for the 2015 RECS will continue to be improved in future versions, thus allowing EIA to better track energy usage nationwide.
