OMB race and ethnicity cognitive testing

Abstract

This study was designed to recruit participants for and conduct cognitive testing of the Office of Management and Budget’s (OMB’s) initial set of recommended revisions to the statistical standards for collecting race and ethnicity data (88 FR 5373). Updates to the race and ethnicity question included (1) collecting race and ethnicity together, with a single question; (2) adding a response category for “Middle Eastern or North African,” separate from the “White” category; and (3) updating terminology, definitions, and question wording. To cognitively test these changes, the RTI International/Research Support Services (RSS) team conducted 100 interviews—80 interviews with English-speaking participants and 20 interviews with Spanish-speaking participants.

Testing examined how participants interacted with and responded to two versions of the combined race and ethnicity question. Half of the participants answered Version A first and half of the participants answered version B first. Version A was programmed as a vertical unfolding question—when participants selected one of the race and ethnicity minimum categories, the detailed categories would immediately open beneath the selected response options. Version B was programmed as a two-page question—when participants selected one of the race and ethnicity minimum categories, they would click “Next” to advance to the next page where they would answer the detailed categories question for each race/ethnicity category they selected in the minimum categories question.

Overall, testing revealed that the combined race/ethnicity question performed well. Most participants had no difficulty in understanding the question or selecting a response for themselves and others in their household. Additionally, participants displayed a strong understanding of the question instructions (i.e., “Select all that apply,” “Provide details below,” and “Enter, for example…”). When participants did struggle to select a response, it was commonly because they were either unsure of (1) some of their racial/ethnic background, (2) how far back in their ancestry to report, (3) the racial/ethnic background of non-family members who lived in their household, or (4) how to report for U.S.-born children of immigrants among Spanish-speaking participants.

Participants who identified as Middle Eastern or North African (MENA), provided positive feedback on the inclusion of this category as a response option and felt they were able to identify themselves more accurately within the response options. Participants who identified as Hispanic or Latino expressed a similar appreciation for the inclusion of this response option as part of the race/ethnicity question as opposed to a separate question. More generally, participants liked that they were able to select multiple response options and write in a description of their background if they did not see themselves represented in the listed response options of the detailed categories question.

Though some participants expressed preferences for one version of the question over another, there were no differences in how participants answered the two versions of the question. Both versions performed similarly well, and participants did not change their responses when answering the alternate version of the question. Based on the findings detailed in this report, we present limited recommendations for changes to the combined race/ethnicity question and also outline some potential areas for future research.