Operationalizing Equity in Evaluation Work with the Colorado Department of Public Health and Environment

Both RTI and CDPHE have prioritized principles of equity, diversity, inclusion, and belonging (EDIB) throughout this evaluation in line with the values of both organizations.  CDPHE’s Office of Health Equity’s mission is to “build partnerships to mobilize community power and transform systems to advance health equity and environmental justice.” Similarly, RTI’s Transformative Research Unit for Equity serves as a leader in RTI’s commitment to center equity “in our values as researchers, in the process of conducting research, and in the outcomes of our research.”

From the beginning of this evaluation, EDIB has been critical to both CDPHE and RTI to ensure that the campaign benefits LGBTQIA+ youth and other youth that experience higher rates of substance use and lower levels of social support. With this goal in mind, we created systemic processes to prioritize an equity-centered framework in all aspects of the project. This includes an intentionality around topics of racial equity, LGBTQIA+ equity, accessibility equity, data equity, etc. Some examples include:

1) Forming specific project roles to guide equity conversations throughout the project

The project investigators created two specific roles that would help guide this work: a data equity specialist and a community expert. RTI staff with personal and professional expertise working with LGBTQIA+ communities filled these roles. Although the project budget did not permit the establishment of a full community advisory board in line with principles of community based participatory research, these two roles ensured that there were project experts with dedicated time to create, review, and improve evaluation processes through an equity- and community-based lens. 

The data equity specialist led conversations, created documentation, and provided guidance and insights into how the project could center equity throughout the project’s lifespan, including study design, study measures development, data collection and analysis, and deliverable development. The community expert served as a liaison between the RTI project team and focus group participants recruited for qualitative interviews. 

2) Forming a project team with personal and professional experience working with LGBTQIA+ communities

In addition to the data equity specialist and community expert roles, the RTI project team was comprised mostly of staff who have personal and professional experience working with LGBTQIA+ communities and/or share LGBTQIA+ identities. Representation of LGBTQIA+ staff on our team ensured that the work was being created, conducted, and reviewed by researchers that shared identities with the participants.

In this way, we were attuned to the various priorities of LGBTQIA+ participants and able to provide meaningful insights into the development of study materials and interpretation of findings. 

3) Creating space to have regular team conversations about how to operationalize equity  

Prior to the first team meeting, the data equity specialist compiled discussion points to guide a conversation about how the team would define principles like equity, intersectionality, and community-based research for this project, drawing from sources like We All Count’s Data Equity Framework and the Racial Equity Institute’s The Groundwater Approach. 

During that meeting, we discussed examining  our  implicit biases and taking actionable steps to distribute the direction of decision-making power among individuals connected to the work (e.g., CDPHE, RTI, focus group participants, etc.). We brainstormed a list of specific ways we could build equity into project processes and deliverables, such as creating  time for the data equity specialist and community expert to review materials and providing focus group participants with the opportunity to review and discuss findings from the evaluation (see numbers 4 and 5 below).

4) Developing timelines that would allow for study procedures and deliverables to be reviewed specifically in the context of EDIB 

Recognizing that many people contribute to the creation of documents like surveys, focus group moderator guides, and reports, we made time for the data equity specialist and community expert to review materials and propose suggestions to ensure that the study deliverables embodied the principles of equity. 

From this process, the interorganizational team was able to make meaningful changes, such as creating more inclusive survey measures, adding language to moderator guides to establish a safe, welcoming space for participants, and reframing findings using strength-based language instead of deficit-framing language. 

5) Examining how RTI could meaningfully involve participant voices in the evaluation process given limited funds

Drawing on the principles of community-based research practices, we gave focus group participants the opportunity  to review the research findings through a compensated review session with RTI moderators.  From this session, we received helpful feedback about conclusions that could be prioritized and interpretations that could be reframed to capture the community’s sentiments more accurately. After making revisions, we shared the findings back with focus group participants so that they could see how their perspectives were incorporated and presented and how they contributed to the evaluation. 

Prioritizing Equity in Evaluation Work

Although not an exhaustive list, these five processes represent some of the steps that our team took to prioritize equity in this evaluation. We found that these steps helped strengthen the evaluation and its findings and led to meaningful collaboration with the client, participants, and other study collaborators.

We hope this example can support other project teams as they seek to operationalize equity in a meaningful way that aligns with the mission of RTI and its clients and improves the human condition.