Intention-to-Treat Failures in the Presence of Post-Intervention Confounding

Abstract

Prevention trials often report the intention-to-treat (ITT) effect as the causal effect of an intervention. While randomization balances confounding at baseline, post-intervention confounders (PICs) can a) affect outcomes” and b) PICs can be affected by intervention, even when randomization is “successful”, leading to bias in ITT effects; moreover, self-selection on PICs such as intervention dosage, non-study services, etc. may be more common than we realize in RCTs. Rosenbaum (1984, 2020) proposed the “net treatment difference” (NTD) framework for post-intervention confounding, where the PTC is treated as a de facto mediator but unlike mediation analysis, the focal estimate in this framework is the direct effect which captures treatment efficacy “net of” the PIC. Most dosage-related applications in prevention have used the complier average causal effect (CACE) framework, yet CACE’s assumption that the intervention does not affect compliance is often violated in practice. Far fewer applications of NTD modeling exist in prevention; more recently, it has been recognized that NTD modeling can potentially disentangle efficacy effects from effectiveness effects.
Methods: Using an adaptation of the potential outcomes mediation framework where dosage is both a moderator and a PIC, two illustrations are presented. The first uses data from an RCT of the Seeking Safety intervention (SSI) to illustrate SEM-based PIC modeling for designs where dosage is assessed in both conditions. The second uses data from the Early Adolescent Coping Power trial using a multiple imputation approach to PIC modeling when dosage was only observed in the active intervention group and estimated (conditional on observed covariates) in the control condition.

Results: In the first study, the ITT effect of SSI on PTSD severity was statistically non-significant (p=.74); under the NTD model, while the direct/efficacy effect was also non-significant (p=.36), the indirect effect was statistically significant (-.029 [95% CI: -.062, -.002]), suggesting SSI was more effective in reducing PTSD by increasing the number of sessions patients attended. In the EACP trial, the ITT effect of EACP on conduct problems was statistically non-significant (p=.26). Under the NTD model, the direct/efficacy effect was non-significant (p=.41), but the indirect effect was statistically significant (-.109 [95% CI: -.062, -.002]); EACP was effective in reducing conduct problems through higher projected session attendance compared to controls.