Assessing the impact of unpublished data on network meta-analysis outcomes in outpatient adults with acute migraine

Abstract

BACKGROUND: Methodological guidance recommends including both published and unpublished data in systematic reviews to enhance reliability and reduce potential bias.

OBJECTIVE: To evaluate the impact of unpublished data from double-blind, pharmacologic randomized controlled trials on the results of network meta-analysis (NMA) of migraine treatments.

METHODS: We supplemented the search of a recent systematic review with targeted searches for unpublished data on ClinicalTrials.gov, the World Health Organization International Clinical Trials Registry Platform, regulatory agency reports, The Preprints Citation Index, Europe PMC, and Embase.com, and conference abstracts from the past five years. Two independent reviewers selected eligible studies, verified publication status, extracted data, and reassessed certainty of evidence (COE). We reproduced the original NMA including unpublished data for four dichotomous outcomes. We compared our results with the original NMA in terms of risk ratio (RR), absolute risk difference (ARD) with 95% confidence interval (CI), COE assessments, and conclusions.

RESULTS: Seventeen (6,735 participants) out of 37 eligible unpublished trials had posted results and were analyzed, with most (59%) identified via trial registries. Additionally, unpublished outcome data were retrieved for four published trials from the original analysis. These unpublished trials added two previously unrepresented interventions to the NMA, increasing the number of direct comparisons and closed loops. Comparisons of RRs (95% CI) with the original analysis showed all effects maintained the same direction, though six CIs newly crossed the null effect (RR = 1). Among 144 COE assessments, four changed meaningfully: one was rated down from high to moderate due to imprecision, and three were rated up from very low/no evidence to low COE based on new direct evidence. Overall conclusions remained unchanged after including unpublished data.

CONCLUSIONS: In this case study, adding unpublished data had minimal impact on results and conclusions, with only minor changes in network geometry and COE.