A Proposed Framework for Designing Trials Evaluating the Effectiveness and Implementation of Digital Interventions for Substance Use

Theresa Matson, PhD
Kaiser Permanente Washington Health Research Institute

Joseph Glass, PhD
Kaiser Permanente Washington Health Research Institute

ABSTRACT:
Background: Clinicians and community health workers may wish to use digital interventions to reach more patients with unhealthy substance use, optimize costs of care, and improve outcomes. However, digital interventions have unique implementation considerations (e.g., technology infrastructure, digital literacy, monitoring and follow-up) and may not fit traditional care pathways. Effectiveness and implementation trials are needed to understand how well digital interventions work and how to best deploy them in the real-world. This presentation presents a framework to help researchers design their trials in such a way that maximizes scientific understanding.

Methods: This framework draws from the literature on trial design, expert perspectives on the use of digital interventions, and lessons learned from implementation science research programs. It outlines three major steps for designing trials of digital interventions: 1) framing the research question; 2) delineating components of the intervention, implementation strategy, and delivery approach; and 3) specifying the experiment and other elements of trial design.

Results: In Step 1 of this framework, researchers frame the research question in terms of the goals or activities to be tested (i.e., features of the digital intervention itself, specific implementation strategies, or level of clinical support). In Step 2, researchers define and delineate each study component as actor, activity, action target, or outcome to maximize inference and reproducibility across studies. Steps 1 and 2 inform Step 3, in which researchers specify features of the trial design (i.e., experimental/comparator selection, outcome selection, and design classification). To illustrate the utility of this framework, we compare and contrast implementation and effectiveness studies of digital interventions for substance use.

Conclusion: The proposed framework provides a foundation for designing trials of digital interventions for substance use in healthcare and community settings. This framework can help researchers decide on appropriate methodology and help decision-makers understand how to apply findings.