A Proposed Framework for Designing Trials Evaluating the Effectiveness and Implementation of Digital Interventions for Substance Use

Theresa Matson, PhD
Kaiser Permanente Washington Health Research Institute

Joseph Glass, PhD
Kaiser Permanente Washington Health Research Institute

ABSTRACT:
Background: Clinicians and community health workers may wish to use digital interventions to reach more patients with unhealthy substance use, optimize costs of care, and improve outcomes. However, digital interventions have unique implementation considerations (e.g., technology infrastructure, digital literacy, monitoring and follow-up) and may not fit traditional care pathways. Effectiveness and implementation trials are needed to understand how well digital interventions work and how to best deploy them in the real-world. This presentation presents a framework to help researchers design their trials in such a way that maximizes scientific understanding.

Methods: This framework draws from the literature on trial design, expert perspectives on the use of digital interventions, and lessons learned from implementation science research programs. It outlines three major steps for designing trials of digital interventions: 1) framing the research question; 2) delineating components of the intervention, implementation strategy, and delivery approach; and 3) specifying the experiment and other elements of trial design.

Results: In Step 1 of this framework, researchers frame the research question in terms of the goals or activities to be tested (i.e., features of the digital intervention itself, specific implementation strategies, or level of clinical support). In Step 2, researchers define and delineate each study component as actor, activity, action target, or outcome to maximize inference and reproducibility across studies. Steps 1 and 2 inform Step 3, in which researchers specify features of the trial design (i.e., experimental/comparator selection, outcome selection, and design classification). To illustrate the utility of this framework, we compare and contrast implementation and effectiveness studies of digital interventions for substance use.

Conclusion: The proposed framework provides a foundation for designing trials of digital interventions for substance use in healthcare and community settings. This framework can help researchers decide on appropriate methodology and help decision-makers understand how to apply findings.

Systems science modeling for implementation research: An application to tobacco smoking cessation for persons with serious mental illness

Takeru Igusa, PhD
Johns Hopkins University

Elizabeth Stuart, PhD
Johns Hopkins University

Gail Daumit, MD, MHS
Johns Hopkins University

ABSTRACT:
Implementation researchers have sought ways to use simulations to support the core components of implementation, which typically include assessing the need for change, designing implementation strategies, executing the strategies, and evaluating outcomes. The goal of this presentation is to explain how methods in systems science, with an emphasis on agent-based simulations, could fulfill this role. The presentation will include a discussion on explainability, in which modeling results are formulated in terms of constructs used in implementation science frameworks to facilitate the engagement of practitioners in the design and use of the models. This is part of an ongoing project focused on scaling up evidence-based tobacco smoking cessation practices in community mental health clinics in Maryland.

Reducing Opioid Use Disorder and Overdose in the United States: Model Development and Policy Analysis

Mohammad Jalali, PhD
Harvard University

Wayne Wakeland, PhD
Portland State University

ABSTRACT:
The opioid crisis is one of the most pressing public health issues in the U.S. today. Opioid overdoses are the proverbial “tip of the iceberg,” arising within a complex adaptive system characterized by rapidly changing dynamics combined with significant time lags and large uncertainties in the data. System dynamics modeling is a critical tool to guide policymaking and avoid unintended consequences. We developed a simulation model of the opioid system, spanning from medical use of prescription opioids to opioid misuse and heroin use, use disorder, treatment, and remission. The model aims to help policymakers address the crisis by aiding in policy analysis and decision-making under uncertainty. We project the effects of several policies to reduce opioid use disorder and overdose, and analyze intended and unintended effects of the policies over the next 10 years. Model simulations suggest most policies implemented on their own will achieve only modest reduction in either fatal overdoses or prevalence of OUD.