Idiographic Clinical Trials: What are they, When are they Useful, and Recent Developments

Ty Ridenour, PhD
RTI International

Stephen Tueller, PhD
RTI International

ABSTRACT:
Within-subject clinical trials have long been considered a lesser option to randomized clinical trials in psychology and medicine and have fallen out of favor for several decades. However, these perceptions of within-subject clinical trials (also referred to as case study experimental designs, quasi-experimental designs, or N-of-1 studies) are largely due to historical limitations or misunderstandings about their purposes and uses. Over the last 10 years, methodological advances and innovative applications of these methods have led to a resurgence in within-subject clinical trials. This presentation will first review the logic and uses of within-subject clinical trials, their traditional methods, and their limitations. With this historical backdrop, recent advances will be presented as well as how these advances elevate the rigor of these studies to merit a new moniker of idiographic clinical trials. Illustrative studies will demonstrate intensive hierarchical regression, unified structural equations modeling, and N=1 analytic approaches.

Examining the Representation of Racial and Ethnic Minority Groups in Preventive Intervention Research

Pamela Buckley, PhD
University of Colorado Boulder

Velma McBride Murry, PhD
Vanderbilt University

ABSTRACT:
There are major gaps in research on racial/ethnic minoritized groups that impede effectiveness of preventive interventions, including insufficient attention to protective processes that prevent and avert risk, discounting input and guidance from community stakeholders of diverse communities, and overlooking crucial information about how to effectively transition interventions from white to racial/ethnic minority populations. Despite inclusion of racial ethnic minoritized populations, interventions validated with largely white samples are often recommended for all populations, which then heightens external validity concerns about widely disseminated treatments that are tested for one group but exported, perhaps uncritically, to others. Empirical evidence documenting the prevalence of racial/ethnic minoritized groups represented in preventive intervention research studies, however, is largely unknown. This project uses data from the Blueprints for Healthy Youth Development database (herein referred to as Blueprints), which provides an online clearinghouse of evidence-based interventions (EBIs) that prevent or reduce the likelihood of antisocial and violent behavior and promote a healthy course of youth development. Using data collected by Blueprints, which is the longest standing clearinghouse among up to 20 within the United States alone, we examined the representation of ethnic minority groups in preventive intervention research. Specifically, we reviewed and evaluated the nature, size, and scope of extant research across an 11-year period involving representation of racial and ethnic minority groups in preventive intervention research, thus serving as a vehicle for decision-making regarding the generalizability of EBIs. This presentation provides an overview of preliminary findings, presents recommendations for reporting race and ethnicity in prevention interventions studies, and concludes with a discussion aimed at identifying specific areas where additional research would be beneficial or even gaps in services where new intervention development is needed.

Embedding health equity into preventive science research and recommendations

Melissa Simon, MD, MPH
Northwestern University

ABSTRACT:
Achievement of health equity in preventive care and in prevention science requires critical examination of multiple domains across which structural racism and systems of oppression operate. Design thinking and implementation science are two key areas in which health equity can and should be centered to better design population level recommendation and guideline statements along with operationalizing and implementing such statements across populations. Dr. Simon will discuss these issues and give examples from the US Preventive Service Task Force (USPSTF) Recommendations and her own work.

Partnership and Adaptation to Implement Trauma Focused Therapy in Low Resourced Settings in the US and Latin America

Rosaura Orengo-Aguayo, PhD
Medical University of South Carolina

ABSTRACT:
Clinical psychological science has developed many efficacious treatments for diverse emotional and behavioral difficulties encountered by children and adolescents, yet most of these studies have disproportionally been conducted by American, university-based research labs. Dissemination and implementation of evidence-based services designed to meet the needs of a broader global population may require cultural and contextual adaptation to be successful. The current presentation will describe the implementation of Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) in 3 separate low-resourced settings (rural South Carolina, Puerto Rico, and El Salvador) utilizing the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework and guided by a community-based participatory research framework. We will discuss program development, building collaborative and responsive partnerships, and the use of implementation strategies to guide continuous quality improvement. Program evaluation data comparing baseline to posttreatment trauma symptoms and treatment completion rates for all settings will also presented, which suggests that treatment was associated with a large reduction in symptoms, exceeding that noted in many TF-CBT randomized trials. The implications of attention to context, adaptation, and methods of building partnerships with global communities will be discussed.

Designing for Accelerated Translation (DART) of emerging innovations in health

Alex T. Ramsey, PhD
Washington University St. Louis

ABSTRACT:
Accelerating the translation of emerging innovations, including genetic risk screenings and mobile technologies, is a priority for improving health. This webinar will present a conceptual architecture for dynamic decision-making about whether, when, and how to act on emerging evidence. The Design for Accelerated Translation (DART) framework proposes that underemphasized factors – demand, risk, cost, and an evolving evidence base – can improve our calculus of whether to ramp up or down on implementation efforts. This presentation will also outline key actions to speed movement from evidence to practice, including forming meaningful partnerships, designing for dissemination, and engaging in a learning health system.

Application of an Anti-racism Lens in the Field of Implementation Science: Reflections and Recommendations for Reframing Implementation Research with a Focus on Justice and Racial Equity

Rachel Shelton, ScD, MPH
Columbia University

Derek Griffith, PhD
Georgetown University

April Oh, PhD, MPH
National Institute of Health

Prajakta Adsul, MBBS, MPH, PhD
University of New Mexico

ABSTRACT:
Despite the promise of implementation science (IS) to reduce health inequities, critical gaps and opportunities remain in the field to promote health equity. Prioritizing racial equity and anti-racism approaches is critical in these efforts, so that IS does not inadvertently exacerbate disparities based on the selection of frameworks, methods, interventions, and strategies that do not reflect consideration of structural racism and its impacts. Grounded in extant research on structural racism and anti-racism, we discuss the importance of advancing understanding of how structural racism as a system shapes racial health inequities and inequitable implementation of evidence-based interventions among racially and ethnically diverse communities. We provide guidance for application of an anti-racism lens in the field of IS, focusing on select core elements in implementation research, including: 1) Stakeholder Engagement; 2) Conceptual Frameworks, Theories, Models; 3) Development, Selection, Adaptation of Evidence-based Interventions; 4) Evaluation; and 5) Implementation Strategies. We highlight the need for foundational grounding in anti-racism frameworks among implementation scientists to facilitate ongoing self-reflection, accountability, and attention to racial equity, and provide questions to guide such reflection and consideration.

Design for Digital Mental Health Interventions: Optimizing Engagement and Implementation

Andrea K. Graham, PhD
Northwestern University

ABSTRACT:
Efforts to translate evidence-based digital health interventions from research to real-world settings have struggled with sustained consumer engagement and the successful integration of these tools into their targeted systems of care. User-centered design involves collaborating with end-users throughout the process of intervention design, testing, and implementation to ensure the intervention meets consumers’ needs and preferences, to in turn increase uptake and engagement. This presentation will describe design methods that aim to increase engagement in intervention design and implementation, drawing on examples from applying these methods to mobile interventions for different mental and behavioral health problems.

Systems science modeling for implementation research: An application to tobacco smoking cessation for persons with serious mental illness

Takeru Igusa, PhD
Johns Hopkins University

Elizabeth Stuart, PhD
Johns Hopkins University

Gail Daumit, MD, MHS
Johns Hopkins University

ABSTRACT:
Implementation researchers have sought ways to use simulations to support the core components of implementation, which typically include assessing the need for change, designing implementation strategies, executing the strategies, and evaluating outcomes. The goal of this presentation is to explain how methods in systems science, with an emphasis on agent-based simulations, could fulfill this role. The presentation will include a discussion on explainability, in which modeling results are formulated in terms of constructs used in implementation science frameworks to facilitate the engagement of practitioners in the design and use of the models. This is part of an ongoing project focused on scaling up evidence-based tobacco smoking cessation practices in community mental health clinics in Maryland.

A Social Networked Approach to Implementation: Questions, Methods, and Interventions

Alicia Bunger, MSW, PhD
The Ohio State University

Reza Yousefi-Nooraie, PhD
University of Rochester

ABSTRACT:
Social networks are channels for transmitting knowledge, influence, and resources among patients, professionals, and organizations. These relationships are at the heart of all dissemination and implementation efforts. Examining implementation from a social network perspective has potential to help us better understand the complex social contexts of implementation, their dynamic evolution over time, and implementation success. Networks can also be altered or leveraged deliberately to facilitate and sustain implementation. In this talk, we will introduce terminology and assumptions underlying social network analysis and illustrate how social network analysis can be used to address a range of implementation research questions. We will also introduce a new typology of network interventions (interventions that deliberately alter network structures) that have potential to be used as implementation strategies.

To request powerpoint slides, please email psmg@northwestern.edu

Predictive analytics in child welfare: An overview of current initiatives and ethical frameworks to inform equitable policy and practice

Paul Lanier, Phd
University of North Carolina at Chapel Hill

Daniel Gibbs, JD
University of North Carolina at Chapel Hill

ABSTRACT:
The child welfare system is faced with high-stakes decisions every day that must balance goals of child protection with parental rights. The use of administrative data coupled with advanced analytic methods to drive decision-making in child welfare practice and policy has increased in recent years. For example, predictive analytics and associated approaches have been used to identify families at higher risk for future harm to target appropriate services. However, the potential for bias in algorithmic decision-making must be weighed against the potential benefits of using any new technology. Specifically, we must consider whether predictive analytics exacerbates or ameliorates existing racial differences in experiences with the child welfare system. In this presentation, we will 1) provide an overview of the current state of the field, 2) give several current examples of predictive analytics used in child welfare, and 3) present preliminary findings from our recent study that aimed to apply an existing ethical framework to a data-driven prevention initiative known as Birth Match.

The Implementation and Sustainment Facilitation (ISF) Strategy A promising strategy for improving implementation climate, implementation effectiveness, and intervention effectiveness.

Bryan Garner, Ph.D.
RTI International

ABSTRACT:
Over at least the past 15 years, implementation research has been defined as the scientific study of the use of strategies to adopt and integrate evidence-based health interventions into clinical and community settings in order to improve patient outcomes and benefit population health. Notwithstanding the significant progress that has been made to date, there remains a significant need for implementation research to identify effective and cost-effective strategies for improving the implementation and sustainment of evidence-based health interventions into clinical and community settings. In 2014, the National Institute on Drug Abuse funded a dual-randomized type 2 hybrid trial that focused on experimentally testing the Implementation and Sustainment Facilitation (ISF) Strategy as an adjunct to a staff-focused Addiction Technology Transfer Center (ATTC) Strategy. After providing a brief overview of the ATTC Strategy, this presentation will focus on describing the ISF Strategy, with emphasis on: (a) its guiding theory, framework, and principles, (b) its standardized tools/exercise, (c) study results that support its effectiveness and cost-effectiveness as an adjunct to the ATTC strategy, and (d) how it is currently being tested as part of several on-going implementation research experiments.

Treatment for opioid use disorder in the Florida Medicaid population: A cascade of care approach to assessing quality.

Kimberly Johnson, Ph.D.
University of South Florida

ABSTRACT:
Despite increased funding, opioid overdose rates remain high. Understanding at what stage of the process and for whom the system is failing is critical to improving care. A cascade of care (CoC) model may improve understanding of gaps in addiction treatment availability and quality over current single measure methods. Using a CoC framework, we assessed treatment quality and outcomes for opioid use disorder for the Florida Medicaid population in 2017/2018 by demographics.

Methods: Data from Florida Medicaid claims for 2017 and 2018 were used to calculate the number of enrollees who were diagnosed, began medication, were retained on medication for a minimum of 180 days, and who died.

Results: Only 28% of those newly diagnosed with OUD in 2017 began treatment with an FDA approved medication. Once on medication, 38% of newly diagnosed enrollees were retained in treatment for at least 180 days. Those who remained in treatment for 180 days had a hazard ratio of death of 0.226 (95% CI = 0.174 to 0.294) compared to those that did not initiate treatment, a reduction in mortality from 10% without care to 2% with care.

Conclusions: Initiating medication after diagnosis is the process most in need of improvement, though there is much room for improvement in treatment retention as well. The CoC is an appropriate method of measuring the quality of the functioning of the treatment system at the state level.

Rapid Cycle Systems Modeling and Decision Sampling to Inform Development and Implementation of System-Wide Innovations to Promote Pediatric Mental and Behavioral Health

Thomas I. Mackie, PhD, MPH
Rutgers School of Public Health

R. Christopher Sheldrick, PhD
Boston University School of Public Health

ABSTRACT:
Calls have been made for greater application of simulation modeling and decision sciences to expedite research evidence use into mental health policy. Funded by W.T. Grant Foundation, Drs. Mackie and Sheldrick will report on a study, entitled Research Evidence Adoption for Child Health (REACH), in which they propose the use of Rapid Cycle Systems Modeling (RCSM), as an implementation strategy that aims to assist local decision-makers in needs assessment and prioritization, knowledge exchange, and consensus building when developing system-wide innovations. RCSM requires that one (1) identify gaps in the information available and questions requiring resolve to inform stakeholders’ decisions, (2) build a simulation model and identify relevant estimates to parameterize the model (e.g., evidence reviews), and (3) conduct group facilitation sessions to asses model utility and need for further adaptation.

To demonstrate utility, we provide an illustrative case study from REACH in which we specifically sought to identify how decision-makers developed protocols to identify and treat the trauma of children entering foster care. In conducting the first step of RCSM, we identified information gaps by employing a “decision sampling” framework in qualitative interviews.  In this approach, we anchored key informants’ responses on a recent index decision to minimize recall and response bias and potential desirability of evidence use. Respondents (n=31) described a continuum of 14 decision points relevant to five domains when developing a system-wide approach to identify and address trauma, including: 1. reach of the screening protocol, 2. content of the screening tool, 3. threshold for referral, 4. resources for screening startup and sustainment, and 5. system capacity to respond to identified needs. In the second stage of RCSM, we built a Monte Carlos simulation model that responded to the gaps in available information identified across this decision continuum and used additional data collected in the interviews, coupled with evidence reviews, to inform the model structure and parameterization. Finally, we facilitated group interviews (n=4) learning that the model developed was assessed by relevant stakeholders to hold utility and face validity. We will present an overview of this work and the model to illustrate potential utility of RCSM and decision sampling in other research domains.

Advancing implementation of eHealth interventions for HIV prevention through the scale up of Keep It Up!

Brian Mustanski, PhD
Northwestern University

Nanette Benbow, M.A.S.
Northwestern University

Kathryn Macapagal, PhD
Northwestern University

Dennis Li, PhD, MPH
Northwestern University

ABSTRACT:
Despite substantial NIH investment in developing eHealth HIV prevention interventions, little implementation research has examined strategies to effectively scale up these programs. The Keep It Up! 3.0 study advances our knowledge by comparing two approaches for delivering an online CDC-best-evidence HIV intervention in 44 counties. This presentation describes our county-randomized hybrid type III effectiveness–implementation trial, focusing on the pragmatic design of our sampling methodology, outcome measures, the intervention application, and recruitment/retention protocols. By emulating real-world contexts, we can understand how to not only have the greatest public health impact with but also speed up implementation of eHealth HIV interventions.

Implementation Research Logic Model: A Method for Planning, Executing, Reporting and Synthesizing Implementation Projects

JD Smith, PhD
University of Utah School of Medicine

Dennis Li, PhD, MPH
Northwestern University

Miriam Rafferty, PT, DPT, PhD
Northwestern University

ABSTRACT:
Background: Numerous models, frameworks, and theories exist for specific aspects of implementation research, including for determinants, strategies, and outcomes. However, implementation research projects often fail to provide a coherent rationale or justification for how these aspects are selected and tested in relation to one another. Despite this need to better specify the conceptual linkages between the core elements involved in projects, few tools or methods have been developed to aid in this task. The Implementation Research Logic Model (IRLM) was created for this purpose and to enhance the rigor and transparency of describing the often-complex processes of improving the adoption of evidence-based practices in healthcare delivery systems.

Methods: The IRLM structure and guiding principles were developed through a series of preliminary activities with multiple investigators representing diverse implementation research projects in terms of contexts, research designs, and implementation strategies being evaluated. The utility of the IRLM was evaluated in the course of a two-day training to over 130 implementation researchers and healthcare delivery system partners.

Results: Preliminary work with the IRLM produced a core structure and multiple variations for common implementation research designs and situations, as well as guiding principles and suggestions for use. Results of the survey indicated high utility of the IRLM for multiple purposes, such as improving rigor and reproducibility of projects; serving as a “roadmap” for how the project is to be carried out; clearly reporting and specifying how the project is to be conducted; and understanding the connections between determinants, strategies, mechanisms, and outcomes for their project. 

Conclusions: The IRLM is a semi-structured, principles-guided tool designed to improve the specification, rigor, reproducibility, and testable causal pathways involved in implementation research projects. The IRLM can also aid implementation researchers and implementation partners in the planning and execution of practice change initiatives. Adaptation and refinement of the IRLM is ongoing, as is the development of resources for use and applications to diverse projects, to address the challenges of this complex scientific field. This presentation will cover the primary elements and use of the IRLM, review evidence of its utility, and present multiple completed examples.

System-level Evidence Based Practice Implementation Research: Same questions, different method.

Rebecca Lengnick-Hall, PhD
Washington University in St. Louis

Gracelyn Cruden, PhD
Oregon Social Learning Center

ABSTRACT:
This presentation brings together two researchers with overlapping system-level implementation research interests. While we often ask the same research questions, our methodological and disciplinary differences allow us to answer these questions in different, but complementary ways. The shared line of inquiry questions 1) How are implementation processes affected by multi-level factors over time, and 2) how can implementation support tools be designed to support these complex implementation processes in a manner that is generalizable across contexts, implementation phases, and evidence-based practices (EBPs)?  In this presentation, we will each share an example of our work aimed at understanding the process of responding to evidence-based policy or practice initiatives across implementation phases, while highlighting the unique methodological approaches. Dr. Rebecca Lengnick-Hall will explain how contracting arrangements act as a “bridge” between the public sector systems that fund and the community-based organizations that deliver EBPs. Dr. Gracelyn Cruden will describe a decision support tool based in a multi-criteria decision analysis framework that was developed to support EBP adoption for local implementation. To conclude, we will explore specific ways that our different methodological perspectives have both informed and pushed the other’s research agenda forward. We hope to conclude this presentation by engaging the PSMG community in a discussion about the integration of system science and qualitative methods to understand system-level EBP implementation.

Individual-level mechanisms of behavior change across implementation stakeholders

Clayton Cook, PhD
University of Minnesota

ABSTRACT:
This webinar will discuss the importance of attending to individual-level mechanisms of behavior change across a range of key implementation stakeholders. Whether it is to increase clients pull (I.e., requests) for EBPs, policymakers translation of research into policy, administrators use of implementation strategies, or implementer uptake and use of EBPs, there is evidence suggesting the existence of universal individual-level mechanisms that are applicable to all humans and represent the specific targets of behavior change strategies. This webinar will also discuss methods of adapting and tailoring strategies to the specific stakeholders and the context in which they operate

Implementation Mechanisms: The Next Frontier

Cara C. Lewis, PhD
Kaiser Permanente
Washington Health Research Institute

ABSTRACT:
Implementation mechanisms are the processes through which strategies work, or how strategies achieve their effect. Without knowing implementation mechanisms, one cannot effectively or efficiently select, match, tailor, or optimize implementation strategies. Recent systematic reviews suggest that few researchers attempt to explore implementation mechanisms given theoretical, conceptual, and methodological issues. This talk will describe the current state of implementation mechanisms evaluation, an approach to articulating implementation mechanisms through causal pathway diagrams, and early learnings from an attempt to develop an implementation mechanisms research agenda.

House on Fire: Addressing Opioid Overdose as an Epidemic

Kimberly Johnson, PhD, MBA
Research Associate Professor, University of South Florida

ABSTRACT:
There are standard public health models of addressing epidemics of contagious disease. This presentation will argue that if we use standard protocols that have been developed to address other epidemics, we would make greater progress in reducing the rate of drug overdose deaths in the United States. The presentation will remind participants of how smallpox was eradicated and what was learned about epidemic control in that effort as well as what is being learned in international efforts to eradicate the spread of HIV. The presentation will conclude with a discussion of the strengths and weaknesses of the current response to the opioid epidemic from a public health epidemic control perspective.

Conceptualizing and Testing Multilevel Mechanisms of Change in Implementation Science

Nathaniel J. Williams, PhD., LCSW
Boise State University

ABSTRACT:
An important step toward the development and targeting of optimally effective, efficient, and feasible implementation strategies involves identification of the mechanisms through which these strategies influence implementation and clinical outcomes. This presentation will provide an overview of the conceptualization, measurement, and analysis of multilevel mechanisms in implementation science with an emphasis on the application of multilevel mediation analysis. Dr. Williams will review the state-of-the-science on testing implementation mechanisms and will illustrate a highly general mediation approach for testing multilevel mechanisms, using as examples studies focused on organizational leadership and organizational social context.

Project MIMIC (Maximizing Implementation of Motivational Incentives in Clinics): A type 3 effectiveness-implementation hybrid trial

Bryan Garner, Ph.D.
Research Triangle Institute (RTI)

Sara Becker, Ph.D.
Department for Behavioral and Social Sciences, Brown University School of Public Health

ABSTRACT:
There is an urgent public health need to improve the outcomes of individuals with opioid use disorders (OUDs). There are currently five approved medication formulations, which relative to placebo have demonstrated effectiveness in helping patients attain abstinence from opioids. Nonetheless, patients’ opioid abstinence rates are sub-optimal: even when treated with the newest extended-release formulations only about 40% of patients maintain abstinence during the first 6-months of treatment. Contingency Management (CM) is one of the only behavioral treatment shown to improve OUD pharmacotherapy outcomes, yet implementation of CM within OUD treatment centers remains quite low. Project MIMIC is a type 3 effectiveness-implementation hybrid trial to test the effectiveness of an enhanced-version of the Addiction Technology Transfer Center’s current multifaceted implementation strategy on both implementation outcomes (primary aim) and patient outcomes (secondary aim). For this PSMG web-presentation, Drs. Becker and Garner (Project MIMIC’s principal investigators) will teach others about the project and its protocol, with special emphasis on helping the audience learn more about the project’s two implementation conditions, key challenges, and key lessons learned.

The US opioid epidemic: A public health crisis and opportunity for implementation research

Mark A. McGovern, PhD
Stanford University

Abstract: The US opioid epidemic has exposed a significant gap in access to evidence-based care for persons who suffer from opioid use disorders. Although effective medications exist for opioid addiction, they are not widely available. This presentation describes several system level initiatives to increase reach and adoption, in efforts close the implementation gap. An emphasis will be on lessons learned, and how more rigorous implementation research methods may be brought to bear to address this major public health problem.

Leveraging Treatment & Recovery Services to Address the Opioid Crisis

Todd Molfenter, PhD
University of Wisconsin - Madison, Center for Health Enhancement Systems Studies

Abstract: Dr. Molfenter will provide an overview of the “Opioid Crisis” in the United States and how it is impacting research opportunities in the addiction treatment and Implementation Science fields.  Causes and impact of the epidemic will initially be discussed. Then, opioid use disorder treatment evidence-based practices will be described. Followed by the gaps and challenges being experience in getting these practices implemented and research we have conducted to better understand how to get these practices implemented (or scaled-up) into broader practice.

Evaluating complex interventions: Confronting and guiding (versus ignoring and suppressing) heterogeneity and adaptation

Brian S. Mittman, Ph.D.
Kaiser Permanente

ABSTRACT:
Implementation strategies and many of the clinical and health service delivery interventions they aim to implement are characterized by multiple components targeting multiple behaviors and levels and are often characterized by extreme heterogeneity and adaptability.  Although researchers often attempt to standardize and achieve fidelity to highly-specified manualized intervention protocols, the required actions to suppress adaptation and maximize internal validity often lead to reduced effectiveness:  adaptation to local conditions often increases intervention effectiveness relative to implementation of a fixed version of an intervention across heterogeneous settings.  This presentation introduces the new PCORI Methodology Committee Standards for Complex Interventions and discusses their role in research to (a) study and guide rather than suppress or ignore adaptation, achieving internal validity through adherence to an adaptation algorithm and through fidelity to function rather than form, and to (b) develop empirical evidence, insights and guidance for policy and practice decision makers who are charged with adapting and managing complex interventions rather than simply selecting and deploying simple, fixed interventions.

Hybrid Effectiveness-Implementation Designs: A Review and New Considerations

Geoffrey M. Curran, Ph.D.
University of Arkansas for Medical Sciences

ABSTRACT:
The presentation will provide an overview of effectiveness-implementation hybrid designs. Dr. Curran will review current trends in the use of these designs, provide examples of strong designs across the hybrid design continuum, and offer recommendations for hybrid design specifications, outcome measurement, and reporting.

Keywords: research design, implementation science, hybrid designs

Intensive Longitudinal Data, Multilevel Modeling, and SEM: New Features in Mplus Version 8.1

Bengt Muthén, Ph.D.
UCLA

Tihomir Asparouhov, Ph.D.
Mplus
 

ABSTRACT:
With the closely spaced repeated measurements in intensive longitudinal data (ILD), it becomes critical to allow for autocorrelation.  In many ILD models, the autocorrelation is represented as the outcome at one or more previous time points influencing the current outcome, but an alternative model represents this as auto correlated residuals. The Dynamic Structural Equation Modeling (DSEM) approach of Asparouhov et al. (2018) has therefore been expanded in Mplus Version 8.1 into RDSEM, residual DSEM.  Part 1 of this talk gives a brief overview of some key RDSEM models both for N=1 analysis and for N >1 multilevel analysis with random effects.  The development of Version 8.1 has also led to a new approach to multilevel analysis with random slopes for not only DSEM and RDSEM but also for general multilevel modeling.  Using the Bayes estimator, this makes it possible to use a latent variable decomposition of the predictor that enables latent mean centering avoiding the biases in common approaches.  Part 2 of the talk presents theory and applications for such random slope modeling.  Other general SEM developments in Mplus Version 8.1 include a convenient approach to check if two models are nested as well as several other new features.

Improving the Design and Application of Implementation Strategies

Byron Powell, Ph.D., L.C.S.W.
University of North Carolina-Chapel Hill

ABSTRACT:
Effective implementation often requires the use of multifaceted, multilevel implementation strategies given the myriad barriers to delivering evidence-based practices in community settings. Ideally, the design of implementation strategies would be guided by theory, evidence, and pragmatic input from relevant stakeholders; however, methods to guide the application of implementation strategies are not well developed. There is a clear need for rigorous and practical methods to guide stakeholders in the identification, selection, and tailoring of implementation strategies for their contexts. This presentation will include an overview of emerging approaches to the design and application of implementation strategies, a discussion of key priorities for the field, and an introduction to a study focused on developing and piloting a novel method for tailoring implementation strategies to specific contexts.

Considering the How with the What: Implementation fidelity to foster success

Lisa Saldana, Ph.D.
Oregon Social Learning Center

ABSTRACT:
Previous research has focused on developing and testing the 8-staged Stages of Implementation Completion (SIC)—an observational tool of implementation progress from Engagement (Stage 1) to development of Competency (Stage 8). The SIC provides a unique and rich source of real-world activities conducted by sites attempting to implement an EBP, and an assessment of inner and outer context variables of the sites. The SIC maps onto three phases of implementation (pre-implementation, implementation, and sustainability), and was developed as part of an implementation trial to assess sites' implementation process behavior and obtainment of milestones. Recent advances with the SIC suggest its potential utility as a measure of implementation fidelity; the SIC can reliably predict successful program start-up and achievement of program competency for sustainment based on both the quality and quantity of implementation activity completion.  This presentation will describe recent updates to SIC analyses, and the potential of the SIC to serve as a measure of implementation fidelity, across the span of implementation from Engagement to achievement of Competency in program delivery. Just as achieving high treatment fidelity is important to achieve successful clinical outcomes, this presentation will describe the value of achieving high implementation fidelity for successful implementation outcomes (e.g., sustainability).

“Scaling-out” evidence-based interventions to new populations and/or health care delivery systems: 
Improving efficacy in implementation process

Greg Aarons, Ph.D.
University of California-San Diego

Marisa Sklar, Ph.D
University of California-San Diego

C. Hendricks Brown, Ph.D.
Northwestern University

ABSTRACT:
Background: Implementing treatments and interventions with demonstrated effectiveness is critical for improving patient health outcomes at a reduced cost. When an evidence-based intervention (EBI) is implemented with fidelity in a setting that is very similar to the setting wherein it was previously found to be effective, it is reasonable to anticipate similar benefits of that EBI. However, one goal of implementation science is to expand the use of EBIs as broadly as is feasible and appropriate in order to foster the greatest public health impact. When implementing an EBI in a novel setting, or targeting novel populations, one must consider whether there is sufficient justification that the EBI would have similar benefits to those found in earlier trials.

Discussion: In this paper, we introduce a new concept for implementation called “scaling-out” when EBIs are adapted either to new populations or new delivery systems, or both. Using existing external validity theories and multilevel mediation modeling, we provide a logical framework for determining what new empirical evidence is required for an intervention to retain its evidence-based standard in this new context. The motivating questions are whether scale-out can reasonably be expected to produce population-level effectiveness as found in previous studies, and what additional empirical evaluations would be necessary to test for this short of an entirely new effectiveness trial. We present evaluation options for assessing whether scaling-out results in the ultimate health outcome of interest.

Conclusion: In scaling to health or service delivery systems or population/community contexts that are different from the setting where the EBI was originally tested, there are situations where a shorter timeframe of translation is possible. We argue that implementation of an EBI in a moderately different setting or with a different population
can sometimes “borrow strength” from evidence of impact in a prior effectiveness trial. The collection of additional empirical data is deemed necessary by the nature and degree of adaptations to the EBI and the context. Our argument in this paper is conceptual, and we propose formal empirical tests of mediational equivalence in a follow-up paper.

Conceptualizing and measuring sustainability of prevention programs and initiatives

Lawrence Palinkas, Ph.D.
Sapna Mendon, M.S.W.
University of Southern California

ABSTRACT:
Sustainment of prevention efforts directed at substance use and mental health problems is one of the greatest, yet least understood challenges of implementation science.  A large knowledge gap exists regarding the meaning of the term “sustainment” and what factors predict or measure sustainment of effective prevention programs and support systems. Specifically, it is unclear whether sustainment is an outcome of implementation as described by Proctor and colleagues1, whether it reflects a (final) stage in the process of implementation, or whether it is both process and outcome of implementation.  This presentation describes an effort to design and evaluate a Sustainment Measurement System (SMS), based on interviews with 45 representatives of 10 grantees within 4 SAMHSA programs (Strategic Prevention Framework – State Initiative Grants, Sober Truth on Preventing Underage Drinking [STOP-Act], Garrett Lee Smith Suicide Prevention Program, and Prevention Practices in Schools). Data collection consisted of a semi-structured interview to identify experiences with implementation and sustainment barriers and facilitators; free list exercise to elicit participant conceptions of the word “sustainment” and what it will take to sustain their programs; and a checklist of Consolidated Framework for Implementation Research (CFIR) elements to identify which are important for sustainment.  A draft SMS will be introduced, along with a preliminary conceptual model to explain the sustainment of community-wide prevention programs that are designed for and implemented in community settings, and a qualitative comparative analysis (QCA) of CFIR domains to identify necessary and sufficient conditions for sustainment.