Filtering by: Implementation Methods

Oct
9
12:00 PM12:00

Brian S. Mittman: Evaluating complex interventions: Confronting and guiding (versus ignoring and suppressing) heterogeneity and adaptation

Evaluating complex interventions: Confronting and guiding (versus ignoring and suppressing) heterogeneity and adaptation

Brian S. Mittman, Ph.D.
Kaiser Permanente

ABSTRACT:
Implementation strategies and many of the clinical and health service delivery interventions they aim to implement are characterized by multiple components targeting multiple behaviors and levels and are often characterized by extreme heterogeneity and adaptability.  Although researchers often attempt to standardize and achieve fidelity to highly-specified manualized intervention protocols, the required actions to suppress adaptation and maximize internal validity often lead to reduced effectiveness:  adaptation to local conditions often increases intervention effectiveness relative to implementation of a fixed version of an intervention across heterogeneous settings.  This presentation introduces the new PCORI Methodology Committee Standards for Complex Interventions and discusses their role in research to (a) study and guide rather than suppress or ignore adaptation, achieving internal validity through adherence to an adaptation algorithm and through fidelity to function rather than form, and to (b) develop empirical evidence, insights and guidance for policy and practice decision makers who are charged with adapting and managing complex interventions rather than simply selecting and deploying simple, fixed interventions.

View Event →
Sep
25
12:00 PM12:00

Geoffrey M. Curran: Hybrid Effectiveness-Implementation Designs: A Review and New Considerations

Hybrid Effectiveness-Implementation Designs: A Review and New Considerations

Geoffrey M. Curran, Ph.D.
University of Arkansas for Medical Sciences

ABSTRACT:
The presentation will provide an overview of effectiveness-implementation hybrid designs. Dr. Curran will review current trends in the use of these designs, provide examples of strong designs across the hybrid design continuum, and offer recommendations for hybrid design specifications, outcome measurement, and reporting.

Keywords: research design, implementation science, hybrid designs

View Event →
May
8
12:00 PM12:00

Bengt Muthén & Tihomir Asparouhov: Intensive longitudinal data, multilevel modeling, and SEM: New features in Mplus version 8.1

Intensive Longitudinal Data, Multilevel Modeling, and SEM: New Features in Mplus Version 8.1

Bengt Muthén, Ph.D.
UCLA

Tihomir Asparouhov, Ph.D.
Mplus
 

ABSTRACT:
With the closely spaced repeated measurements in intensive longitudinal data (ILD), it becomes critical to allow for autocorrelation.  In many ILD models, the autocorrelation is represented as the outcome at one or more previous time points influencing the current outcome, but an alternative model represents this as auto correlated residuals. The Dynamic Structural Equation Modeling (DSEM) approach of Asparouhov et al. (2018) has therefore been expanded in Mplus Version 8.1 into RDSEM, residual DSEM.  Part 1 of this talk gives a brief overview of some key RDSEM models both for N=1 analysis and for N >1 multilevel analysis with random effects.  The development of Version 8.1 has also led to a new approach to multilevel analysis with random slopes for not only DSEM and RDSEM but also for general multilevel modeling.  Using the Bayes estimator, this makes it possible to use a latent variable decomposition of the predictor that enables latent mean centering avoiding the biases in common approaches.  Part 2 of the talk presents theory and applications for such random slope modeling.  Other general SEM developments in Mplus Version 8.1 include a convenient approach to check if two models are nested as well as several other new features.

View Event →
May
1
12:00 PM12:00

Byron Powell: Improving the design and application of implementation strategies

Improving the Design and Application of Implementation Strategies

Byron Powell, Ph.D., L.C.S.W.
University of North Carolina-Chapel Hill

ABSTRACT:
Effective implementation often requires the use of multifaceted, multilevel implementation strategies given the myriad barriers to delivering evidence-based practices in community settings. Ideally, the design of implementation strategies would be guided by theory, evidence, and pragmatic input from relevant stakeholders; however, methods to guide the application of implementation strategies are not well developed. There is a clear need for rigorous and practical methods to guide stakeholders in the identification, selection, and tailoring of implementation strategies for their contexts. This presentation will include an overview of emerging approaches to the design and application of implementation strategies, a discussion of key priorities for the field, and an introduction to a study focused on developing and piloting a novel method for tailoring implementation strategies to specific contexts.

View Event →
Apr
24
12:00 PM12:00

Lisa Saldana: Considering the How with the What: Implementation fidelity to foster success

Considering the How with the What: Implementation fidelity to foster success

Lisa Saldana, Ph.D.
Oregon Social Learning Center

ABSTRACT:
Previous research has focused on developing and testing the 8-staged Stages of Implementation Completion (SIC)—an observational tool of implementation progress from Engagement (Stage 1) to development of Competency (Stage 8). The SIC provides a unique and rich source of real-world activities conducted by sites attempting to implement an EBP, and an assessment of inner and outer context variables of the sites. The SIC maps onto three phases of implementation (pre-implementation, implementation, and sustainability), and was developed as part of an implementation trial to assess sites' implementation process behavior and obtainment of milestones. Recent advances with the SIC suggest its potential utility as a measure of implementation fidelity; the SIC can reliably predict successful program start-up and achievement of program competency for sustainment based on both the quality and quantity of implementation activity completion.  This presentation will describe recent updates to SIC analyses, and the potential of the SIC to serve as a measure of implementation fidelity, across the span of implementation from Engagement to achievement of Competency in program delivery. Just as achieving high treatment fidelity is important to achieve successful clinical outcomes, this presentation will describe the value of achieving high implementation fidelity for successful implementation outcomes (e.g., sustainability).

View Event →
Mar
27
12:00 PM12:00

Greg Aarons, Marisa Sklar, and Hendricks Brown: “Scaling-out” evidence-based interventions to new populations and/or health care delivery systems:  Improving efficacy in implementation process

“Scaling-out” evidence-based interventions to new populations and/or health care delivery systems: 
Improving efficacy in implementation process

Greg Aarons, Ph.D.
University of California-San Diego

Marisa Sklar, Ph.D
University of California-San Diego

C. Hendricks Brown, Ph.D.
Northwestern University

ABSTRACT:
Background: Implementing treatments and interventions with demonstrated effectiveness is critical for improving patient health outcomes at a reduced cost. When an evidence-based intervention (EBI) is implemented with fidelity in a setting that is very similar to the setting wherein it was previously found to be effective, it is reasonable to anticipate similar benefits of that EBI. However, one goal of implementation science is to expand the use of EBIs as broadly as is feasible and appropriate in order to foster the greatest public health impact. When implementing an EBI in a novel setting, or targeting novel populations, one must consider whether there is sufficient justification that the EBI would have similar benefits to those found in earlier trials.

Discussion: In this paper, we introduce a new concept for implementation called “scaling-out” when EBIs are adapted either to new populations or new delivery systems, or both. Using existing external validity theories and multilevel mediation modeling, we provide a logical framework for determining what new empirical evidence is required for an intervention to retain its evidence-based standard in this new context. The motivating questions are whether scale-out can reasonably be expected to produce population-level effectiveness as found in previous studies, and what additional empirical evaluations would be necessary to test for this short of an entirely new effectiveness trial. We present evaluation options for assessing whether scaling-out results in the ultimate health outcome of interest.

Conclusion: In scaling to health or service delivery systems or population/community contexts that are different from the setting where the EBI was originally tested, there are situations where a shorter timeframe of translation is possible. We argue that implementation of an EBI in a moderately different setting or with a different population
can sometimes “borrow strength” from evidence of impact in a prior effectiveness trial. The collection of additional empirical data is deemed necessary by the nature and degree of adaptations to the EBI and the context. Our argument in this paper is conceptual, and we propose formal empirical tests of mediational equivalence in a follow-up paper.

View Event →
Mar
20
12:00 PM12:00

Lawrence Palinkas & Sapna Mendon: Conceptualizing and measuring sustainability of prevention programs and initiatives

Conceptualizing and measuring sustainability of prevention programs and initiatives

Lawrence Palinkas, Ph.D.
Sapna Mendon, M.S.W.
University of Southern California

ABSTRACT:
Sustainment of prevention efforts directed at substance use and mental health problems is one of the greatest, yet least understood challenges of implementation science.  A large knowledge gap exists regarding the meaning of the term “sustainment” and what factors predict or measure sustainment of effective prevention programs and support systems. Specifically, it is unclear whether sustainment is an outcome of implementation as described by Proctor and colleagues1, whether it reflects a (final) stage in the process of implementation, or whether it is both process and outcome of implementation.  This presentation describes an effort to design and evaluate a Sustainment Measurement System (SMS), based on interviews with 45 representatives of 10 grantees within 4 SAMHSA programs (Strategic Prevention Framework – State Initiative Grants, Sober Truth on Preventing Underage Drinking [STOP-Act], Garrett Lee Smith Suicide Prevention Program, and Prevention Practices in Schools). Data collection consisted of a semi-structured interview to identify experiences with implementation and sustainment barriers and facilitators; free list exercise to elicit participant conceptions of the word “sustainment” and what it will take to sustain their programs; and a checklist of Consolidated Framework for Implementation Research (CFIR) elements to identify which are important for sustainment.  A draft SMS will be introduced, along with a preliminary conceptual model to explain the sustainment of community-wide prevention programs that are designed for and implemented in community settings, and a qualitative comparative analysis (QCA) of CFIR domains to identify necessary and sufficient conditions for sustainment.

View Event →
Mar
6
12:00 PM12:00

Anne Sales: Frameworks to strategies: Designing implementation interventions

Frameworks to strategies: Designing implementation interventions

Anne Sales, Ph.D.
University of Michigan

ABSTRACT:
In this talk, I’ll describe current state of the science in terms of the use of implementation research frameworks to support the process of implementation and implementation planning, assess determinants of implementation success, and evaluate the outcomes of implementation efforts, as well as describe in more detail two commonly used and synergistic implementation strategies: feedback interventions and learning collaboratives. I’ll discuss how to assess when these strategies are likely to be useful, and when they may not be optimal in deciding how to support implementation of evidence based practices.

View Event →
Feb
27
12:00 PM12:00

Russ Glasgow: Experience applying pragmatic models for D&I research: Current and future directions for the RE-AIM planning and evaluation model

Experience applying pragmatic models for D&I research: Current and future directions for the RE-AIM planning and evaluation model

Russell Glasgow, Ph.D.
University of Colorado-Denver
Dissemination & Implementation Science Program of the ACCORDS Consortium

ABSTRACT:
This talk will discuss the need for pragmatic science, describe how pragmatic research designs are different from ‘research as usual’ and provide examples of pragmatic models and pragmatic measures. Using the My Own Health Report (MOHR) study as an example, I will outline the principles of the pragmatic model, RE-AIM.

View Event →
Feb
6
12:00 PM12:00

David Mandell: Fumbling towards a causal theory in implementation science

Fumbling towards a causal theory in implementation science

David Mandell, Sc.D.
University of Pennsylvania

ABSTRACT:
Implementation science research has led to a broad understanding and consensus regarding barriers and facilitators to successful implementation of evidence-based practices. This consensus has been captured in many heuristic frameworks. Most of these frameworks do not specify casual associations among variables however, limiting our ability to identify targets and mechanisms that could lead to effective and efficient implementation strategies. In this presentation, I will describe our research groups efforts to develop and test a theory of implementation that combines psychological theories of behavior change with organizational theory. I will present the model and its rationale, describe some of the methodological challenges associated with applying and testing these model, and give two examples from different settings that provide some validation of different parts of this model.

View Event →
Jan
23
12:00 PM12:00

Shri Narayanan: Deriving multimodal behavioral informatics for health applications

Deriving multimodal behavioral informatics for health applications

Shrikanth Narayanan, Ph.D.
University of Southern California

ABSTRACT:
The convergence of sensing, communication and computing technologies is allowing capture and access to data, in diverse forms and modalities, in ways that were unimaginable even a few years ago.  These include data that afford the analysis and interpretation of multimodal cues of verbal and non-verbal human behavior to facilitate human behavioral research and its translational applications in healthcare.  These data not only carry crucial information about a person’s intent, identity and trait but also underlying attitudes, emotions and other mental state constructs. Automatically capturing these cues, although vastly challenging, offers the promise of not just efficient data processing but in creating tools for discovery that enable hitherto unimagined scientific insights, and means for supporting diagnostics and interventions. Recent computational approaches that have leveraged judicious use of both data and knowledge have yielded significant advances in this regard, for example in deriving rich, context-aware information from multimodal signal sources including human speech, language, and videos of behavior. These are even complemented and integrated with data about human brain and body physiology.   This talk will focus on some of the advances and challenges in gathering such data and creating algorithms for machine processing of such cues.  It will highlight some of our ongoing efforts in Behavioral Signal Processing (BSP)—technology and algorithms for quantitatively and objectively understanding typical, atypical and distressed human behavior—with a specific focus on communicative, affective and social behavior. The talk will illustrate Behavioral Informatics applications of these techniques that contribute to quantifying higher-level, often subjectively described, human behavior in a domain-sensitive fashion. Examples will be drawn from mental health and well being realms such as Autism Spectrum Disorders, Couple therapy, Depression and Addiction counseling. 

View Event →
Jan
17
12:00 PM12:00

Darius Tandon: Engaging patients and stakeholders in dissemination and implementation research

Engaging patients and stakeholders in dissemination and implementation research

Darius Tandon, Ph.D.
Northwestern University

ABSTRACT:
Community engaged research involves the active and meaningful involvement of patient and community stakeholders throughout a research study.  For investigators conducting dissemination and implementation-related research, involving patient and community stakeholders can provide value as they conceptualize and implement their research. This presentation will highlight key terminology and characteristics of community-engaged research as well as specific strategies and key considerations for dissemination and implementation researchers seeking to meaningfully engage with patient and community stakeholders. 

View Event →
Dec
6
12:00 PM12:00

Shannon Wiltsey-Stirman: Assessment and classification of modification of evidence-based interventions in implementation research

Assessment and classification of modification of evidence-based interventions in implementation research

Shannon Wiltsey-Stirman, Ph.D.
Stanford University

ABSTRACT:
Implementation theory suggests that adaptation of evidence-based interventions is necessary for implementation and sustainment, and that appropriate adaptation may improve the interventions’ outcomes as well (Chambers, Stange, & Glasgow, 2012). Research indicates that providers routinely modify interventions in routine care settings (Cook et al., 2014; Stirman et al., 2013a). However, little is known about the impact of the changes that are made. The focus of this presentation will be to differentiate between modifications and adaptations to evidence-based interventions, to describe challenges to examining the impact of modifications and adaptations on implementation and intervention-level outcomes, to introduce a framework for classifying modifications and adaptations (Stirman et al., 2013b), and to demonstrate methods for making these classifications. Additionally, we will review findings related to the use of the framework, and present recommendations from the literature on how to make and assess the impact of adaptations in routine care settings.

View Event →
Nov
15
12:00 PM12:00

Lindsey Zimmerman: Participatory system dynamics modeling: Identifying, understanding and modifying drivers of implementation outcomes

Participatory system dynamics modeling: Identifying, understanding and modifying drivers of implementation outcomes

Lindsey Zimmerman, Ph.D.
National Center for PTSD

ABSTRACT:
Evidence-based practices (EBPs) may be adopted by providers, prioritized by leadership and supported by health system infrastructure, yet still not reach an adequate proportion of patients. In these systems, local staff expertise and operations data can be synthesized in a participatory system dynamics (PSD) model. PSD capitalizes on health record data, stakeholder expertise and simulation to optimize implementation. PSD modeling simulation empowers stakeholders, enabling them see the potential yield of implementation plans prior to implementation by zooming in on the entire EBP care continuum: referral to EBP, scheduling EBPs, staffing for EBPs, patient initiation of EBPs, completing a therapeutic dose of an EBP and patient benefit from an EBP. PSD models evaluate stakeholders' theories of clinic operation, testing explanatory mechanisms (i.e., local policies and procedures) by which EBP reach could be improved. In this way, the PSD process increases staff general capacity for quality improvement, while the PSD modeling tool identifies strategies most likely to improve implementation outcomes given local EBP-specific capacities and constraints.

View Event →
Nov
8
12:00 PM12:00

David Chambers: Adaptomes, learning systems, and convergence: Exploring frontiers in dissemination and implementation research

Adaptomes, learning systems, and convergence: Exploring frontiers in dissemination and implementation research

David Chambers, Ph.D.
NIH National Cancer Institute

ABSTRACT:
In the developing field of dissemination and implementation research, we have long focused on the assumption that the evidence base for interventions is pre-determined, and that the primary goal of the field is to maximize uptake of these “fixed” interventions. And yet we know that often large gaps exist in our understanding of how these evidence-based programs fit with the diverse group of populations, settings and individual needs. While adaptation of interventions has long been studied to improve fit, the research findings are primarily captured from efficacy and effectiveness trials. Dissemination and implementation research remains underutilized in expanding our knowledge of adaptation. This talk will discuss assumptions inherent in our research process that limit progress in this area, as well as introduce other areas for which the next generation of studies could make significant impact.

View Event →
Nov
1
12:00 PM12:00

Laura Damschroder: Effective implementation: A marriage of context and strategy

Effective implementation: A marriage of context and strategy

Laura Damschroder, M.P.H
VA Center for Clinical Management Research

ABSTRACT:
This presentation will cover use of the Consolidated Framework for Implementation Research (CFIR) to understand context, use of the Expert Recommendations for Implementing Change (ERIC) list of implementation strategies, and top-down and bottom-up efforts to identify appropriate strategies to accomplish implementation goals in challenging contexts. This presentation promises to trigger more questions than answers for a lively discussion.

View Event →
Sep
13
12:00 PM12:00

Patrick Sullivan: Use of agent-based models to inform HIV prevention for men who have sex with men

Use of agent-based models to inform HIV prevention for men who have sex with men

Patrick Sullivan, Ph.D.
Emory University

ABSTRACT: 
Agent-based models allow a virtual “laboratory” to evaluate counterfactual scenarios, evaluate the potential impact of combinations of HIV prevention, and benchmark levels of prevention services required to achieve population-level impacts of programs. We will review agent-based models of HIV prevention approaches for MSM in 5 countries, and describe how the findings of these models have been used to design combination prevention programs and guide programmatic goal setting. The results of agent-based models in different countries support the role of smart combinations of prevention interventions (e.g., PrEP, testing, treatment) and of ancillary services to increase the impact of specific modalities (e.g., adherence promotion for PrEP).

View Event →
Jan
13
12:00 PM12:00

C. Hendricks Brown: Designs that are used, or should be used, for dissemination and implementation research

Designs that are used, or should be used, for dissemination and implementation research

C. Hendricks Brown, Ph.D.
Northwestern University

ABSTRACT:
The fields of medicine and public health have made great progress in determining whether an intervention is efficacious or effective by conducting carefully crafted randomized clinical trials.  In contrast to these designs to evaluate an intervention’s efficacy or effectiveness, the designs for dissemination and implementation (D&I) research are not yet well established, a factor that has no doubt has impeded developing our knowledge of effective D&I.  By its very nature D&I research is intimately connected to understanding how programs, practices, or policies work in different contexts, so there is more attention in D&I research on external validity, as contrasted to the heavy emphasis on internal validity that many of the randomized efficacy and some effectiveness trials address. 

Results:  This presentation is a product of a workgroup meeting of 10 scientists and NIH staff, convened by NIH to facilitate D&I research. This committee addressed differences in terminology and provided a summary of the designs that have been used in D&I research, including both randomized and non-randomized studies.  We identified 27 designs and found it useful to categorize these designs into several broad categories.  One category of designs involves what can be termed the “traditional translational pipeline” of interventions that move step by step from efficacy, to effectiveness, to implementation research.  A second major class of designs involves “hybrid designs,” which combine elements of effectiveness and implementation research in one single design.  Thirdly, we describe designs that are focused on quality improvement as the primary goal, in contrast to producing generalizable knowledge.  Several of these latter designs borrow from diverse areas of engineering.  

Discussion:  We provide illustrations of these alternative designs and discuss cross-cutting issues, including community engagement and ethics in conducting implementation research.

View Event →
Feb
18
12:00 PM12:00

Lisa Saldana: Defining and measuring implementation processes and milestones: The stages of implementation completion

Defining and measuring implementation processes and milestones: The stages of implementation completion

Lisa Saldana, Ph.D.
Oregon Social Learning Center

ABSTRACT:
There is a dearth of knowledge regarding what it takes to implement and sustain evidence-based practices (EBP) into children’s social service settings. Some postulate that it takes greater time, coordination, and resources than other settings. In recognition of the complexity of the process, which involves planning, training, quality assurance, and interactions among developers, system leaders, practitioners, and consumers, the Stages of Implementation Completion (SIC) was developed. The SIC is an 8-stage tool that maps onto three phases of implementation (pre-implementation, implementation, and sustainability), and was developed to assess sites’ implementation process and obtainment of implementation milestones for Multidimensional Treatment Foster Care (MTFC). Using the SIC, the influence of MTFC sites’ early implementation behaviors on their abilities to obtain successful program start-up and competency have been demonstrated. Following the validation of the SIC for MTFC, efforts have begun to adapt and examine the SIC for other EBPs in children’s social service systems including juvenile justice, school, mental health, substance abuse, and child welfare.  Currently, the adaptation process is underway for multiple EBPs (e.g., MST, KEEP, Coping Cat) within these service settings. This presentation will describe the adaptation process and the complexities that arise when defining implementation processes and outcomes across different service systems. Psychometric properties of the MTFC SIC will be presented and the outcomes linked to decisions for future adaptations. The need to balance developer rigor with real-world site resources in defining successful implementation will be described. Initial evidence for common implementation factors identified between the different EBPs and social service systems will be presented. Common factors thought to be most critical for successful achievement and sustainment of competent EBP delivery for these vulnerable child populations will be highlighted. 

View Event →
Jan
14
12:00 PM12:00

John Landsverk: Emerging design issues in dissemination and implementation research

Emerging design issues in dissemination and implementation research

John Landsverk, Ph.D.
Oregon Social Learning Center

ABSTRACT:
Over the last decade, dissemination and implementation research has made great progress as a field but must still be considered a an emerging science in terms of conceptual models, design and measurement issues, and the building of a set of coherent substantive findings.  This presentation will provide an overview of design issues in the emerging field, going forward from the chapter on "Design and Analysis in Dissemination and Implementation Research,led by Landsverk, Brown and colleagues that was published in 2012 in the edited volume Dissemination and Implementation Research in Health: Translating Science to Practice (edited by Brownson, Colditz, and Proctor.  The presentation will overview the continuing issue of randomization in D&I research, the development of hybrid designs, the development in Consort 2010 of extensions to cluster randomized designs, pragmatic trials, in-inferiority and equivalence trials, and how these relate to the possible development of a consort extension to implementation designs, and two emerging issues related to the sustainment stage in implementation science and the emerging issue of designing evaluations for the rapid improvement of psychosocial interventions, especially with translation of such interventions into new technology platforms.

View Event →