Rollout designs in implementation research:  Often necessary and sometimes preferred.

C. Hendricks Brown, PhD
Northwestern University

Gregory Simon, MD, MPH
Kaiser Permanente Washington Health Research Institute

ABSTRACT:
Background: Rollout designs, which include stepped wedge designs, are defined by staggered implementation of new or alternative programs or services. Critiques of stepped wedge and other rollout designs have raised concerns regarding the confounding of true implementation or program effects with unrelated, global changes in service delivery, with some recommending they only be used when traditional parallel-group designs are not practicable. However, rollout designs may sometimes be more suitable than traditional parallel group designs for ethical, scientific, or practical reasons.

Results: We define and provide rationale for and examples of stepped wedge and the larger class of rollout designs, in which all participating units receive a new program or service implementation. Staged implementation in a rollout design may be necessary when denying, rather than delaying, implementation of a known effective service is ethically unacceptable. Scientifically, stepped wedge has increased statistical power relative to an equivalent parallel group design, and some rollout designs have the capability to compare different phases of implementation and sustainment. A rollout design may be practically necessary either because of limited resources and other logistical challenges or community requirements that no site serve as a control. Examples of completed and ongoing rollout trials illustrate how these ethical, scientific, and practical considerations influenced trial designs.

Conclusions: Stepped wedge and other rollout trial designs may be well suited to evaluation of implementation strategies or policy changes. In implementation trials, rollout designs may be necessary for practical reasons, may be required for ethical reasons, and may be preferred for scientific reasons. We summarize when such rollout designs have advantages and drawbacks.

How to Make Scientific Inferences and Conduct Power Analyses for Randomized Implementation Rollout Trials

C. Hendricks Brown, PhD
Northwestern University

Ian Cero, PhD
University of Rochester Medical Center

ABSTRACT:
This two-part presentation continues the virtual presentations on Randomized Implementation Rollout Designs and Trials, which include Stepped Wedge Implementation Designs.  These designs are commonly used to examine how well an evidence-based intervention or package is being implemented in community or healthcare settings.  The multitude of implementation research questions and specific hypotheses suggest the need for diverse randomized rollout implementation trial designs, assignment principles and procedures, and statistical modeling.  In the first part we discuss key research questions and identify mixed effect models for randomized implementation rollout trials involving 1) a single implementation strategy that tests how this strategy varies over time and/or resources that are allocated, 2) comparison of two distinct implementation strategies, and 3) three distinct strategies or components tested in a single trial. 

In the second part of the presentation we present the use of Rollout, a general statistical package written in R that can be used to conduct detailed statistical power and sample size analyses for diverse rollout designs.  Users specify both the underlying generative data model as well as the analytic model and output includes power and bias in the parameters.  We discuss how the package can account for misspecified modeling and robustness.  Only limited knowledge of R is necessary to use this package, and we provide examples for planning new implementation trials and examining the effects on power when modifications of a design during the conduct of a trial are necessary.