Rapid Cycle Systems Modeling and Decision Sampling to Inform Development and Implementation of System-Wide Innovations to Promote Pediatric Mental and Behavioral Health

Thomas I. Mackie, PhD, MPH
Rutgers School of Public Health

R. Christopher Sheldrick, PhD
Boston University School of Public Health

ABSTRACT:
Calls have been made for greater application of simulation modeling and decision sciences to expedite research evidence use into mental health policy. Funded by W.T. Grant Foundation, Drs. Mackie and Sheldrick will report on a study, entitled Research Evidence Adoption for Child Health (REACH), in which they propose the use of Rapid Cycle Systems Modeling (RCSM), as an implementation strategy that aims to assist local decision-makers in needs assessment and prioritization, knowledge exchange, and consensus building when developing system-wide innovations. RCSM requires that one (1) identify gaps in the information available and questions requiring resolve to inform stakeholders’ decisions, (2) build a simulation model and identify relevant estimates to parameterize the model (e.g., evidence reviews), and (3) conduct group facilitation sessions to asses model utility and need for further adaptation.

To demonstrate utility, we provide an illustrative case study from REACH in which we specifically sought to identify how decision-makers developed protocols to identify and treat the trauma of children entering foster care. In conducting the first step of RCSM, we identified information gaps by employing a “decision sampling” framework in qualitative interviews.  In this approach, we anchored key informants’ responses on a recent index decision to minimize recall and response bias and potential desirability of evidence use. Respondents (n=31) described a continuum of 14 decision points relevant to five domains when developing a system-wide approach to identify and address trauma, including: 1. reach of the screening protocol, 2. content of the screening tool, 3. threshold for referral, 4. resources for screening startup and sustainment, and 5. system capacity to respond to identified needs. In the second stage of RCSM, we built a Monte Carlos simulation model that responded to the gaps in available information identified across this decision continuum and used additional data collected in the interviews, coupled with evidence reviews, to inform the model structure and parameterization. Finally, we facilitated group interviews (n=4) learning that the model developed was assessed by relevant stakeholders to hold utility and face validity. We will present an overview of this work and the model to illustrate potential utility of RCSM and decision sampling in other research domains.