Beyond Jacobson & Truax: Modern Methods for Estimating Clinically Significant Change

Antonio Morgan-Lopez, PhD
RTI International

ABSTRACT:
In the majority of randomized controlled trials (RCTs) the focus is on differences in the average change over time on outcomes across intervention conditions, with variation in individual trajectories often treated as nuisance. In contrast, a primary focus on inferences regarding the improvement (or worsening) of individual participants is best represented by clinical significance or clinically significant change (CSC). One of the primary tools in the assessment of CSC is Jacobson and Truax (1991’s) Reliable Change Index (RCI). The RCI is still very popular, as evidenced by 12,000 total citations and over 300 citations in 2023 alone. However, three specific limitations have been identified with the RCI: a) the RCI estimate is based on a pre-post difference score, b) the scores upon which the RCI estimate is based (typically total scores) often contain both measurement bias and measurement error and c) the RCI standard error of measurement (SEM) is erroneously assumed to be constant across participants and time. We present an approach that addresses all three limitations simultaneously: a) scale score and SEM estimation using moderated nonlinear factor analysis and b) RCI estimation using a modification of a three-level multilevel model with modeling of observation-specific measurement uncertainty. We focus on two illustrations: one from a treatment trial targeting comorbid PTSD/alcohol use disorder among OEF/OIF Veterans and second from a school-based selective preventive intervention trial targeting conduct problems in late elementary through high school. We also provide sample SAS code for implementation that is easily accessible to those with experience with conventional multilevel models.