Estimating the reproducibility of psychotherapy effects in mood and anxiety disorders: the possible utility of multicenter trials

FLÜCKIGER, CHRISTOPH; PAUL, JESSICA; HILPERT, PETER; VÎSL?, ANDREEA; GÓMEZ PENEDO, JUAN-MARTIN ; PROBST, GRETA HELENE; WAMPOLD, BRUCE E.

WORLD PSYCHIATRY

ELSEVIER MASSON

Año: 2021 vol. 20 p. 445 - 446

1723-8617

Estimating the reproducibility of psychotherapy effects is es­sential. This is particularly crucial for trials with largeeffects, as the inclusion of false-positive trials can lead to erroneous con­clusionsabout treatment efficacy in research syntheses1. Multicenter studies allow researchers to estimate thereproduc­ibility of effects directly across centers under comparable studyconditions (e.g., comparable enrollment procedures, inclusion/exclusioncriteria, assessment plans). In an important sense, imple­mentation of trialsat various centers is close to a direct replication of findings. Accordingly,recent standards recognize the benefit of describing individual center effectsin multicenter studies2. We aimed to review what we know about center effects inmul­ticenter trials with psychotherapy components for the treatment of mood andanxiety disorders. We examined the extent to which such multicenter trials: a)reported the variability of treatment outcomes for individual centers (i.e.,random center effects) and/ or b) provided an estimate of the strengths oftreatment by center interactions (i.e., fixed center effects)3. To obtain a representative sample of recent multicenterstud­ies, we conducted on July 18, 2020 a systematic search of studies indexedbetween 2010 and 2020 in Medline, PsycINFO and Edu­cational ResourcesInformation Center (ERIC). We used the key words “multicenter or multi-center”combined with “psychother­apy or therapy or counseling” and “depression oranxiety” and publication type “clinical trial” and “adult population”. Weidenti­fied 184 papers, of which 30 referred to treatment outcomes in amulticenter randomized clinical trial (overall 6,638 patients, range 22-1025).Descriptive characteristics of the 30 identified multi­center studies can beobtained from the authors upon request. In all 30 reports, “multicenter” was mentioned in thetitle or ab­stract and in the Methods section. The number of centers rangedfrom 2 to 30, but in four reports this number was not reported. The majority ofthe trials investigated treatment efficacy (e.g., changes in symptoms) and fourstudies investigated economic outcomes (e.g., cost-effectiveness analyses). In20 studies, at least one signifi­cant treatment effect was reported (max.Cohen’s d ranged from 0.23 to 3.44). Only one(3%) out of the 30 studies4 consideredsites a random factor, thereby permitting conclusions about variability in out­comesdue to sites in general. Only three (10%) studies5-7 reportedan estimate of the treatment by center interactions. Furthermore, seven studiesreported that center effects were “not significant”, without furtherspecification of the effect. Among the seven stud­ies with large significanttreatment contrasts (max. Cohen’s d >0.80), only one4 reporteda statistical estimate of a center effect. One ofthe strengths of multicenter studies is the opportunity to estimate thereproducibility of effects. The results of our sys­tematic review indicatethat, although studies state clearly that they involve multiple sites and oftenindicate that this adds to the importance of the trial, they typically do notuse the full potential of this design to estimate center effects (either randomor fixed), thereby obscuring evidence about reproducibility of effects. To properlyassess the degree to which results are reproduc­ible, we recommend that theauthors of multicenter studies re­port the outcomes for all centers andestimate center effects (i.e., differences in effects amongst centers)8.