Rapid, responsive and relevant? A systematic review of rapid evaluations in healthcare

CECILIA VINDROLA PADROS; BRAGE, EUGENIA; GINGER JOHNSON

AMERICAN JOURNAL OF EVALUATION

SAGE

Año: 2020

1098-2140

Rapid evaluations are becoming more frequent in healthcare contexts. Changing healthcareclimates mean evaluators need to be responsive to changing priorities and deliver evaluationfindings within shorter timeframes (McNall and Foster-Fishman 2007). Timeliness has become afeature that might determine if and how findings can be used to inform decision-makingprocesses (McNall et al. 2004). However, the delivery of findings at a time when they can beactionable remains a challenge and research and evaluations continue to lag behind the needsof evidence-based decision-making (Riley et al. 2013; Glasgow et al. 2014). Riley andcolleagues (2013) argued that in order for research and evaluations to have an impact onhealthcare organization and delivery, they would need to align to the 3 Rs: rapid, responsiveand relevant. Alignment with the 3 Rs requires the creation of ?rapid-learning research systems?,which bring together researchers, funders, practitioners and community partners to ask relevantquestions and use efficient and innovative research designs (Riley et al. 2013). The challengethat remains is the creation of research and evaluation designs capable of delivering findings tothese systems when they can inform decision-making processes.Researchers have been experimenting with different types of research designs to makeevaluations more efficient and to organize regular feedback loops so findings can be shared atkey points in time. Rapid assessment procedures (RAP), rapid appraisals, rapid ethnographicassessments (REA) and rapid ethnographies were developed as research approaches (Beebe2014; Johnson and Vindrola-Padros 2017; Vindrola-Padros and Vindrola-Padros 2018), butrapid evaluation designs were also created through approaches such as rapid evaluationmethods (REM), real-time evaluations (RTE), rapid feedback evaluations (RFE) and rapid cycleevaluations (RCE). Over 10 years ago, McNall and Foster-Fishman (2007) reviewed thelandscape of rapid evaluation and appraisal methods (REAM), documenting the diversity ofrapid approaches and highlighting the challenges they shared. They identified an intrinsictension between speed and trustworthiness and argued that rapid approaches would need toaddress issues of validity and data quality to gain greater popularity in the evaluation landscape(McNall and Foster-Fishman 2007).Despite evident advances in the field of rapid evaluations, challenges remain in the way wedefine, design and implement rapid evaluations. For example, there is variability in the ways inwhich we define rapid timeframes as well as what we mean by evaluation (Nunns 2009; McNallet al. 2004). Short timeframes are often associated with evaluations that might appear to berushed, less rigorous and lacking engagement with theory (McNall and Foster-Fishman 2007).These assumptions can influence how evaluation findings are viewed and, ultimately, used inpractice (McNall and Foster-Fishman 2007). Evident trade-offs are present in relation to thebreadth and depth of data and, some services, interventions or contexts might be moreamenable to rapid evaluations, but this might not be true for others (Nunns 2009). Finally, gapsremain in our understanding of the value and use of rapid evaluations in comparison to longer-term evaluations.The purpose of this review was to grapple with these challenges and explore the ways in whichrapid evaluations have been used in healthcare. We considered common rapid evaluationdesigns, issues in implementation and strategies used for the sharing of findings and offerrecommendations for the design of evaluations over short periods of time. We identified gaps inknowledge and future areas of exploration.