Novel evaluation approach for molecular signature-based deconvolution methods

NAVA, AGUSTÍN; ALVES DA QUINTA, DANIELA; PRATO, LAURA; GIROTTI, MARÍA ROMINA; MORON, GABRIEL; LLERA, ANDREA S.; FERNÁNDEZ, ELMER A.

JOURNAL OF BIOMEDICAL INFORMATICS

ACADEMIC PRESS INC ELSEVIER SCIENCE

Año: 2023 vol. 142

1532-0464

The tumoral immune microenvironment (TIME) plays a key role in prognosis, therapeutic approach and pathophysiological understanding over oncological processes. Several computational immune cell-type deconvolution methods (DM), supported by diverse molecular signatures (MS), have been developed to uncover such TIME interplay from RNA-seq tumor biopsies. MS-DM pairs were benchmarked against each other by means of different metrics, such as Pearson´s correlation, R2 and RMSE, but these only evaluate the linear association of the estimated proportion related to the expected one, missing the analysis of prediction-dependent bias trends and cell identification accuracy. We present a novel protocol composed of four tests allowing appropriate evaluation of the cell type identification performance and proportion prediction accuracy of molecular signature-deconvolution method pair by means of certainty and confidence cell-type identification scores (F1-score, distance to the optimal point and error rates) as well the Bland-Altman method for error-trend analysis. Our protocol was used to benchmark six state-of-the-art DMs (CIBERSORTx, DCQ, DeconRNASeq, EPIC, MIXTURE and quanTIseq) paired to five murine tissue-specific MSs, revealing a systematic overestimation of the number of different cell types across almost all methods.