Novel evaluation approach for molecular signature-based deconvolution methods

NAVA, A.; ALVES DA QUINTA, D.; PRATO, L.; GIROTTI, R.; MORON, G.; LLERA, AS.; FERNÁNDEZ, EA.

JOURNAL OF BIOMEDICAL INFORMATICS

ACADEMIC PRESS INC ELSEVIER SCIENCE

Lugar: Amsterdam; Año: 2023

1532-0464

  The tumoral immunemicroenvironment (TIME) plays a key role in prognosis, therapeutic approach andpathophysiological understanding over oncological processes. Severalcomputational immune cell-type deconvolution methods (DM), supported by diversemolecular signatures (MS), have been developed to uncover such TIME interplayfrom RNA-seq tumor biopsies. MS-DM pairs were benchmarked against each other bymeans of different metrics, such as Pearson’s correlation, R2 and RMSE, butthese only evaluate the linear association of the estimated proportion relatedto the expected one, missing the analysis of prediction-dependent bias trendsand cell identification accuracy. We present a novel protocol composed of fourtests allowing appropriate evaluation of the cell type identificationperformance and proportion prediction accuracy of molecularsignature-deconvolution method pair by means of certainty and confidencecell-type identification scores (F1-score, distance to the optimal point anderror rates) as well the Bland-Altman method for error-trend analysis. Ourprotocol was used to benchmark six state-of-the-art DMs (CIBERSORTx, DCQ,DeconRNASeq, EPIC, MIXTURE and quanTIseq) paired to five murine tissue-specificMSs, revealing a systematic overestimation of the number of different celltypes across almost all methods.@font-face{font-family:Helvetica;panose-1:0 0 0 0 0 0 0 0 0 0;mso-font-charset:0;mso-generic-font-family:auto;mso-font-pitch:variable;mso-font-signature:-536870145 1342208091 0 0 415 0;}@font-face{font-family:"Cambria Math";panose-1:2 4 5 3 5 4 6 3 2 4;mso-font-charset:0;mso-generic-font-family:roman;mso-font-pitch:variable;mso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face{font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;mso-font-charset:0;mso-generic-font-family:swiss;mso-font-pitch:variable;mso-font-signature:-536859905 -1073732485 9 0 511 0;}@font-face{font-family:"Charis SIL";panose-1:2 11 6 4 2 2 2 2 2 4;mso-font-alt:Calibri;mso-font-charset:0;mso-generic-font-family:swiss;mso-font-pitch:auto;mso-font-signature:3 0 0 0 1 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal{mso-style-unhide:no;mso-style-qformat:yes;mso-style-parent:"";margin:0cm;mso-pagination:widow-orphan;font-size:12.0pt;mso-bidi-font-size:11.0pt;font-family:Helvetica;mso-fareast-font-family:Calibri;mso-fareast-theme-font:minor-latin;mso-bidi-font-family:Arial;mso-font-kerning:1.0pt;mso-ligatures:standardcontextual;mso-fareast-language:EN-US;}p.Default, li.Default, div.Default{mso-style-name:Default;mso-style-unhide:no;mso-style-parent:"";margin:0cm;mso-pagination:widow-orphan;mso-layout-grid-align:none;text-autospace:none;font-size:12.0pt;font-family:"Charis SIL",sans-serif;mso-fareast-font-family:Calibri;mso-fareast-theme-font:minor-latin;mso-bidi-font-family:"Charis SIL";color:black;mso-ligatures:standardcontextual;mso-ansi-language:ES-MX;mso-fareast-language:EN-US;}.MsoChpDefault{mso-style-type:export-only;mso-default-props:yes;mso-bidi-font-size:11.0pt;font-family:Helvetica;mso-ascii-font-family:Helvetica;mso-fareast-font-family:Calibri;mso-fareast-theme-font:minor-latin;mso-hansi-font-family:Helvetica;mso-bidi-font-family:Arial;mso-fareast-language:EN-US;}div.WordSection1{page:WordSection1;}