Positivity Predictive Value surfaces as a complementary tool to assess the performance of virtual screening methods

MORALES, JUAN FRANCISCO; CHUGURANSKY, SARA; ALBERCA, LUCAS N.; ALICE, JUAN I.; GOICOECHEA, SOFÍA; RUIZ, MARÍA ESPERANZA; BELLERA, CAROLINA L.; TALEVI, ALAN

MINI-REVIEWS IN MEDICINAL CHEMISTRY

BENTHAM SCIENCE PUBL LTD

Lugar: Oak Park; Año: 2020

1389-5575

Background: Since their introduction in the virtual screening field, Receiver Operating Characteristic (ROC) -derived metrics have been widely used for benchmarking and optimization purposes of computational methods intended for virtual screening applications. Whereas in classification problems the ratio between sensitivity and specificity for a given score value is very informative, a practical concern in virtual screening campaigns is to predict the actual probability that a predicted hit will prove truly active when submitted to experimental testing (in other words, the Positivity Predictive Value - PPV). Estimation of such probability is however obstructed due to its dependency on the yield of actives of the screened library, which cannot be known a priori. Objective: To explore the use of PPV surfaces derived from simulation of ranking experiments as a complementary tool to ROC curves, for both benchmarking and optimization purposes. Method: The utility of the proposed approach is assessed in retrospective virtual screening experiments with four datasets used to infer QSAR classifiers: inhibitors of Trypanosoma cruzi trypanothione synthetase; inhibitors of Trypanosoma brucei N-myristoyltransferase; inhibitors of GABA transaminase and; anticonvulsant activity in the 6 Hz seizure model. Results: Besides illustrating the utility of PPV surfaces to compare the performance of machine learning models for virtual screening applications and to select and adequate score threshold, our results also suggest that ensemble learning provides models with better predictivity and more robust behavior. Conclusion: PPV surfaces are valuable tools to assess virtual screening tools and choose score thresholds to be applied in prospective in silico screens. Ensemble learning approaches seem to consistently lead to improved predictivity and robustness.