Computer Science · Computational Theory and Mathematics
Comparing and Validating Machine Learning Models for <i>Mycobacterium tuberculosis</i> Drug Discovery
Thomas Lane (3337914), Daniel P. Russo (4587349), Kimberley M. Zorn (5146448), Alex M. Clark (98125), Alexandru Korotcov (659222), Valery Tkachenko (4587352), Robert C. Reynolds (37662), Alexander L. Perryman (652120), Joel S. Freundlich (261762), Sean Ekins (162)
See on the mapAbstract
Tuberculosis is a\nglobal health dilemma. In 2016, the WHO reported 10.4 million incidences\nand 1.7 million deaths. The need to develop new treatments for those\ninfected with <i>Mycobacterium tuberculosis</i> (<i>Mtb</i>) has led to many large-scale phenotypic screens and\nmany thousands of new active compounds identified <i>in vitro</i>. However, with limited funding, efforts to discover new active molecules\nagainst <i>Mtb</i> needs to be more efficient. Several computational\nmachine learning approaches have been shown to have good enrichment\nand hit rates. We have curated small molecule <i>Mtb</i> data and developed new models with a total of 18,886 molecules with\nactivity cutoffs of 10 μM, 1 μM, and 100 nM. These data\nsets were used to evaluate different machine learning methods (including\ndeep learning) and metrics and to generate predictions for additional\nmolecules published in 2017. One <i>Mtb</i> model, a combined <i>in vitro</i> and <i>in vivo</i> data Bayesian model\nat a 100 nM activity yielded the following metrics for 5-fold cross\nvalidation: accuracy = 0.88, precision = 0.22, recall = 0.91, specificity\n= 0.88, kappa = 0.31, and MCC = 0.41. We have also curated an evaluation\nset (<i>n</i> = 153 compounds) published in 2017, and when\nused to test our model, it showed the comparable statistics (accuracy\n= 0.83, precision = 0.27, recall = 1.00, specificity = 0.81, kappa\n= 0.36, and MCC = 0.47). We have also compared these models with additional\nmachine learning algorithms showing Bayesian machine learning models\nconstructed with literature <i>Mtb</i> data generated by\ndifferent laboratories generally were equivalent to or outperformed\ndeep neural networks with external test sets. Finally, we have also\ncompared our training and test sets to show they were suitably diverse\nand different in order to represent useful evaluation sets. Such <i>Mtb</i> machine learning models could help prioritize compounds\nfor testing <i>in vitro</i> and <i>in vivo</i>.