Reporting of artificial intelligence prediction models

Reporting of artificial intelligence prediction models
The Lancet; London Vol. 393(10181), 20 April 2019 [NHS OpenAthens]

• The number of prediction model studies is increasing rapidly, with hundreds of different models being developed for some of the same targeted populations and outcomes.
• "Therefore, the clinical community must not get mesmerised by the artificial intelligence and machine learning revolution, and artificial intelligence and machine learning prediction models must be appropriately developed, evaluated, and—if needed—tailored to different situations before they are used in daily medical practice."

Abstract

Prediction models to support clinical decision making have existed for decades, and these include well known tools such as the Framingham Risk Score, QRISK3, Model for End-stage Liver Disease, ABCD2 score, and the Nottingham Prognostic Index.5Health-care professionals, medical researchers, policy makers, guideline developers, patients, and members of the general public are all potential users of prediction models. [...]concerns have been raised that artificial intelligence in clinical medicine is overhyped and, if not used with proper guidance, knowledge, or expertise, has methodological shortcomings, poor transparency, and poor reproducibility. Methodological concerns include an often incorrect focus on classification over prediction, overfitting (whereby too many predictors or features are included for the sample size), lack of robust assessment of predictive accuracy when used with other data than those from which they were developed (validation), weak and unbiased comparison with simpler modelling approaches, and lack of transparency of the artificial intelligence and machine learning algorithm, which limits independent evaluation. Clearly, the consequences of making a wrong or inaccurate prediction are substantial for the clinical application of a machine learning prediction model, such as the deep learning models for detection of stroke or wrist fractures approved by the US Food and Drug Administration. Therefore, the clinical community must not get mesmerised by the artificial intelligence and machine learning revolution, and artificial intelligence and machine learning prediction models must be appropriately developed, evaluated, and—if needed—tailored to different situations before they are used in daily medical practice.