With the availability of large databases of electronic medical records, biomedical researchers have rich resources for studies that predict prognoses and treatment outcomes. Yet, prediction models, including those that employ artificial intelligence technology, can be prone to data bias that may produce inaccurate results and unintended harm.
Causal inference — an approach used for many years in epidemiology and statistics fields to help researchers consider the cause and effect of variables upon outcomes — could help AI models overcome blind spots that lead to bias.
In a new article published in BMJ Open, University of Florida researchers, along with colleagues at Cornell University, University of Texas at Houston and UCLA, propose a protocol to develop the first reporting guidelines on causal prediction models in order to improve the overall quality, transparency and reproducibility of results produced by studies using these models.
In causal prediction models, variables can be modified, allowing researchers to evaluate alternative “what-if” scenarios, such as predicting if a change in behavior would reduce one’s risk of disease, or if one treatment would be better than another, said the paper’s senior author Mattia Prosperi, Ph.D., a professor in the department of epidemiology at the UF College of Public Health and Health Professions and UF College of Medicine, and PHHP’s coordinator of artificial intelligence.
“With these alternative predictions (called counterfactuals) one has to make sure that the modifiable variables are causally related to the outcome to be predicted, and that there is no underlying bias for which the predictions might be untrue and possibly harmful,” said Prosperi, director of the UF Data Intelligence Systems Lab. “Nowadays, there are several methodologies that can be used to develop causal prediction models and techniques to assess their performance and validity. However, this is also a relatively novel field, especially when we consider the fusion of traditional causal inference with machine learning/AI.”
While there are established guidelines for reporting on regular prediction models, currently no guidelines exist for causal prediction models. In their BMJ article, Prosperi and fellow authors outline a protocol for the development of causal prediction model guidelines. In collaboration with other scientists, they intend to develop a reporting standard over the next nine to 12 months, using focus groups and surveys.
“The availability of the reporting guidelines will help researchers carry out studies that comply with methodological rigor, completeness and reproducibility,” Prosperi said. “While adhering to a reporting guideline does not automatically ensure the quality of a study, it is for sure a step in that direction. Ultimately, causal prediction models that are proven to work in observational settings can be moved to the next step, that is prospective studies, and be used for interventions to better health outcomes in individuals and populations.”