Rxivist logo

A framework for making predictive models useful in practice

By Kenneth Jung, Sehj Kashyap, Anand Avati, Stephanie Harman, Heather Shaw, Ron Li, Margaret Anne Smith, Kim Fai Kenny Shum, Jacob Javitz, Yohan Vetteth, Tina Seto, Steven C Bagley, Nigam Shah

Posted 11 Jul 2020
medRxiv DOI: 10.1101/2020.07.10.20149419

Objective To analyze the impact of factors in healthcare delivery on the net benefit of triggering an Advanced Care Planning (ACP) workflow based on predictions of 12-month mortality. Materials and Methods We built a predictive model of 12-month mortality using electronic health record data and evaluated the impact of healthcare delivery factors on the net benefit of triggering an ACP workflow based on the models' predictions. Factors included non-clinical reasons that make ACP inappropriate, limited capacity for ACP, inability to follow up due to patient discharge, and availability of an outpatient workflow to follow up on missed cases. We also quantified the relative benefits of increasing capacity for inpatient ACP versus outpatient ACP. Results Work capacity constraints and discharge timing can significantly reduce the net benefit of triggering the ACP workflow based on a model's predictions. However, the reduction can be mitigated by creating an outpatient ACP workflow. Given limited resources to either add capacity for inpatient ACP versus developing outpatient ACP capability, the latter is likely to provide more benefit to patient care. Discussion The benefit of using a predictive model for identifying patients for interventions is highly dependent on the capacity to execute the workflow triggered by the model. We provide a framework for quantifying the impact of healthcare delivery factors and work capacity constraints on achieved benefit. Conclusion An analysis of the sensitivity of the net benefit realized by a predictive model triggered clinical workflow to various healthcare delivery factors is necessary for making predictive models useful in practice.

Download data

  • Downloaded 566 times
  • Download rankings, all-time:
    • Site-wide: 51,429
    • In health informatics: 201
  • Year to date:
    • Site-wide: 26,128
  • Since beginning of last month:
    • Site-wide: 42,544

Altmetric data

Downloads over time

Distribution of downloads per paper, site-wide