Rxivist logo

Improving Hospital Readmission Prediction using Individualized Utility Analysis

By Michael Ko, Emma Chen, Pranav Rajpurkar, Ashwin Agrawal, Anand Avati, Andrew Ng, Sanjay Basu, Nigam Shah

Posted 29 Jul 2020
medRxiv DOI: 10.1101/2020.07.26.20156943

Importance: Machine learning (ML) models for allocating readmission-mitigating interventions are typically selected according to their discriminative ability, which may not necessarily translate into utility in allocation of resources. Objective: To determine whether ML models for allocating readmission-mitigating interventions are ranked differently based on their overall utility and their discriminative ability. Design: A retrospective analysis of ML models using claims data acquired from the Optum Clinformatics Data Mart. Setting: Health plan claims from all 50 states for commercially-insured individuals. Participants: 513,495 patients who were admitted as inpatients over the period January 2016 through January 2017. Main Outcomes and Measures: Maximum utility achieved by three machine learning models for allocating readmission-mitigating interventions, determined using cost accrued in the 90 days post-discharge of an index admission and estimated counterfactual cost. Data were analyzed between April 2019 and March 2020. Results: The study sample consisted of 513,495 patients (mean [SD] age 69 [19] years; 294,895 [57%] Female) mean 90 day cost of $11,552 for the study period. Allocating readmission-mitigating interventions based on a LightGBM model trained to predict readmissions achieved a maximum utility of $-12,645 per patient, and an AUC of 0.74 (95% CI 0.74, 0.75); allocating interventions based on a model trained to predict cost as a proxy achieved a higher maximum utility of $-12,472 per patient, and an AUC of 0.63 (95% CI 0.62, 0.63). A hybrid model combining both intervention strategies achieved a maximum utility of $-12,472, and an AUC of 0.71 (95% CI 0.71, 0.71), comparable with the best models on either metric. Conclusion and Relevance: We demonstrate that machine learning models may be ranked differently based on overall utility and discriminative ability. Machine learning models for allocation of limited health resources should consider directly optimizing for utility.

Download data

  • Downloaded 338 times
  • Download rankings, all-time:
    • Site-wide: 86,937
    • In health informatics: 318
  • Year to date:
    • Site-wide: 53,500
  • Since beginning of last month:
    • Site-wide: 55,183

Altmetric data

Downloads over time

Distribution of downloads per paper, site-wide