Aligning Objectives of Machine Learning with Clinical Care

Hybrid Event: Zoom

Abstract: The availability of large datasets in healthcare presents an opportunity to leverage machine learning techniques to learn potentially complex relationships between an individual’s characteristics, health status, and response to interventions.  However, there is often a mismatch between how such models are developed and clinical need. As a case study, we consider sepsis, the body’s overwhelming response to an infection, which is estimated to be associated with nearly 1 in 5 deaths. Many tools have been created to perform risk stratification for sepsis. Yet, most tools focus on modeling likelihood of developing sepsis prior to disease onset, and do not account for disease severity or evaluate with respect to clinical recognition of the disease. In my dissertation, I study how clinical needs can and should inform model development for greater impact in clinical care. First, in survival analysis, deep learning approaches optimize for discriminative performance, ignoring model calibration. However, well-calibrated survival curves that present realistic and meaningful probabilistic estimates of the true underlying survival process for an individual are essential in many clinical contexts. In light of the shortcomings of existing approaches, we propose a new training scheme for learning deep survival analysis models that achieves both strong discriminative performance and good calibration. Second, in causal inference, most efforts focus on learning accurate estimates of conditional average treatment effects (CATEs).  However, in many settings, clinicians only require a ranking of individuals based on their expected benefit from treatment. To address this mismatch, we propose a new approach that directly optimizes the ranking of individuals to maximize benefit. By recognizing and understanding clinical needs, we can focus our effort on models that better serve those needs and ultimately have the kind of impact we desire.