Abstract: There is increased interest in and potential benefits from using large language models (LLMs) in medicine. As a result, applications powered by LLMs are increasingly used to perform medical tasks without the underlying language model being trained on medical records and without verifying their purported benefit in performing those tasks. Using examples of studies conducted at Stanford, we will build the case for how the adoption of LLMs in medicine needs to be shaped by performing the evaluations that specify the desired benefits, and verify those benefits via testing in real-world deployments.

Speaker Bio: Dr. Nigam Shah is Professor of Medicine at Stanford University, and Chief Data Scientist for Stanford Health Care. His research is focused on bringing AI into clinical use, safely, ethically and cost-effectively. Dr. Shah is an inventor on eight patents, has authored over 200 scientific publications, and has co-founded three companies. Dr. Shah was inducted into the American College of Medical Informatics (ACMI) in 2015 and the American Society for Clinical Investigation (ASCI) in 2016. He holds an MBBS from Baroda Medical College, India, a PhD from Penn State University and completed postdoctoral training at Stanford University.