How Much Decision Power Should (A)I Have?

Abstract

Despite the growing potential of artificial intelligence (AI) in improving clinical decision making, patients' perspectives on the use of AI for their care decision making are underexplored. In this paper, we investigate patients' preferences towards the autonomy of AI in assisting healthcare decision making. We conducted interviews and an online survey using an interactive narrative and speculative AI prototypes to elicit participants' preferred choices of using AI in a pregnancy care context. The analysis of the interviews and in-story responses reveals that patients' preferences for AI autonomy vary per person and context, and may change over time. This finding suggests the need for involving patients in defining and reassessing the appropriate level of AI assistance for healthcare decision making. Departing from these varied preferences for AI autonomy, we discuss implications for incorporating patient-centeredness in designing AI-powered healthcare decision making.