Opportunities and Challenges for Black-Box AI Implementations in Healthcare

Aligning implementation frameworks with Dutch stakeholder needs

Master thesis (2024)

Authors

J.A. Lin Technology, Policy and Management

Contributors

Sepinoud Azimi (graduation committee member)
J.M. Duran (graduation committee member)
Nitesh Bharosa (graduation committee member)
P. Budding (mentor)
Faculty
Technology, Policy and Management
More Info
expand_more

Published Date

17-10-2024

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Technology, Policy and Management

Abstract

This thesis explores the sociotechnical challenges surrounding the implementation of black-box AI in the Dutch healthcare sector, with a focus on aligning AI implementation frameworks to the needs of key stakeholders. The research addresses the growing demand for AI in healthcare as a potential solution to challenges like staff shortages, operational inefficiencies, and resource limitations. Despite its potential, the adoption of black-box AI, characterized by its opaque decision-making processes, faces resistance due to issues such as trust, lack of transparency, and difficulties in technical integration. To better understand these barriers, the study conducted a series of interviews with Dutch healthcare professionals, including technical, clinical, and business stakeholders, and analyzed existing AI implementation frameworks from the literature. The research identifies three primary challenges: trust in AI systems, technological integration with existing workflows, and organizational readiness, particularly regarding AI literacy.

In response to these findings, the study proposes a combined framework that bridges the sociotechnical gap in AI implementation, drawing from established implementation models and stakeholder insights. Key propositions include the use of Computational Reliabilism (CR) to build trust in AI systems without requiring full transparency, and the introduction of explainability tools to help end-users, particularly clinicians, trust the AI's outputs and ensure responsible use. Additionally, a maturity model is suggested to assess and improve organizational readiness for AI adoption, focusing on enhancing AI literacy and preparing healthcare institutions for more complex AI integrations. This framework and its propositions aim to foster a collaborative approach to AI adoption in Dutch healthcare, addressing both social and technical barriers to pave the way for ethical, reliable, and effective use of black-box AI.