The principle of trust has been placed at the centre as an attitude for engaging with clinical machine learning systems. However, the notions of trust and distrust remain fiercely debated in the philosophical and ethical literature. In this article, we proceed on a structural level ex negativo as we aim to analyse the concept of “institutional distrustworthiness” to achieve a proper diagnosis of how we should not engage with medical machine learning. First, we begin with several examples that hint at the emergence of a climate of distrust in the context of medical machine learning. Second, we introduce the concept of institutional trustworthiness based on an expansion of Hawley’s commitment account. Third, we argue that institutional opacity can undermine the trustworthiness of medical institutions and can lead to new forms of testimonial injustices. Finally, we focus on possible building blocks for repairing institutional distrustworthiness.

@en

The advancement of AI-based technologies, such as machine learning (ML) systems, for implementation in healthcare is progressing rapidly. Since these systems are used to support healthcare professionals in crucial medical practices, their role in medical decision-making needs to be epistemologically and ethically assessed. However, a central issue at the intersection of the ethics and epistemology of ML has been largely neglected. This pertains to the careful scrutiny of how ML systems can degrade individuals’ epistemic standing as receivers and conveyors of knowledge and, thereby, perpetrate epistemic injustice. Since ML systems are powerful epistemic entities that are not easily contestable, and their decision-making rationale is often inaccessible, it is crucial to consider their role in creating imbalances in patients’ disfavor and the ways to mitigate such imbalances. This is especially important when it comes to interactions between patients and physicians, in which questions of credibility, trust, and understanding are central. Against this background, the overarching purpose of this dissertation is to fill this research gap by providing a framework to identify and, on occasion, mitigate epistemic injustices that are ML-induced, i.e., that emerge specifically due to the role that ML systems play in patient-physician interactions.@en

In this paper, we discuss epistemic and ethical concerns brought about by machine learning (ML) systems implemented in medicine. We begin by fleshing out the logic underlying a common approach in the specialized literature (which we call the informativeness account). We maintain that the informativeness account limits its analysis to the impact of epistemological issues on ethical concerns without assessing the bearings that ethical features have on the epistemological evaluation of ML systems. We argue that according to this methodological approach, epistemological issues are instrumental to and autonomous of ethical considerations. This means that the informativeness account considers epistemological evaluation uninfluenced and unregulated by an ethical counterpart. Using an example that does not square well into the informativeness account, we argue for ethical assessments that have a substantial influence on the epistemological assessment of ML and that such influence should not be understood as merely informative but rather regulatory. Drawing on the case analyzed, we claim that within the theoretical framework of the informativeness approach, forms of epistemic injustice—especially epistemic objectification—remain unaddressed. Our analysis should motivate further research investigating the regulatory role that ethical elements play in the epistemology of ML.

@en

In their contribution, Ugar and Malele shed light on an often overlooked but crucial aspect of the ethical development of machine learning (ML) systems to support the diagnosis of mental health disorders. The authors restrain their focus on pointing to the danger of misdiagnosing mental health pathologies that do not qualify as such within sub-Saharan African communities and argue for the need to include population-specific values in these technologies’ design. However, an analysis of the nature of the harm caused to said populations once their values remain unrecognised is not offered. [...]@en

Machine learning (ML) systems play an increasingly relevant role in medicine and healthcare. As their applications move ever closer to patient care and cure in clinical settings, ethical concerns about the responsibility of their use come to the fore. I analyse an aspect of responsible ML use that bears not only an ethical but also a significant epistemic dimension. I focus on ML systems' role in mediating patient-physician relations. I thereby consider how ML systems may silence patients' voices and relativise the credibility of their opinions, which undermines their overall credibility status without valid moral and epistemic justification. More specifically, I argue that withholding credibility due to how ML systems operate can be particularly harmful to patients and, apart from adverse outcomes, qualifies as a form of testimonial injustice. I make my case for testimonial injustice in medical ML by considering ML systems currently used in the USA to predict patients' risk of misusing opioids (automated Prediction Drug Monitoring Programmes, PDMPs for short). I argue that the locus of testimonial injustice in ML-mediated medical encounters is found in the fact that these systems are treated as markers of trustworthiness on which patients' credibility is assessed. I further show how ML-based PDMPs exacerbate and further propagate social inequalities at the expense of vulnerable social groups.@en

In my paper entitled 'Testimonial injustice in medical machine learning',1 I argued that machine learning (ML)-based Prediction Drug Monitoring Programmes (PDMPs) could infringe on patients' epistemic and moral standing inflicting a testimonial injustice.2 I am very grateful for all the comments the paper received, some of which expand on it while others take a more critical view. This response addresses two objections raised to my consideration of ML-induced testimonial injustice in order to clarify the position taken in the paper. The first maintains that my critical stance toward ML-based PDMPs idealises standard medical practice. Moreover, it claims that the ML-induced testimonial injustice I discuss is not substantially different from situations in which it emerges in human-human interactions. The second claims that my analysis does not establish a link to issues of automation bias, even if these are to be considered the core of testimonial injustice in ML. @en

Artificial intelligence-based (AI) technologies such as machine learning (ML) systems are playing an increasingly relevant role in medicine and healthcare, bringing about novel ethical and epistemological issues that need to be timely addressed. Even though ethical questions connected to epistemic concerns have been at the center of the debate, it is going unnoticed how epistemic forms of injustice can be ML-induced, specifically in healthcare. I analyze the shortcomings of an ML system currently deployed in the USA to predict patients’ likelihood of opioid addiction and misuse (PDMP algorithmic platforms). Drawing on this analysis, I aim to show that the wrong inflicted on epistemic agents involved in and affected by these systems’ decision-making processes can be captured through the lenses of Miranda Fricker’s account of hermeneutical injustice. I further argue that ML-induced hermeneutical injustice is particularly harmful due to what I define as an automated hermeneutical appropriation from the side of the ML system. The latter occurs if the ML system establishes meanings and shared hermeneutical resources without allowing for human oversight, impairing understanding and communication practices among stakeholders involved in medical decision-making. Furthermore and very much crucially, an automated hermeneutical appropriation can be recognized if physicians are strongly limited in their possibilities to safeguard patients from ML-induced hermeneutical injustice. Overall, my paper should expand the analysis of ethical issues raised by ML systems that are to be considered epistemic in nature, thus contributing to bridging the gap between these two dimensions in the ongoing debate.@en