Recent advancements in artificial intelligence (AI), particularly in deep learning, have significantly enhanced AI capabilities but have also led to more complex and less interpretable algorithms. This research addresses the challenge of Explainable AI (XAI) by focusing on enhancing the interpretability of AI decisions through the use of Explainable User Interfaces (XUI). The study identifies two primary knowledge gaps: the predominance of XAI research targeting technically skilled users, neglecting the end-user who often lacks technical expertise, and the insufficient exploration of user-centric design principles in real-world XUI applications.

The research adopts the Design Science Research Method (DSRM) to develop an XUI tailored for the FOKUS project, which uses Electrocardiogram (ECG) data to detect myocardial infarctions. The study emphasises the strategic application of interactive design principles such as complementary naturalness, flexibility in explanation methods, and responsiveness through progressive disclosure to improve the system’s interpretability. Notably, sensitivity to context and mind, though not initially implemented, emerged as a critical design principle from the analysis and was subsequently positioned at the pinnacle of a restructured pyramid model of design principles.

Key findings highlight the effectiveness of the selected design principles in enhancing interpretability and underscore the importance of involving stakeholders early in the development process to align the XAI and XUI with end-user needs. The research proposes a structured design approach framework for XUI, involving sequential phases from pre-XAI to XUI design, to systematically integrate user feedback and improve the design iteratively. The proposed framework restructured pyramid model of the design principles aim to guide future developments in XAI and XUI, enhancing their practical application and effectiveness in various contexts.

Explainable AI (XAI) systems are rapidly gaining significance. While frameworks for XAI interpretability for experts abound, metrics for laypeople’s comprehension are absent. This study addresses this gap by investigating interpretability factors from both developer and layperson perspectives. The core research question is: "How can XAI developers assess to what extent XAI is interpretable for laypeople?".

Applying the Design Science Research Methodology, findings from multiple literature reviews are combined to construct a preliminary XAI interpretability framework for laypeople, featuring crucial factors and their relationships, as well as associated principles. The proposed framework underwent validation through semi-structured interviews with 12 XAI experts, informing revisions and refinement of our key principles. Subsequent layperson surveys, considering a specific use case, offered insights into preferences about interpretability factors, informing further refinement.

The final theoretical framework highlights pivotal factors including simplicity, transparency, comprehensiveness, complexity, clarity, generalizability, trustworthiness, explanation fidelity, model fidelity, intentionality, relevance, affordance, coherence with prior beliefs, and actionability. Surrounding the framework are key principles emphasizing trustworthiness, relevance, simplicity, clarity, coherence, intentionality, actionability, fidelity, contextualization, and ethical considerations, serving as actionable guidelines for XAI developers and researchers.

The implications of the study are profound, offering valuable insights for advancing XAI research and system design. The refined framework and principles act as a foundation for both novice and experienced XAI developers, fostering interdisciplinary research among AI, human-computer interaction, psychology, and philosophy experts. These findings can drive the responsible adoption of AI systems across sectors like healthcare, finance, and transportation, while informing policies and regulations governing AI technologies. Our study promotes responsible AI practices, enhancing user trust and understanding, while facilitating the creation of more effective guidelines and standards.