Despite its enormous economical and societal impact, lack of human-perceived control and safety is re-defining the design and development of emerging AI-based technologies. New regulatory requirements mandate increased human control and oversight of AI, transforming the development practices and responsibilities of individuals interacting with AI. In this paper, we present the SPATIAL architecture, a system that augments modern applications with capabilities to gauge and monitor trustworthy properties of AI inference capabilities. To design SPATIAL, we first explore the evolution of modern system architectures and how AI components and pipelines are integrated. With this information, we then develop a proof-of- concept architecture that analyzes AI models in a human-in-the- loop manner. SPATIAL provides an AI dashboard for allowing individuals interacting with applications to obtain quantifiable insights about the AI decision process. This information is then used by human operators to comprehend possible issues that influence the performance of AI models and adjust or counter them. Through rigorous benchmarks and experiments in real- world industrial applications, we demonstrate that SPATIAL can easily augment modern applications with metrics to gauge and monitor trustworthiness, however, this in turn increases the complexity of developing and maintaining systems implementing AI. Our work highlights lessons learned and experiences from augmenting modern applications with mechanisms that support regulatory compliance of AI. In addition, we also present a road map of on-going challenges that require attention to achieve robust trustworthy analysis of AI and greater engagement of human oversight.@en

The ageing European population and the expected increasing number of medical emergencies put pressure on the medical sector and existing emergency infrastructures, which calls for new innovative digital solutions. In parallel, the increasing utilization of the Internet of Things (IoT) has enabled the collection of real-Time data, allowing for the autonomous detection of acute medical emergencies. In this context, this paper presents two distinct machine learning (ML) models that leverage electrocardiogram (ECG) sensor data to autonomously detect Myocardial Infarctions (MI), a leading cause of emergencies. These models are intended to be integrated into an IoT-enabled next-generation emergency communications system (NG112) capable of detecting emergencies, initiating emergency calls (eCalls), and providing relevant information to emergency call takers, which reduces response time. To realize this, two disparate models working on fundamentally different data structures are proposed and compared: A one-dimensional convolutional neural network (CNN) operating on the raw ECG signals and a GoogLeNet-based model trained on ECG images. The PTB-XL dataset is used to evaluate the proposed models, and the results indicate the 1D CNN exhibits a favourable trade-off between precision and recall for the eCall use case. Finally, the paper also discusses applying eXplainable AI (XAI) methods to achieve explainability for the ML models, paving the way for an accountable and reliable implementation in safety-critical systems.

@en

Various AI systems have taken a unique space in our daily lives, helping us in decision-making in critical as well as non-critical scenarios. Although these systems are widely adopted across different sectors, they have not been used to their full potential in critical domains such as the healthcare sector enabled by the Internet of Things (IoT). One of the important hindering factors for adoption is the implication for accountability of decisions and outcomes affected by an AI system, where the term accountability is understood as a means to ensure the performance of a system. However, this term is often interpreted differently in various sectors. Since the EU GDPR regulations and the US congress have emphasised the importance of enabling accountability in AI systems, there is a strong demand to understand and conceptualise this term. It is crucial to address various aspects integrated with accountability and understand how it affects the adoption of AI systems. In this paper, we conceptualise these factors affecting accountability and how it contributes to a trustworthy healthcare AI system. By focusing on healthcare IoT systems, our conceptual mapping will help the readers understand what system aspects those factors are contributing to and how they affect the system trustworthiness. Besides illustrating accountability in detail, we also share our vision towards causal interpretability as a means to enhance accountability for healthcare AI systems. The insights of this paper shall contribute to the knowledge of academic research on accountability, and benefit AI developers and practitioners in the healthcare sector. @en