Powerful predictive AI systems have demonstrated great potential in augmenting human decision-making. Recent empirical work has argued that the vision for optimal human-AI collaboration requires ‘appropriate reliance’ of humans on AI systems. However, accurately estimating the trustworthiness of AI advice at the instance level is quite challenging, especially in the absence of performance feedback pertaining to the AI system. In practice, the performance disparity of machine learning models on out-of-distribution data makes the dataset-specific performance feedback unreliable in human-AI collaboration. Inspired by existing literature on critical thinking and explanation-based human debugging, we propose the use of debugging an AI system as an intervention to foster appropriate reliance. In this paper, we explore whether a critical evaluation of AI performance within a debugging setting can better calibrate users’ assessment of an AI system and lead to more appropriate reliance. Through a quantitative empirical study (N = 234), we found that our proposed debugging intervention does not work as expected in facilitating appropriate reliance. Instead, we observe a decrease in reliance on the AI system after the intervention — potentially resulting from early exposure to the AI system’s weakness. We explored the dynamics of user confidence to help explain how inappropriate reliance patterns occur and found that human confidence is not independent of AI advice, which is potentially dangerous when trying to achieve appropriate reliance. Our findings have important implications for designing effective interventions to facilitate appropriate reliance and better human-AI collaboration

P3D is an injection moulding company that uses a technology called PRIM® (Printed Injection Mould) to create products for its customers. Different from traditional moulding companies, P3D’s main business model resolves around an efficient workflow and fast delivery of products to its customers. At the moment P3D is able to quickly fabricate and deliver its products be-cause of a small team of skilled designers and engineers who have lots of expertise in the field of injection moulding. Unfortunately, relying on the expertise of its employees, a lot of data about the company’s workflow is memorised or written down. This poses a problem as P3D would like to grow in the future and will not be able to entirely rely on memorised or analogue data.The goal of this project is to create a data management system (DMS) that digitises information about P3D its workflow and saves it in an easily accessible centralised system.