From experimenting with AI to a new way of working

Abstract

This thesis is conducted in collaboration with Directie X of the Ministry of Justice and Security (JenV), who aims to strengthen the innovation capacity within JenV to keep up with societal needs. One way they do this is by supporting AI experiments. These experiments are procedures undertaken to improve the current way of working, not just to validate hypotheses.The experiments are driven by two drivers: innovation and AI, which bring along some challenges: currently, JenV learns from the AI experiments, but the result is not an innovation. In addition, JenV is reluctant to implement innovative improvements as they are closely watched by society. Besides, the magnitude of these challenges increases when AI is involved. Hence, most of these AI experiments end after the proof of concept and experience a silent death.A strategy supported by a tool is desired to prevent the experiments’ results from being left unharnessed. Therefore, this graduation project will implement these takeaways by solving the following research question: “Why does experimenting with AI rarely lead to a new way of working at JenV?”
The research has provided insights into the many challenges behind this question; the challenges can be subdivided into five perspectives: strategy, ecosystem, results, process & governance and culture.During the research, it became clear that the technology (AI) was not a limiting factor of significant importance for implementing the outcome. However, the scope of this master thesis does not reach all experiments conducted at JenV but is specifically focused on AI experiments. The research has resulted in the following problem statement: “Initiators experience uncertainty prior to AI experiments because they have no clarity about the trajectory of the AI experiment and mainly lack clarity about who the stakeholders are and how they should approach them. This unclarity about the stakeholders, in turn, results in more uncertainty about the AI experiment.”Directie X should support the initiators to bring about implementation without being part of the whole process. Therefore, three principles have been developed. Directie X hands over to the initiators by translating the principles into a tool. The tool, therefore, only serves as a means of communication of the principles and acts as a hands-on concept that the initiators can get started with.The three principles:1. It’s all clear! (Clarity)2. Doing it with! (Engagement)3. It’s clear to all! (Transparency)
Ultimately, based on several design requirements, a toolkit has been developed that complies with the three principles: The CET toolkit. The CET toolkit helps to reduce the uncertainties of the initiators prior to an AI experiment by focusing on clarity about the stakeholders, engaging the stakeholders and transparency.The CET toolkit consists of an introductory page, Experience stories, six Stakeholder canvases and eight Reflection canvases.
The purposes of the different parts of the CET toolkit are the following:- The purpose of The Experience stories is that the initiators learn from others without there being consequences.- The purpose of the Stakeholder canvases is to clarify the stakeholders to the initiator to reduce the uncertainty prior to the experiment. In addition, it entails engagement by providing insights about the roles’ wants & needs, and concerns, which can function as guidelines for involving the roles.- The purpose of the Reflection canvases is to help the next initiator gain insights that can be incorporated in the Experience stories and learn by letting the initiator become aware of the actions taken during the AI experiment.
An evaluation of the concept answers whether the CET toolkit is viable, desirable and feasible and meets the design requirements. The assessment is done through an online collaborative validation session and an online survey. The review has led to a few adjustments. Therefore, the thesis finishes with recommendations in the field of exploring, designing and sharing.The CET toolkit is a validated first step to reduce the initiators’ uncertainties through clarity, engagement and transparency.
This thesis aims to contribute to the way of working of JenV. The result is a strategy supported by a tool. This thesis helps Directie X increase the chance AI experiments go into practice and lead to innovation.