This research looked into Algorithmic Systems used in the execution of the SUWI-law by executing agencies. It analysed the use and governance of these systems and their possible harms to citizens and society by means of an Actor Analysis and System-Theoretical Process Analysis (STPA), supported by semi-structured interviews.

It aimed to answer the following research question:
How can a systems safety perspective combined with an actor perspective be used to provide a sociotechnical description of algorithmic systems, their governance and possible hazards at agencies executing the SUWI-law?

By means of the Actor Analysis, this research was able to describe the actor field and implications of it on the problem situation. The main insight from the actor field was the presence of ‘double binds’, several value conflicts that trouble use of AS. STPA, a method that focusses on safety of a system as a whole rather than individual components, provided the ability to map algorithmic systems within their sociotechnical context. It focussed on three hazards, which are system states that lead to accidents. These three hazards were the use of flawed logic in algorithmic systems, the use of flawed information in algorithmic system, and the flawed understanding citizens have of the role of the systems. They were explored through a Safety Control Structure, which displays controlled processes and those controlling them, and the construction of causal scenarios.
Semi-structured interviews focussed on the governance of algorithmic systems, and possible improvements. They provided empirical input for STPA.
The main contributions of this research are the proposal it makes for a sociotechnical description of algorithmic systems and their harms, using concepts derived from STPA. Such descriptions can be used to build shared understanding of algorithmic systems and their harms. It showed how such a description can be constructed using an Actor Analysis, interviews, and STPA, describing how these methods work together.

The thesis explores the complexities and risks associated with Automated Decision-Making (ADM) systems in social welfare, focusing on the phenomenon of ”digital cages.” The digital cage concept refers to a situation where rigid systems, perpetuated through algorithmic misclassification, inadvertently trap citizens in bureaucratic complications without recourse. This thesis analyzes the digital cages concept from a system safety perspective, on the example of the Dutch Toeslagenaffair. The thesis centers on understanding how digital cages form within social welfare systems and seeks methods to mitigate their emergence through targeted interventions using system safety theory. The pivotal question addressed is: ”What are safety control interventions to curb the emergence of algorithmic decision-making systems-induced digital cages in the context of national social welfare administration in the Netherlands?” The research concludes by highlighting the potential of assumption-based leading indicators. Assumptionbased leading indicators are predictive measures that rely on underlying assumptions about how certain inputs or actions correlate with future outcomes. The leading indicators provide a novel approach to enhancing system safety in social welfare administration by addressing the underlying assumptions of the system in operation. This offers a substantial potential to transform how social welfare systems manage and utilize ADM, potentially reducing the incidence of digital cages. By detecting early signs of hazardous systems states digital cages can be mitigated before they solidify into systemic issues. The thesis not only addresses the immediate concerns regarding digital cages but also opens pathways for future inquiries into safer and more equitable administrative practices. Future research is suggested to further refine these indicators and explore their applicability in diverse administrative contexts, ensuring they can effectively adapt to the evolving landscape of public administration and technology.

The electricity grid in the Netherlands is highly congested and poses a significant challenge due to increasing demand and the transition to sustainable energy sources, which overwhelms the existing infrastructure. To address this issue, Distribution System Operators (DSOs) like Alliander are tasked with managing waiting lists for power connections and transport capacity requests. The traditional First-Come-First-Serve (FCFS) principle unintentionally delays transport requests that can be argued are more important to society. This motivated the policymakers to propose a prioritization framework based on societal importance. This framework prioritizes clients in categories such as congestion softeners, safety, basic needs, and sustainability, aiming to benefit society as a whole.

However, the introduction of this framework raises several implications, particularly concerning distributive justice. To systematically identify and deliberate these implications, a Design Science Research methodology was used. This methodology involved a comprehensive literature review across various research areas including Energy Justice, Waiting List Management, and Distributive Justice. Subsequently, a qualitative and quantitative analysis was conducted to understand the impacts on waiting times for both priority and non-priority clients.

Stakeholder opinions were analyzed across different industries, revealing concerns and suggestions regarding the social prioritization framework. The quantitative analysis estimated the real impact on waiting lists, considering technical constraints and proposing distributive justice metrics such as time equity disparity and change in the estimated time-to-solution ratio.

The analysis of real waiting lists demonstrated mixed outcomes, with priority clients experiencing decreases in waiting times while non-priority clients faced delays. Some non-priority clients were needlessly delayed. Distributive justice metrics highlighted disproportionate burdens on non-priority clients. Moreover, the framework's impacts extended beyond delays to financial, and psychological effects.

Recommendations for future research include incorporating historical data, obtaining more client information, improving stakeholder feedback loops, and further analyzing other aspects of Energy Justice. Additionally, connecting quantitative implications to societal values could enhance understanding of the framework's benefits to society.

Overall, this research provides valuable insights into the implications of implementing a social prioritization framework for electricity grid waiting lists, highlighting the importance of considering distributive justice and societal impacts in policy development and decision-making processes.

This thesis investigates the integration of Large Language Models (LLMs) within large organizations operating in low resource language (LRL) regions, with a particular focus on Íslandsbanki, one of the prominent banks in Iceland currently grappling with the adoption of LLM technology. The research aims to understand the frameworks governing LLM integration within the bank, explore stakeholder perspectives on the integration process, and identify blind spots and gaps in their understanding of associated risks in order to understand what a comprehensive approach would look like.

The methodology had mixed methods. It consisted of a comprehensive literature review followed by interviews with stakeholders involved in the LLM integration at Íslandsbanki. Subsequently, an Impact-Probability Assessment survey is conducted, where stakeholders are asked to evaluate the identified risks in terms of their potential impact and probability. This approach facilitates the identification and prioritization of risks, thereby enabling future integrators to navigate the integration process more effectively. Furthermore, data analysis techniques were applied to the survey results to provide quantitative insights into the identified gaps and perspectives.

Findings reveal that perspectives on risks associated with LLM integration vary significantly, particularly between individuals with technical backgrounds and decision-makers within the bank. Moreover, the concept of text inflation emerges as a novel risk factor, highlighting the importance of considering diverse perspectives in risk assessment and mitigation strategies. Emphasis is placed on adopting a sociotechnical perspective to comprehensively assess risks at an organizational level.

This research contributes to the understanding of the organizational impact of LLM integration, offers insights into diverse risk perspectives, and underscores the importance of considering sociotechnical factors in the integration process. By shedding light on the challenges and opportunities associated with LLM integration in the financial sector, this study aims to inform future research and practice in this evolving domain.

The aim of algorithmic recourse (AR) is generally understood to be the provision of "actionable" recommendations to individuals affected by algorithmic decision-making systems in an attempt to present them with the capacity to take actions that would guarantee more desirable outcomes in the future. Over the past few years, the literature has predominantly focused on the development of solutions to generate "actionable" counterfactual explanations that further satisfy various desiderata, such as diversity or robustness. We believe that algorithmic recourse, by its nature, should be seen as a practical challenge: real-world decision-making systems are complex dynamic entities involving various actors – end users, domain experts, system owners, etc. – engaging in social and technical processes. Thus, research on algorithmic recourse should account for the characteristics of systems where such mechanisms could be implemented. This necessitates a rich understanding of the problem space of AR but, as we observe, it remains largely uncharted in the existing literature.

We focus on algorithmic recourse in real-world contexts, applying Design Science Research methods to bridge the gap between its technical affordances and the social constraints of real-world decision-making systems where it could be applied. First, we conduct a systematized literature review of 127 publications to learn about the authors' perception of the problem. Next, we consider a case study of a risk profiling model developed to support the authorities of a major Dutch city in the detection of welfare fraud. We employ a desk research approach to learn about the system, reinforce our understanding of the requirements for algorithms in public administration settings through interviews with experts, and make use of accident analysis methodologies to theorize about the value of AR interventions in this setting. We draw on these insights to propose a conceptual framework for the evaluation of AR in real-world contexts and provide its proof-of-concept instantiation as a simulation tool that facilitates the study of such mechanisms within decision-making processes. Finally, we design and prove an algorithm to generate actionable recommendations in expert systems. These are commonly used in public administration systems but overlooked in existing research.

On the example of our endeavor, we learn about the ways to strengthen the connections between the problem space and the solution space of algorithmic recourse. We argue that AR can be discussed on three levels of complexity: (1) as actionable recommendations, (2) as the process of improving outcomes, or (3) as the task of developing mechanisms to support end-users in this process. We advocate for computer science authors to focus on the final, broadest meaning of the challenge to improve the applicability of their solutions in real-world contexts. We also encourage researchers from other fields to contribute their perspectives and for practitioners to support further research by building upon our approach to reason about the place for AR solutions in their domains of expertise.

Integrating artificial intelligence (AI) into Japan's life insurance sector marks a significant move towards data-centric precision, reflecting the nation's shift towards Society 5.0. In this domain, AI is revolutionizing decision-making processes and enhancing operational efficiency, with applications ranging from fraud detection in credit card systems to predictive underwriting. While AI offers notable benefits, it also introduces risks, including privacy breaches, algorithmic biases, and inadequate human supervision. To tackle these risks, the Japanese government has initiated guidelines for societal protection, but gaps remain in the insurance industry's implementation, especially in translating social norms into the industry's context to protect stakeholders' values.

The industry needs a guide for safely designing, developing, and deploying AI systems, considering stakeholders' perspectives. This guide fills two knowledge gaps: a framework for translating high-level values into Japanese life insurance industry requirements and an initial process for converting these high-level values into organizational guidelines.

An empirical study on predictive underwriting informed the research, identifying 13 values and four informal social institutions for the AI design process. It involved eight experts who defined 54 norms, which were later refined and categorized into process and assessment norms focusing on data and AI.

The result is ten design guidelines for AI system developers, which are validated by experts, addressing the full AI lifecycle. These guidelines contribute scientifically by introducing an initial process combining design for values with system safety concepts, reporting standardization, and AI governance frameworks.
Future research should replicate this process in various contexts, reevaluate the value framework with broader stakeholder inputs, investigate the dynamics between Japanese society and AI in more detail, and delve deeper into system theoretic hazards analysis. This approach promises to strengthen the value framework and process applicability in different organizational settings.

In our thesis, we conducted an in-depth exploration of the integration of Artificial Intelligence (AI) in public sector decision-making, focusing particularly on AI case routing for welfare benefits allocation. Initially, our research was guided by a techno-optimistic viewpoint, presuming that municipalities would inherently benefit from AI adoption. However, as our study progressed, we encountered the complexities and challenges inherent in the practical implementation of AI in public services.

We employed the Design Science Research methodology, utilizing a diverse array of methods including literature reviews, case studies, expert interviews, and system safety analysis. Our objective was to develop a method-type artifact to enhance decision-making in civil services, with a special emphasis on the application of AI in welfare benefits allocation. Our findings indicated that AI implementation in the public sector is highly context-dependent, requiring a tailored approach that addresses various technical, organizational, and cultural barriers.

Throughout our research, we identified critical factors such as the need for transparency, fairness, and accountability in AI systems. We recognized the importance of adopting a balanced perspective that considers both AI and non-AI solutions. Our study highlighted the potential benefits of AI in public services, such as increased efficiency and improved quality of decision-making. However, we also noted significant challenges in ensuring system safety and maintaining ethical standards.

Through expert interviews and a detailed system safety analysis, we emphasized the necessity of establishing clear rules for the responsible use of AI case routing and addressing the associated risks. Our research concluded that AI might not always be the most suitable approach, particularly if safety concerns and ethical considerations outweigh the potential benefits.

In summary, our thesis underscores the need for a critical and balanced approach to the integration of AI in public services. We advocate for a holistic strategy that involves collaboration among government, academia, industry, and civil society. This approach is vital to harness the potential benefits of AI while mitigating risks and ensuring the well-being of society.

To combat the effects of climate change, there is a worldwide shift to reduced emissions and increased use of renewable energy sources. Solar energy is a vital part of this transition and necessary to be able to achieve the targets. The rapid pace of adaptation raises questions about the negative equity and justice associated with this development. Along with the rapid growth, regional differences in solar panel adoption have been found across the world. Besides, government incentives and policies have played an important role in both adoption growth and justice implications.

This study explores the pace and equitability of the solar energy transition in Amsterdam, the Netherlands. The study evaluates the potential for solar energy, the disparity of adoption, what socio-economic factors are correlated to adoption patterns, and how adoption might evolve in the future and under different policy measures. To do so, a structured, integrated and data-driven approach is designed to meet the research objectives, and possibly serve as a policy tool for future studies.

The study shows that there is an "adoption gap" in Amsterdam, meaning that solar panels and the benefits of incentives end up with a specific group of citizens. Therefore, targeted policy measures are necessary to ensure an equitable transition.

There is a need for a more comprehensive sociotechnical systems view on ML. Such a view looks at the development and use of an ML system in practice as being a sociotechnical ML system: "a system consisting of technical artefacts, human agents and institutions, in which a machine-based subsystem influences its real or virtual environment by automating, supporting or augmenting decision-making". This research takes on this view to design a sociotechnical guide for ML practice, centring the specication of sociotechnical ML systems. Taking on the guidelines contributes to a safe and effective development and use of ML systems.

Explainable AI is the field concerned with trying to make AI understandable to humans. While efforts have resulted in significant improvement in research and practical methods of Explainable AI, there is an urgent need for additional research and empirical studies. The academic research gaps identified in this thesis show that Explainable AI is still in its infancy and is mostly approached with a technocentric perspective while not being focused on the audience the explainability is actually intended for. Next to this, there is no structured approach to defining and establishing explainability in dynamic complex systems that involve people, institutional, and organizational elements. Lastly, there are limited empirical studies that investigate the needs, usage, and risk of explainability in complex systems.

The research tries to define and address explainability in the socio-technical context within the Machine Learning decision support systems of Transaction Monitoring. It does so by performing an extensive literature review and by conducting semi-structured that try to collect empirical knowledge within local practice. The goal of this research is to expand the definitions and view on explainability by incorporating the social, organizational, and institutional elements that influence explainability. Next, the goal is to develop a method that can help practitioners approach explainability in a structured manner taking the audience into account while applying this socio-technical perspective.

The user-centered method for operationalizing explainability takes on a socio-technical perspective and can provide requirements for design choices, in addition to this the method shows how these requirements can be satisfied and controlled by instantiating control structures. The method has been demonstrated and evaluated within the bank by providing a workshop using a Toy Case during a focus group. Practitioners experienced the method as useful and actionable, also the method provides broader perspectives and insights on explainability and invites the discussion of dilemmas and questions. The practitioners added that the method could be further refined by focusing on additional guidance on the control structure because the method assumes prerequisite knowledge of systems theory and system safety theory.

Governmental AI-supported decision-making is paramount when impacting citizens. Citizens are subject to their government’s decision-making, which is crucial when they transact, as examining the transaction’s rightfulness is executed by the same government. Thus, control and monitoring are evident, which are increasingly applied through AI-supported tools (Hoekstra et al., 2021). It is deemed socially unacceptable when government falsely accuses their citizens of unrightful transactions. Vice versa, it is also deemed unacceptable when criminals taking advantage of public money are exonerated. The citizens whom are trapped in these systems, like the benefits system, are often vulnerable and have low incomes. The Dutch government wrongly accused 20.000 parents of fraud, resulting in major con-
sequences. Therefore, research is crucial for both a better functioning government and protecting the safety of vulnerable citizens. Scientific relevance is emphasised by the narrow understanding of how harm of next similar case is prevented, as such systems are barely researched holistically. Limited
understanding of the relations within and between socio-technological contexts and the safekeeping of citizens lead to the main question:

“How can citizen’s safety be safeguarded in governmental AI-supported decision-making?”

Design science is the main methodology, consisting of three cycles: the rigour cycle, relevance cycle, and design cycle (Hevner, 2014). The method allows for innovative thinking and combining empirical (relevance) and scientific (rigour) knowledge toward exploring a new solution, in this research, safeguarding citizen’s safety (design). Additionally, it allows for a system’s approach required to understand the relationships of different decision-making components and for combining empirical and scientific insights (vom Brocke et al., 2020). Combining this with a holistic perspective, the research incorporates strategical, tactical, and operational challenges and specifically including the political dimension; more than factors and actors. Insight is created between the different context, which are all crucial in establishing a well balanced system.
In seeking an answer to the main question, three parts are crucial: the system of AI-supported decision-making, the concept of citizen’s safety and how both can be integrated. The system and its boundaries are discovered through scientific and empirical exploration, serving as a framework. The definition of citizen’s safety and its implications are discovered by relating to the system and its characteristics. With these two parts, the influence of the system on citizen’s safety is explored and leads to the third crucial part, validated by a serious game. All parts use different sub-methodologies to obtain
the required knowledge for the design science methodology.

The use of Automated Decision Making (ADM) systems in the public sector will become increasingly prevalent in the future, making citizens increasingly likely to be confronted with decisions that have been made fully automatically, without human intervention. Ever more digitization increases the opaqueness of the social benefits system in the Netherlands and leads to citizens facing most of the adverse, sometimes unforeseen effects of automation. Why and how unsafeness and unexpected effects emerge is not well known. In this research, these effects, defined to be ripple effects, are explored using a system safety lens and proposing an agent-based model design to explore why and how the effects emerge.

The leading cause of ripple effects was found to be the lack of a hierarchical safety structure, seen in laws being organized per organization, and the lack of an overarching controller to ensure that overlap does not occur.
This is partly due to a lack of an accurate model representing a Dutch citizen. Several policy recommendations are presented, including a rudimentary set-up on how to apply a safety control structure within the social benefits system.

Scandals in which governmental ADM tools played a role, have recently brought
about political and societal debate about the potential harms to citizens that such automated systems potentially bring. This thesis focuses on ADM systems which contain an AI component. Based on the knowledge gaps perceived,
the main research question for this study is: In public sector AI systems, what are emerging vulnerabilities for citizens and how do these translate into
governance requirements for decision-makers? The approach to answer this question is an adjusted form of Theory Building from Case Study. A layered ’onion model’ presents four relevant contexts to consider for AI System vulnerabilities specifically in the public sector: AI model, Model deployment, Political-administrative, and Societal. The case study results demonstrate that dealing with vulnerabilities in one of the four model contexts often complicates dealing with vulnerabilities in the other contexts. Hence, the vulnerabilities model points to so-called governance requirements dilemmas.