This study introduces KarGus, a novel system for multi-document question answering (MD-QA) designed for diverse domains. KarGus integrates advanced Natural Language Processing techniques with Knowledge Graph (KG) construction and Graph Neural Networks (GNNs) to enhance retrieval performance across various specialized fields.
We explore the efficacy of combining semantic similarity, TF-IDF, and Named Entity Recognition features in KG construction and information retrieval. Experimental evaluation on a corpus of 30 documents (1810 pages, 10,853 text chunks) from corporate intelligence demonstrates that KarGus outperforms traditional embedding-based methods, achieving a Recall@5 of 0.850 compared to the baseline's 0.823 (p < 0.05). The optimal configuration emphasized semantic similarity (weight 0.75), keyword relevance (0.2), and entity information (0.05).
Analysis of the KG structure revealed moderately well-defined community structures and efficient information traversal properties. While GNN models showed promising training results, they underperformed in the retrieval task, highlighting challenges in GNN application to MD-QA.
This research contributes to the field of information retrieval by demonstrating the efficacy of integrating NLP techniques with graph-based approaches in MD-QA. The adaptable nature of KarGus suggests potential applications across various specialized domains. Future work will focus on validating cross-domain performance and refining GNN implementations for diverse retrieval tasks.

Haptic bilateral teleoperation technology aims to transmit the sensation of touch over the internet, enabling remote work for all. However, it faces significant challenges, including stringent latency requirements (sub 1 ms) for the Tactile Internet and the need for advanced models in Model Mediated Teleoperation. This work explored the feasibility of haptic bilateral teleoperation over long distances through a new framework that leverages live video and predictive haptic feedback. This framework was implemented on a real experimental setup where users draw with a haptic device controlling a remote robotic arm on a moving whiteboard. A user study was conducted on this setup which confirmed that the method is sound and revealed that force feedback latency requirements are significantly more stringent than those of visual feedback. In this application users did not perceive the application as more difficult at 50 ms network latency (110 ms visual latency) with user experience only degrading at 75 ms added network latency. These findings prove that through the Demonstrate Imitate Adjust framework, long-distance haptic bilateral teleoperation is achievable.

This project aimed to investigate reinforcement learning (RL) algorithms to improve water management policy development in the Nile Basin, with a focus on the Multi-Objective Natural Evolution Strategies (MONES) and Evolutionary Multi-Objective Direct Policy Search (EMODPS) algorithms. This project intended to refactor a Nile Basin simulation to be compatible with the MONES algorithm, which continues the exploration of different machine learning algorithms in water resource management. Additionally, the RL algorithms were aimed at training using two climate data sets: human-favourable and climate-varying conditions, and then evaluating on the satisfaction and regret metrics. The successful integration of the MONES framework shows the feasibility of utilizing advanced RL algorithms for water management problems. Initial results indicate that the MONES algorithm underperforms compared to the EMODPS algorithm according to hypervolume and diversity of solutions, however, further research is needed to test whether this claim holds. The EMODPS algorithm faced challenges in finding optimal solutions when dealing with variable climate conditions scenarios, accentuating the need for robust solutions, which consider a variety of possible climate outcomes. The observed sensitivity to variable climate conditions underlines the crucial importance of accurate and recent data, as well as the need to consider the climate change effects on water management. The study concluded with suggestions that future simulations of water management strategies may be improved if a broader set of external factors and a more realistic representation of objectives are included in the simulation model. These improvements stand to positively impact the accuracy, applicability and reliability of future simulations.

We investigate whether a teacher-student curriculum learning approach using a teacher network with a simpler structure than the student network can achieve better results at meta-learning. The goal of meta-learning is to learn from a set of tasks, and then perform well on a new, structurally similar but unseen task with minimal retraining. Instead of sampling uniformly from all data to create the training batches, the curriculum-learning approach aims to create a sequence of mini-batches that enhances the training process, also known as a curriculum. During teacher-student curriculum learning a "teacher" network is trained in the standard manner, and then its outputs are used to order the training samples by difficulty and categorise them into mini-batches. This curriculum is then used to train the "student" network. Previous teacher-student models either had pre-trained more complex teachers, or teachers with the same structure as the student network. We investigate whether a teacher network with a simpler structure can also increase accuracy, while preserving computational resources. We find that using such a curriculum worsens performance compared to not using any curriculum at all.

Meta-learning is an important emerging paradigm in machine learning, aimed at improving data-efficiency and generalization performance across learning tasks. Challenges caused by noisy data has been extensively researched in traditional learning settings. However, its impact in the context of meta-learning, especially concerning label noise in meta-training, remains under-explored. Curriculum Learning (CL), is an approach where training data is ordered from easy to complex, and models learn from easier to harder samples. A type of CL , Self-Paced learning (SPL) offers adaptive data curriculum, where ordering is based on per sample model performance during training. Self-Paced Learning (SPL) has proven effective in enhancing model robustness and convergence under noisy data scenarios. However, its application in meta-learning under these conditions remains limited. In this paper, we use a Neural Process model on 1D sinusoidal function regression tasks, with different ratio of clean / noisy training data scenarios to empirically observe the same benefits SPL can potentially offer for noisy meta-learning. In line with findings in traditional learning settings, SPL improved overall training convergence, also lead to increase in generalization thus noise robustness. Furthermore SPL lead the model to be more robust to increasing scale of noise for tasks within the training data distribution.

Meta-Learning is an emerging field where the main challenge is to develop models capable of distilling previous experiences to efficiently learn new tasks. Curriculum Learning, a group of optimization strategies, structures data in a meaningful order which aids learning. However, the extent to which curriculum strategies can optimize the performance of meta-learners remains unclear. Here we study the separate and joint effects of a model-based (ScreenerNet) and a statistics-based (Active Bias) curriculum strategy on the training of a meta-learning model (Neural Processes) which solves 1-D function regression tasks. The findings show that ScreenerNet increases in-task accuracy and accelerates convergence, but decreases the generalization performance. Active Bias achieves mixed generalization results and significantly decreases training efficiency when trained on noisy data-sets. Combining them partially mitigates ScreenerNet's overfitting and stabilizes Active Bias' susceptibility to noise, but further research is necessary in order to achieve consistent improvements to the baseline.

Amidst the rampant spread of misinformation, fact-checking of diverse claims made on the internet has become a pertinent task to mitigate this problem. Manual fact-checking cannot scale up with this demand and is very cumbersome, therefore instead automated fact-checking can be used. However, existing work has primarily focused on the fact-verification part rather than evidence retrieval for large data collections, leading to scalability issues for practical applications. In this study, we address this gap by exploring various methods for indexing a succinct set of supporting facts extracted from large data collections and enhancing the retrieval phase of the fact-checking pipeline. Our evaluation, consisting of measuring the performance and efficiency, is performed on the state-of-the-art claim datasets HoVer and WiCE, where we utilised the English Wikipedia as a large evidence data collection. Overall our results underscore the effectiveness of integrating supporting facts and advanced retrieval techniques for fact-checking pipelines in practical applications. We achieve, through a combination of indexing supporting facts together with Dense retrieval and Index compression, a massive improvement over the original fact-checking pipeline. This is up to a 10.0x speedup using a CPU-based approach and up to a 20.0x speedup using a GPU-based approach, while only incurring a modest loss of less than 6 points in accuracy.

The aim of algorithmic recourse (AR) is generally understood to be the provision of "actionable" recommendations to individuals affected by algorithmic decision-making systems in an attempt to present them with the capacity to take actions that would guarantee more desirable outcomes in the future. Over the past few years, the literature has predominantly focused on the development of solutions to generate "actionable" counterfactual explanations that further satisfy various desiderata, such as diversity or robustness. We believe that algorithmic recourse, by its nature, should be seen as a practical challenge: real-world decision-making systems are complex dynamic entities involving various actors – end users, domain experts, system owners, etc. – engaging in social and technical processes. Thus, research on algorithmic recourse should account for the characteristics of systems where such mechanisms could be implemented. This necessitates a rich understanding of the problem space of AR but, as we observe, it remains largely uncharted in the existing literature.

We focus on algorithmic recourse in real-world contexts, applying Design Science Research methods to bridge the gap between its technical affordances and the social constraints of real-world decision-making systems where it could be applied. First, we conduct a systematized literature review of 127 publications to learn about the authors' perception of the problem. Next, we consider a case study of a risk profiling model developed to support the authorities of a major Dutch city in the detection of welfare fraud. We employ a desk research approach to learn about the system, reinforce our understanding of the requirements for algorithms in public administration settings through interviews with experts, and make use of accident analysis methodologies to theorize about the value of AR interventions in this setting. We draw on these insights to propose a conceptual framework for the evaluation of AR in real-world contexts and provide its proof-of-concept instantiation as a simulation tool that facilitates the study of such mechanisms within decision-making processes. Finally, we design and prove an algorithm to generate actionable recommendations in expert systems. These are commonly used in public administration systems but overlooked in existing research.

On the example of our endeavor, we learn about the ways to strengthen the connections between the problem space and the solution space of algorithmic recourse. We argue that AR can be discussed on three levels of complexity: (1) as actionable recommendations, (2) as the process of improving outcomes, or (3) as the task of developing mechanisms to support end-users in this process. We advocate for computer science authors to focus on the final, broadest meaning of the challenge to improve the applicability of their solutions in real-world contexts. We also encourage researchers from other fields to contribute their perspectives and for practitioners to support further research by building upon our approach to reason about the place for AR solutions in their domains of expertise.

This thesis investigates the prevalence of Pylint warnings in open-source Python projects and evaluates the effectiveness of an AI-driven tool for automatically fixing these warnings. The study also explores how developers perceive automated code suggestions and seeks to streamline consent mechanisms for research-related code changes. The primary research questions addressed are: (1) What is the prevalence of Pylint warnings across open-source Python projects? (2) How effective is the AI tool developed to fix Pylint warnings? (3) How do developers perceive the automated suggestions and (4) how can the process of proposing research-related code changes with developer consent be streamlined?

To address these questions, the research draws on literature related to static code analysis, fault detection, and the increasing use of artificial intelligence (AI) in automated code repair. Previous studies highlight the challenges developers face in maintaining consistent code quality and the role of AI in automating such tasks.

The research follows a mixed-method approach. Quantitatively, a dataset of 205 open-source Python projects was analyzed to identify and address common Pylint warnings. An AI-driven tool was employed to attempt fixing these warnings, achieving a success rate of 88\%. In 60 projects, pull requests were submitted to open source maintainers to assess the effectiveness and reception of the tool. Qualitative feedback from maintainers was collected and analyzed, leading to a shift in the contribution strategy from pull requests to submitting issues first, as this was perceived as less intrusive and more manageable by developers.

The analysis revealed a high prevalence of Pylint warnings, particularly \textit{missing-function-docstring} and \textit{line-too-long}, across projects of all sizes. The AI-driven tool effectively fixed 88\% of the warnings, resulting in 70\% of the projects being fully warning-free. However, developer responses to automated pull requests were mixed, prompting the adoption of a more collaborative issue-first approach. These results suggest that AI tools can significantly improve code quality, but challenges remain to foster developer engagement and integrating such tools into established open source workflows.

The study has certain limitations, mainly the focus on Python projects, which may limit the generalizability of the findings to other languages or more complex projects. Furthermore, developer consent and participation were limited, which affected the full implementation of automated changes. Future research should focus on improving the integration of AI tools into developer workflows and expanding the scope of automated code fixes to more diverse and complex projects.

Conventional fixed-line bus services are ineffective and costly in areas of low demand. Bus operators are motivated to scale down their service to reduce costs. Consequently, many rural areas in the Netherlands suffer from diminishing accessibility to jobs, hospitals, and education using public transport. Demand responsive transport (DRT) systems can offer a solution, but have been struggling in adoption by travellers because the need to make a reservation is perceived as an obstacle. Furthermore, the performance of DRT systems is complex and requires evaluation before implementation. In this context, this thesis considers the replacement of three low-demand bus lines by a DRT system in Voorne-Putten Rozenburg, Netherlands. Two scenarios are formulated: the replacement of a single bus line and the replacement of all three low-demand bus lines. The objective of this thesis is to evaluate the performance of a DRT system in serving the travel demand currently served by the bus lines. To obtain the travel demand, individual trips from the existing bus line were generated from aggregated data on boarding and alighting in 2023. The travel demand in a DRT system was evaluated using an agent-based simulation approach. To address the perceived obstacle of making reservations, the DRT system also considered on-demand requests and a proposed stop-based request where the user can request the DRT service physically at the bus stop with the consequence that the DRT system initially only knows the users' origin and not the destination. The main results suggest that a DRT system is an improvement over fixed-line buses in terms of waiting time and travel time, and CO₂ emissions when considering current travel demand in both scenarios. However, the costs are comparable. In the case that travel demand increases more than twofold, fixed-line buses are to be preferred. Considering the different request methods, only a small difference in user times was observed. This thesis concludes that replacing the bus lines with a DRT system can lead to an increase in accessibility. The proposed stop-based request method was shown to be a feasible alternative to the request methods that require the use of an app or phone call and could potentially lower the obstacle to using DRT.

A 2022 Harvard Business Review report critically examines the readiness of AI for real-world decision-making. The report cited several incidents, like an experimental healthcare chatbot suggesting a mock patient commit suicide in response to their distress or when a self-driving car experiment was called off after it resulted in the death of a pedestrian.

These incidents, leading to media frenzies and public outcries, underscore a pressing concern: "How do these AI systems reach their conclusions?" It has created an urgent demand for transparency and clarity in AI decision-making processes. This urge to understand has translated into a significant uptick in the volume of work in Explainable AI (XAI). This makes it crucial to have consistent evaluation standards for streamlined growth in the field.

However, XAI, being a multidisciplinary field, faces the challenge of a lack of consensus on what constitutes a "good" explanation. Stakeholders with diverse backgrounds and needs can have diverging expectations from XAI. Some might prioritize simple and concise explanations, while others prioritize detailed information about AI predictions, all depending on their end goal.

This thesis addresses the standardization of an evaluation framework for XAI methods, that accounts for stakeholders' needs in different usage contexts. It presents a prototype that can be customized and extended to suit various XAI methods and tasks. Findings affirm the framework’s ability to yield insightful comparisons between different XAI methods. It also highlights issues with human perception of specific XAI features in those methods. The efforts in this work contribute to XAI techniques being integrated into real-world applications, ensuring more reliable and consistent performance assessment.

When making decisions, people are automatically guided by their moral compass. However, AI agents need to be conditioned in order to be steered towards moral behaviour. An environment that can be used to train and test agents is the Jiminy Cricket environment. The Jiminy Cricket environment consists of a set of text-based narrative games, where every action possible is annotated with the morality of that action. However, to create a more morally nuanced agent, we have annotated all of the actions according to the following moral values: Care/harm, fairness/cheating, loyalty/betrayal, authority/subversion, and purity/degradation. To morally condition the agent, we calculate the predicted progress of a potential action and combine it with an oracle to retrieve the moral annotation of the potential action. Using both of these components, the score per generated action is calculated and based on the score the eventual action is chosen. The score can be calculated differently based on the weights assigned to the overall progress and morality, as well as based on the sub-weights assigned to each moral value. Using this environment we pose the question, if we focus on only one moral value, what is the most optimal configuration that can be achieved in order to maximize both progress and morality? From the results we can observe that the lowest relative immorality can be achieved by imposing no moral constraints on the agent. Posing constraints on the agent will lead to a relatively bigger decrease of the completion percentage than to the immorality decrease. One-hot encoding the moral values will reveal which immoral actions are needed to progress in the game, and which immoral actions should to be prevented to lower immorality.

Transformer models have proven to be effective tools when used for determining the readability of texts. Models based on pre-trained architectures such as BERT, RoBERTa, and BART, as well as ReadNet, a transformer model which is dedicated to readability assessment, have shown some very promising results. However, there is a lack of research focused on comprehensively analyzing these models' performance at a more granular level. Moreover, GPT-2, a member of the very popular GPT transformer family, has never been adapted to and tested in readability assessment. The work presented in this paper fills these knowledge gaps by analyzing the behavior of the five aforementioned models and reflecting on their performance on separate classes of text difficulty. Seeing how they perform on texts of various complexity levels is vital to understanding their behavior and limitations, which will in turn further the knowledge of the situations in which each readability tool achieves the most optimal results.

Readability plays a vital role in the transfer of knowledge or information. This is especially true for children who are more than anyone gaining new knowledge every day, yet their reading abilities are still in active development. Although readability assessment of children's literature is an active topic of research in many different languages, research into the assessment capabilities of well-known Dutch traditional readability formulas on texts aimed at Dutch children remains in its infancy. In this paper, we explore the performance of four well-known Dutch traditional readability formulas when applied to different types of texts targeting Dutch children. More specifically, we examine the applicability of the Flesch-Douma, Leesindex A, CLIB and CILT formulas on excerpts of school materials, books, and media for children aged 6-14 years old. Outcomes from this empirical exploration reveal that the readability formulas do not perform similarly across reading materials. As a result, choosing a readability formula based on both the type of reading material and a child's age can improve the readability estimation, which in turn can help connect children to understandable resources.

As AI is progressively incorporated into several spheres of society. This rapid growth has also brought a lot of challenges such as discriminating or skewed results and a lack of accountability. To address these challenges, there is a growing interest in Human-AI teams where AI-assisted decision-making includes humans in the loop. This approach has been widely explored to address the issue regarding transparency, reliability, and trustworthiness.

However, the essential premise of Human-AI teams in critical applications is that humans must comprehend the reasoning behind an AI system’s decisions. Because of the opaqueness of the AI systems, it has been proved very challenging for humans. The field of explainable AI, is promoted as the link that permits human comprehension of AI systems. A wide range of machine learning explainability techniques have been created. However, standalone explanation techniques have been found to have limited success in ensuring a coherent understanding of AI systems. The primary cause is the insufficient interactivity, absence of actionable human feedback, limitation to specific information, and lack of personalization. Other approaches, such as XAI Dashboards that provide users with multiple standalone explanations have been found to cause information overload. Recent studies suggest that an overload of information can lead to suboptimal AI reliance and understanding. Additional studies also show that XAI dashboards because of their limited interactive nature, the information interchange is mostly unidirectional. Further studies pointed out that XAI dashboard may fail due to unidirectional information exchange, which hinders active user exploration. This may result in an incoherent understanding of the AI system.

Delivering explanations through conversations (conversational XAI) can be a potential solution to address the research gap. Recent studies have shown that the interactive exchange of information may promote a better understanding and uncertainty awareness of AI systems. Additionally, the ability to selectively answer user-specific queries may help users create a better mental model of the AI system and hence improve appropriate trust and reliance. Finally, the personalized conversation may also help in higher perceived trust and address user information need about AI systems.

In this research work, we performed an empirical study (𝑁 = 245) to evaluate the impact of conversational XAI on the understanding, trust and reliance of the AI system. The interface for conversational XAI is built with a rule-based approach. To understand how the impact varies compared to widely adopted alternatives — XAI Dashboard, we compared the understanding, trust, and reliance of AI systems with a between-subjects setup. Additional effects of user-specific personalization of conversational XAI were also studied.

Overall, we found that participants with explainer interfaces showed improved trust and reliance compared to the control condition (i.e., no XAI). However, such increased reliance are not necessarily appropriate reliance. The experimental results observed a clear over-reliance on the AI system for participants with XAI. Additionally, no significant difference was observed in user understanding, trust and reliance between XAI dashboard and conversational XAI interface. Our results and findings may provide useful guidelines to future work about conversational XAI interface and XAI-assisted decision-making.

Synthetic lethality (SL) is a relationship between two genes, exploited for targeted anti-cancer therapy, whereby functional loss of both genes induces cell death, but the functional loss of either gene alone is non-lethal. Computational prediction of SL gene pairs is sought after because it is expensive to do lab screening for SL. Existing SL labeled pairs from wet- lab experiments often focus on specific genes or pathways, resulting in notable selection bias. Current SL prediction methods ignore this bias when training on available SL labels, and fail to generalize if test sets follow a different selection bias. One way to mitigate bias is to incorporate unlabeled pairs during model learning. However, conventional semi-supervised methods such as self-training can reinforce bias by adding confidently pseudolabeled pairs, which tend to be most similar to previously included samples. We present DBST, a self-training strategy that addresses the issue by promoting diversity in the selection of pseudolabeled samples. This is achieved using metric learning to find a class-contrastive representation of the feature space, based on which DBST selects diverse (or dissimilar) pseudolabeled pairs. In results for five cancer types, semi-supervised models, including DBST, delivered improved SL prediction performance over the supervised model. Additionally, DBST successfully incorporated unlabeled samples that were more dissimilar among them compared to standard self-training. In experiments with differing biases between train and test sets, DBST showed a slight improvement in performance compared to the supervised model.

Search engines are used to gather and collect information. This interaction sometimes influences the user and changes their attitude towards a topic after such interaction. Prior work has shown that it is a complex endeavour to understand attitude change, as there are many things that can influence a user during a search session. One of the aspects that have not been researched is the influence of informedness. Therefore we examine the role of informedness in attitude change. To do that, we first defined what a well-informed user in the context of web search is. Then a user study is conducted to understand informedness’s role in attitude change. From the platform Prolific, we recruited participants (N=320) for the main user study. The experiment is a 3 x 3 (Informedness level x SERP viewpoint bias) factorial between-subjects study and it requires interaction with the provided search results. The participants were placed in one of three categories, ”Well Informed”, ”Mildly Informed”, and ”Uninformed”, based on their results on a knowledge questionnaire. We created 3 different conditions for the search engine results page (SERP). The SERP had a viewpoint-biased ranking that was either supportive, opposing or balanced towards the topic. There were two topics, abortion and obesity, to which the participants were assigned. Our findings showed that 37% of the participants with a valid submission had changed attitudes. The findings from the user study suggest that the level of informedness does not play a role in facilitating attitude change among users during a web search. This has implications for the design of information retrieval systems and web search experiences in general.

Mind-wandering happens when one's current train of thought, related to a specific task, is interrupted, due to internal disconnected thoughts. This phenomenon is highly subjective, and its detection is really important due to the internal understanding of the human mind that can be obtained. Several methods have been used in order to detect mind-wandering, such as thought probes, self-reports, electrophysiological measures or even eye-based tracking methods, however the detection of mind-wandering solely from sound has not been researched about. Therefore, this study is going to investigate if automatic detection of mind-wandering through sound is feasible. In this work, this question is tackled through a machine learning approach, where a linear SVM model is trained through acoustic features. Two methods of oversampling are considered, due to the high data imbalance between the classes, and these two are evaluated and compared. The approach is evaluated through different metrics, such as recall, precision, F1-score and accuracy, but also a comparison with other techniques is done. Results of this work show that sound as its own is not a reliable way of automatically detecting mind-wandering. These results however, might be implementation specific, as the ground truth values of mind-wandering were created through the means of perceived mind-wandering, and the techniques of random oversampling and SMOTE were also used. These could be causes of unreliability of the research. Future work should should take this into consideration and also apply this approach to a different data set, to assess its feasibility.

The aim of this research is to discuss if it is possible or feasible enough to detect Mind-wandering of individuals using their hand and body movements from video recordings. The basis for this research is “Mementos”[9] data set, containing over 2000 recordings of people watching music videos. During experiment videos from data set were used to create software, that would automatically determine Mind-wandering. Results of the study have shown that body and hand movements are useful for detection of the phenomenon, however are not self-sufficient indicators for reliable detection. This study has found a reproducible methodology for automatic detection of Mind-wandering from pre-recorded videos, which is contributing towards exploration of this complex phenomenon in the field of Computer Science.