This thesis addresses the semantic gap in visual understanding, improving visual models with semantic reasoning capabilities so they can handle tasks like image captioning, question-answering, and scene understanding. The main focus is on integrating visual and textual data, leveraging human cognitive insights, and developing a robust multi-modal foundation model. The research starts with the exploration of multi-modal data integration to enhance semantic and contextual reasoning in fine-grained scene recognition. The proposed multi-modal models, which combine visual and textual inputs, outperform traditional models that rely solely on visuals. This is particularly true in complex urban environments where visual ambiguities often occur. This method emphasizes the significance of semantic enrichment through multi-modal integration, which helps resolve visual ambiguities and improve scene understanding. @en

Modern factories showlittle resemblance to the assembly lines froma century ago. Nowadays, a single human might be responsible for setting up, operating, and maintaining a complex chain of machines. This shift has made the work of factory operators more cognitively demanding, requiring constant monitoring and problem-solving. Key to tackling these challenges is the effective collaboration of human operators in exchanging ideas and knowledge, from high-level problem-solving strategies to solutions for emerging issues. Yet, the heightened productivity requirements and small shifts mean fewer opportunities to share this knowledge face-to-face and reporting practices are deprioritized. Thus, the manufacturing industry faces a knowledge management crisis.
This dissertation investigates the integration of conversational AI assistants into manufacturing settings to facilitate knowledge sharing among factory operators. Capitalizing on recent advancements in Natural Language Processing (NLP), particularly in Large Language Models (LLMs), this research investigates the designing and evaluation of conversational AI tools that efficiently capture and share human knowledge on the factory floor while addressing operator needs and concerns. The introduction of conversational AI assistants for knowledge sharing—referred to as cognitive assistants (CA) in this work—in factory environments promises significant benefits but comes with numerous challenges....@en

Conversational agents (CAs) are increasingly being adopted across various domains, with enterprises particularly interested in leveraging these technologies to explore new opportunities. CAs, such as Copilot, are transforming human-computer collaboration in the workplace by serving as effective work assistants. Additionally, their advanced natural language understanding capabilities position them as powerful tools for facilitating crowdsourcing within enterprises.
This thesis investigates the dual role of CAs in enterprises: as work assistants for employees and as facilitators of crowdsourcing. By acting as work assistants, CAs interact with a broad range of employees, making them well-suited to gather valuable insights that contribute to the company’s knowledge base and support collaborative problem-solving. The central research question explores how to effectively engage and motivate employees in this crowdsourcing process through CA personality design, which is relatively unexplored. Drawing on the "Computers Are Social Actors" (CASA) framework by Nass et al. (1994), this thesis examines CA personality design, aiming to enhance human-CA interaction by eliciting social responses from employees and thereby increasing their engagement.
We approached these research questions from three perspectives: (1) designing and specifying CA personalities for effective implementation in enterprises, (2) identifying the most and least promising qualities of CA personality designs in this context, and (3) designing an empirical study within the enterprise to measure which personality archetype design is the most effective.
We reviewed literature on the context of conversational crowdsourcing in enterprise settings, and CA design methods with a focus on CA anthropomorphism. These findings informed the initial design phase, where CA personality archetypes were tested in an enterprise environment. Based on these insights, we developed design guidelines for enterprise CA personality design. After refining the archetypes, we proposed three unique CA personalities aimed at enhancing employee participation in enterprise crowdsourcing through their interactions with CAs. These archetypes were then implemented in the enterprise's internal chat platform for an experiment, where employee interactions with the CAs were collected and analysed both quantitatively and qualitatively.
The results of our experiment reveal how different CA personalities impact user engagement in enterprise crowdsourcing, demonstrating how distinct CA personalities can be effectively translated into LLM prompts and perceived by users. The study also highlights the relationship between CA personality and the triggering of users' social responses, as observed through sentiment analysis. Additionally, our research identifies the challenge of balancing social engagement with maintaining professionalism in enterprise CAs to build user trust.
This thesis concludes with a discussion of the findings and their implications for future research in enterprise CA design.

This thesis aims to design the future roles of human operators in an autonomous bus system in Autonomous Airside Operations at Schiphol Airport. The research was conducted in collaboration with Delft University of Technology and Royal Schiphol Group, focusing specifically on the implementation of autonomous buses for passenger transportation between terminals and aircraft.

The study employs a human-centered design approach since the successful integration of technology relies not only on its technical capabilities but also on human operators, who will work in and interact with an autonomous bus system. By placing humans at the center of the design process, a valuable future human operator role is created, that contributes to the successful implementation of an autonomous bus system in AAO.

To emphathize with the current bus operations, context studies at Schiphol Airport are conducted with 3 key roles in the current bus operation. Additionally, 11 expert interviews were conducted to gather different perspectives on the future human operator role in AAO.

The context research revealed 4 current operator responsibilities besides driving-related tasks. Additionally, based on the expert interviews, 5 future human operator responsibilities in an Autonomous Airside Operations are identified. Combinations between current and future responsibilities resulted in the design of 4 potential future scenarios, each with a different future human operator role.

The study provides 4 human operator roles in potential future scenarios where human operators remain crucial in an autonomous bus system, which is: (1) the role of the safety driver to enable autonomous driving as an addition to manual operations; (2) the role of host/hostess or fleet coordinator to provide onboard assistance; (3) the role of the (de)boarding assistant to ensure a safe boarding and deboarding procedure; and (4) the role of the rescuer to immediately operate in case of an emergency. These scenarios are further developed into animation videos, which serve as speculative design artifacts to envision, reflect, and discuss potential future scenarios.

Current literature that aims to describe the use of (Gen)AI in the domain of design is hindered by the lack of conceptual integration of design theories which in turn blurs the connection between the design process and human-AI interaction and collaboration frameworks. This thesis’ objective is to contribute to the conceptual understanding of LLM-based design process augmentation. This thesis shows how such LLM-based design process augmentation can look like by establishing 1) a time-based framework of a human – Augmentation-System collaborative design process, 2) a perspective on the design process augmentation capabilities of LLMs, and 3) an augmentation system architecture and interface for the practical implementation of these theoretical considerations. Other contributions that have enabled these three main contributions include 1) the unification of existing design theories (problem-solution co-evolution, situated Function-Behavior-Structure (FBS) framework, Concept-Knowledge theory, and Uncertainty Driven Action (UDA) model) to attend to a wider range of person-related characteristics required for the description of a collaborative process of design, 2) the connection between human intellect augmentation and creativity literature, resulting in 3) the application of creativity literature in the domain of GenAI and specifically LLMs, and 4) the application of the layers of behavior as described in the UDA (Uncertainty Driven Action) model for the creation of a computational augmentation system. Finally, a reflection on the feasibility and desirability of different forms and applications of augmentation systems is provided. The results of this thesis have created a fundament for further research into the direction of the initial objective, as the results have increased the conceptual power to describe and explain process, output, and other augmentation-related phenomena connected to the domain of human-LLM (or GenAI) co-design.

Accessibility is a widely employed concept across a variety of disciplines to evaluate the degree to which individuals can reach a desired destination. Conventionally, accessibility is determined by the attractiveness of a destination and the associated travel cost to reach it. However, existing place-based accessibility measures do not differentiate between destinations accessible to individuals from a single demographic group and those accessible to individuals from diverse demographic groups. This hinders our ability to discern the encounter potential of different destinations. We address this gap by introducing the concept of co-accessibility to measure how accessible a given destination is to different individuals and demographic groups.@en

Music annotation and transcription of music sheets are traditionally performed by experts. Although these processes result in high quality data, the scope of each effort is relatively narrow resulting in highly specialised and specific datasets of annotated music compositions, which leads to a fragmentation in the design efforts for automated tools. In music traditions such as classical music, the shortcomings of current digitization work- flows become even clearer: due to the vast corpus and varying stylistic intricacies, experts tend to have specific knowledge and take up projects that concern very specific periods or composers, limiting our reach in regards to conserving classical music information as a whole.

On the other hand, crowdsourcing has been successfully utilized in other domains for annotating different modalities (text, image, video, audio), despite the unreliable pool of expertise on online platforms. Commercially successful projects have utilized the crowd, which provided annotations of adequate quality. These annotations were later used to fuel machine learning methods that rely on bulks of annotated data to perform automatic classification, regression, and detection tasks. However, due to the complexity of music as an artifact, there are still only a few examples where the crowd was integrated into any form outside of subjective annotation tasks (e.g., indicate the mood of the excerpt).

In this thesis, we tackle this research gap of integrating the crowd in the annotation processes of music compositions. We surveyed current practices on Optical Music Recognition to identify parts where the crowd could assist, alongside proposing hybrid annotation workflows for music compositions. We studied the capabilities of online participants with unknown musical expertise, quantified their musical abilities, and related them to their performance in music annotation tasks. With our goal being to identify ways to expand the preservation efforts for classical music through the assistance of the general public, we investigated potential online sources of music information and prospective participants outside the currently available crowdsourcing platforms. To that end, we studied how composers’ popularity manifests on community-driven platforms through the interactions of music enthusiasts online. We also conducted interviews and focus group discussions with experts and semi-experts, to understand their quality requirements on semantically-rich digital music scores, and identified transcription patterns that could inform our task designs. We finally delivered our system architecture, which combines computer vision and algorithmic scheduling, with microtasks designed to be performed by human annotators in parallel.

Our findings show that with the right methods to quantify the musical competence of a person, paired with careful design of the annotation tasks and interfaces, we can successfully integrate the crowd in the music annotation processes, to generate meaningful and useful information regarding classical music compositions and beyond. This thesis enables future research by showcasing the versatility of the crowd and providing task design methods to accommodate their lack of formal training in the field. It also provides experimental methods to reliably identify how different music composition elements affect the crowd’s performance, alongside proposing user interface elements that can mediate the complexity of the artifacts. Practices such as those presented in this thesis can lead to scaling up our digitization efforts, generating accurate and useful annotations through the crowd, even in such domain-specific and knowledge-intensive topics as classical music compositions.@en

Over the last two decades, the machine learning (ML) field has witnessed a dramatic expansion, propelled by burgeoning data volumes and the advancement of computational technologies. Deep learning (DL) in particular has demonstrated remarkable success across a wide range of domains, including healthcare, mobility, life sciences, and energy systems. This success has been further accelerated by the availability and efficiency of open-source ML frameworks like TensorFlow and PyTorch, making ML methodologies more accessible than ever.

However, this rapid growth has brought its own set of challenges. The proliferation of ML models and related artifacts, such as datasets, have brought abundant information during the ML lifecycle. The descriptive and property information of these artifacts is referred as metadata. Yet current practices, such as model cards used in public model zoos and tools to track metadata within scripts, cannot fully captured the metadata of these artifacts, let alone a standardized approach for their management, and access. In addition, the prevailing practice of managing ML/DL scripts via traditional software repositories, while adequate for software engineering, falls short in addressing the unique needs of ML workflows, such as model reuse and comparative analysis. These practices hinder the effective use of structured and comprehensive metadata representation. This disconnect points to a pressing need for improved methodologies and tools in the ML field.

In response to these challenges, this thesis delves into the development and exploitation of structured metadata representations within ML model zoos. In Chapter 2, we first propose a metamodel that represent different types of metadata, thus transforming the metadata from being merely descriptive to being queryable and machine-readable. The structured nature of our metamodel allows for more efficient querying and retrieval of information, which is a substantial improvement over the traditional, text-based descriptions.

Additionally, the thesis explores the use of metadata to optimize various ML processes, particularly in the selection of appropriate models for specific tasks, i.e., model inference and fine-tuning. In Chapter 3, we investigate the optimization of ML inference queries in heterogeneous model zoos using a Mixed-Integer-Programming-based optimizer. This optimizer, which considers multiple objectives such as accuracy and inference speed, provides a robust framework for model selection and execution planning. In Chapter 4, the research extends to model selection for fine-tuning. We investigate on predicting model performance, particularly accuracy, in scenarios where data domains shift, thus negating the need for constant model fine-tuning. By selectively choosing only the most promising candidates, this method substantially lowers the computational burden and associated costs of extensive model fine-tuning.

Overall, this thesis investigates the representation and application of metadata. The insights and methodologies presented not only improve the efficiency and effectiveness of ML workflows but also pave the way for further exploration in the integration of metadata within ML practices, highlighting the continual development and potential for advancements in ML.
@en

Correctly processing accumulated information is beneficial for our survival. Berghman and Hekkert (2017) argue that this is why we humans derive pleasure from having a sense of aesthetics. These aesthetic experiences can be seen as our brain’s reward system for correctly perceiving and interpret- ing the world around us. While our senses have evolved to perceive and organise the physical world, these very mechanisms also come into play when we interact with the digital realm. Aesthetics in visual training datasets are of importance as it allows us to derive a sense of aesthetic pleasure from digital media. Integrating aesthetics into artificial intelligence, especially in text-to-image generators, becomes important to cater to humans psychological reward systems and to engage them at a deeper level.

This thesis is focused on investigating the annotation method used in the development of the LAION- Aesthetics V2 datasets and comparing it to other annotation methods for measuring aesthetics. The purpose is to explore whether there are more suitable alternatives to the current annotation method (where people are asked to annotate images with the instruction ”how much do you like this image on a scale from 1 to 10?”, (Schuhmann, 2022) which is not backed by literature to actually measure aesthetics), and to evaluate the alignment between the LAION Aesthetics Predictor scores and human ratings.

This thesis explores different distinct levels of inquiry: one focuses on the design of instructions for image annotation tasks (alternative task design), while the other centers around measuring aesthetics during the annotation process (alternative metrics). Both lines of inquiry are supported by relevant literature, indicating their potential capacity to capture aesthetics. In addition to comparing alternative annotation methods, this thesis investigates three hypotheses related to the annotation of aesthetics within the project’s context.

Four experiments are conducted using crowdsourcing to compare alternative task design and alternative metrics. The experiments include semantic concept activation, different phrasing of the annotation instruction, and alternative modalities (such as ranking and two-alternative forced choice). Next to these four experiments, a separate fifth experiment is deployed which looks into the evaluation of image content versus overall image liking. Two post hoc analyses are performed, one which compares scores that the LAION Aesthetics predictor assigns to the stimulus set to human image liking ratings, and one examining the influence of region on image liking ratings.

The LAION aesthetics approach performed equal to the alternatives with scientific backing. The ranking treatment even performed worse. For this data, region did not impact image liking ratings. No significant difference was found between participants’ overall image liking and content liking. The LAION Aesthetics predictor scores partially aligned with human liking ratings but showed some disparities, par- ticularly in extreme ratings. Qualitative analysis suggests that more research is necessary to make a judgement on whether ”liking” is a relevant and appropriate approach for capturing aesthetics.

The limitations of the experiments include small sample sizes and the focus on a specific image class (buildings). Recommendations for future research include exploring different image classes, investigating other ranking modalities, and considering n-alternative forced choice experiments. It is also suggested to examine the influence of regions on aesthetic experiences in more detail, explore Gibbs Sampling with People for measuring image aesthetics, and explore different demographic groups and contexts.

This thesis delves into the development of a project that aims to boost the awareness and willingness of TU Delft's research staff to share laptop data, needed to promote the further adoption of the Circular Economy (CE) in the Information and Communication Technology (ICT) domain. The project navigates the intricate landscape of data privacy challenges, particularly focusing on laptop hardware and organizational device management, with the ultimate goal of supporting AI processes such as predictive maintenance that require laptop-specific data, which can be categorized as personal data.
The project employs the Agile methodology and the Double Diamond Model to iteratively design user interfaces that visually articulate informed consent for ICT (i.e. laptop) data sharing for CE processes, such as laptop’s repair, reuse, and return. The main objective is to help cultivate a more sustainable ICT service in TU Delft by enhancing CE data sharing transparency and offering incentives for informed consent with the User Interface (UI) design. The design of the user interface aims to simplify the comprehension of consent for data sharing, bring more transparency, and raise awareness about data privacy. This approach is expected to motivate the research staff to actively participate in data sharing, thereby contributing to sustainability efforts within the organization. The thesis culminates with the presentation of key findings and potential avenues for future exploration and research, providing valuable insights into the interplay of data privacy, user experience, and sustainability within ICT services.

Immersive technologies, such as head-mounted devices for Virtual Reality (VR), have experienced increased availability on the consumer market. This opens up numerous possibilities for scientific research and industrial applications. But, configuring and combining data from various sources to create an effective visualization to explore, collaborate in and interact with, has only been researched in limited aspects. This thesis aims to demonstrate the feasibility of integrating these diverse aspects into a single application while preserving their individual potential.

To achieve this objective, an extensive analysis of the literature was done to establish the state of the art. All aspects necessary to create a single application were investigated to define how much of the literature fully or partly implemented them. Findings of the literature were summarized through definition, progress, limitations and improvements.

Based on the analysis, a proof of concept was designed and implemented, which established a virtual reality framework for real-time analysis and exploration of heterogeneous urban data. The framework involved the development of both a client and server applications with real-time communication. The server application processes and caches input data based on a structured configuration file, while the client application renders and visualizes points and shapes based on instructions from the server. The layered processing approach in this proof of concept facilitated real-time data handling and exploration. Which also makes it possible to collaborate with multiple people working on or manipulating the same data at the same time.

Evaluation of the server application showed its capability of processing over 1.000.000 objects from GeoJSON data. After initial rendering, the client application could update over 4.000 objects with a latency of just 200ms, which is considered to be interactive. Feedback from experts further supported the validation of the proof of concept application as a viable solution to the research question.

This research established the validity of a configurable real-time analysis and exploration solution in virtual reality. With further development and research, this tool holds potential for being used in diverse environments dealing with urban data.

Responding to the trend of increasing use of artificial intelligence (AI), we need to ensure applications of AI are designed, implemented, utilised and evaluated in a careful manner. Explainable AI, or XAI, meaning; - given a certain audience, the details and reasons of both technical processes of the algorithm-support system and the reasoning behind the system to make its functioning clear or
easy to understand - is one of the ways to responsibly design and implement AI systems. This research looks into AI-supported public decision-making processes in the Netherlands and the role and possible contribution of XAI in such a context. To this end, I conducted a mixed-method qualitative study; interviewing sixteen respondents from three key-actor groups within two Dutch national public sector executive bodies, additionally performing three observations and document-analysis. Differentiating between different phases of an AI system’s implementation life-cycle, the study unveils how the respective actors - managers, data scientists and domain experts/(potential) AI users - encounter various challenges in bringing an AI system from idea to production. The empirical findings show that many AI systems, whilst technically developed, are not deployed or adopted by the wider organisation. The study discerns the challenges hindering the AI implementation process from an organisational,
human and technical point of view. Moreover, the study highlights the need to approach XAI from a multi-purpose, multi-actor perspective; both acknowledging that various actors need different kinds of explanations, but also bridging different respective professional worldviews to apprehend one another.
XAI is often seen as a one-size-fits-all solution for various implementation challenges, however the study shows that certain challenges need to be addressed at least beyond traditional ways of XAI from a computer science perspective, and perhaps beyond XAI all together. As such, the insights of this
thesis contribute to generating a more realistic idea about the opportunities and limitations of XAI, within real-world AI implementation processes in the public sector.

As the sharing of machine learning (ML) models has increased in popularity, more so-called model zoos are created. These repositories facilitate the sharing of models and their metadata, and other people to find and re-use an existing model. However, the metadata provided for models is insufficient, with little focus on practical aspects of a model such as performance and limitations. This leaves model zoos to provide little functinality beyond the sharing of existing models, and potential users unable to find an optimal model for their needs. In this thesis we focus on the reporting of model performance, and aim to answer two questions: (1) What are the limitations of model performance reporting found in current model zoos, and (2) How can we provide rich, comparable performance metadata for these models. As a result, we have created a framework which can be used to create a benchmarking system for ML models. Our framework enables the automatic evaluation of models from any source, and through its design encourages the collection of rich and comparable performance metadata for those models. To demonstrate the effectiveness of our framework, we have created a benchmarking system and evaluated 1215 models from existing model zoos, along with a new interface to view this new metadata, and have enabled new use-cases such as the performance comparison of ML models.

Supporting designers working with AI is important, yet the tools they have at their disposal are in many ways not satisfactory. The aim of this project was to design a digital canvas tool which lets designers explore AI and ML, adopt the affordances that AI and ML as a design material give, and which can be used to map out and design integrated products, services and systems (iPSSs) by not just guiding the design process but by becoming an integral part of it.

In highly complex sources of data for pattern recognition, like audio, it is hard to obtain a set of information that is both extensively annotated and includes the wide variety of interfering noises that real life applications would encounter. In order to address these issues, information sharing techniques were devised, known as multi-task learning. These forms of learning algorithms learn multiple tasks at the same time, sharing numerical updates of their parameters. By doing this, a whole new amount of opportunities are opened up to mix and match tasks and datasets and the amount of applications of this are growing. However, while more and more promising results have been achieved using multi-task set-ups, there is an added amount of developmental complexity by having to deal with multiple datasets and tasks. This complexity grows signicantly the more dierences there are in the combinations. Furthermore, research requires experimentation and comparison of dierent set-ups, which is quickly complicated by the combinations, compared to single task set-ups. In order to promote research in this eld, these developmental roadblocks must be cleared up. So far however, no framework seems to aid in the development of multi-dataset, let alone multi-task set-ups. This work addresses the technical diculties in implementing, evaluating and experimenting with multi-task set-ups. By devising the multi-task set-up as a pipeline going from the raw datasets to trained and evaluated models with interchangeable parts, a framework is built that brings the development closer to single dataset, single task learning set-ups. The idea is for developers to only have to focus on implementing single pipeline parts, with the freedom to assemble and re-assemble them without having to worry about the combinatorial problems. This starts by investigating the current literature, where the elds in audio recognition and multi-task learning are analysed, specically for how and what they research and consecutively how these areas came together. Frameworks for the development of deep learning are also examined, determining which lessons to take in tackling this issue and nding the current state of aairs. This study provides the basis for the developmental software framework, CombinaTorch, built on top of PyTorch, that allows free assembly of multi-task pipelines. The framework is subsequently used to implement several set-ups described in the literature, where their comparative reduction in amount of work is assessed. In order to validate the expressiveness of the framework, another delve into the literature is made, discussing several scenarios and their implementation viability with the framework.

Nowadays, a public library is no longer a place just for the collection of books. It has offered multiple functions to take the responsibility of being a public cultural institution for all citizens. However, in the Netherlands, libraries are losing their members, especially young adults. With the development of technology, a public library is no longer the only choice for books. Libraries are encountering a challenge to attract young people back. They need to strive for providing new kinds of public services to attract this potential new audience.

Westfriese Bibliotheken is one of these libraries which has 10 decentralized community libraries in 6 places in the north of the Netherlands. Westfriese Bibliotheken is facing a challenge to reach a bigger group of the generation of youngsters who think a public library is a dull place to go. To narrow down the scope, the book discovery experience for young adults between 18-25 years old was focused. Augmented reality as the technology this project worked with has been proved to benefit the library context.

Hence, the objective of this thesis is to explore a new book discovery experience for young adults in the public library in Stede Broec.

From the theoretical study (chapter 2) about the book discovery behavior, the behavior pattern was investigated and worked as the design requirements. User research (chapter 3) was conducted to understand the current book discovery experience in the context through contextmapping. It discovered users' concerns and expectations about the book discovery experience. Moreover, the opportunity of this project: designing a new book discovery experience to facilitate serendipity was concluded based on user research findings. Under the framework of the serendipity theory (chapter 4), design opportunities for facilitating serendipity in the current library were gathered. The study on augmented reality (chapter 5) helped to define design qualities and technologies that this project can make use of.

With the research insights, a design goal with three design qualities (fascinating, playful, and inviting) and relevant design requirements were formulated (chapter 6). They contributed as a guideline to the design solution.

Ideas were gathered through a creative session and based on these ideas three initial concepts were generated. Concepts were evaluated and iterated with users, clients, and fellow students. The final design outcome is a phone-based AR application with two main parts: exploring and browsing parts. It offers both basic and additional information about books to stimulate users to experience serendipity in the public library.

Finally, an evaluation test was conducted to validate if the final design can fulfill the design goal. The final design achieves a good usability and desirability performance evaluated by 6 participants in the library. The evaluation test demonstrated that the additional information about books and interaction between the physical and digital world can let users feel fascinated, invited, and playful during the book discovery experience.

Our world climate is an incredibly complex and ever-changing system. The reality of climate change and the consequential pressure to act fast to reduce greenhouse gas emissions requires immediate attention. And yet, despite this unprecedented urgency, active public engagement needed to motivate meaningful change is still missing.

This graduation project explores how to utilize Generative Adversarial Networks (GANs) – a subset of artificial intelligence (AI) – to motivate public engagement through the emotion of fascination and element of play. The final design of this exploration is an interactive exhibition piece that allows its users to create novel, AI-generated landscapes.

In the first phase of the project, hands-on exploration of the use of GANs as a design opportunity was conducted and the experiences of the exploration process documented. Further research through interviews with climate researchers and literature review into the topics of climate change helped to get an understanding of the climate context and map out and narrow down the solution space for the following ideation phase. The context research revealed that the current climate debate is deeply emotional, with mostly negative emotions like fear and hopelessness in its center. Yet, it was also found that fostering positive emotions like fascination and joy is a potential means to increase interest and motivate people to engage with climate initiatives. Based on the context exploration, a series of design prototypes were created and tested with participants to inform the final design.

The final concept LANDSHAPES is an interactive exhibition piece intended for the museum context. It features a circular projection of AI-generated aerial landscapes – images that look real, yet have never been seen by anyone before. The installation enables participants to become absorbed in the morphing landscapes and experience their potential impact on our surroundings in a playful, open-ended manner.
A preliminary installation was evaluated with participants using qualitative methods of observation and interviews. The installation evoked a deep sense of beauty, calmness, and inspiration that contributed to a meditation-like, engaging experience. Even without a predefined narrative, participants automatically associated the morphing images with our changing planet and ascribed meaning to the abstract visuals. The addition of control over the moving landscapes resulted in a mindset shift, as the experience transformed from a passive and meditative experience to an experience with additional cognitive involvement. The preference of experience differed between participants.

The design research conducted in this project strengthens the assumption that experiences with positive emotion in its center can be used to foster initial engagement with the topic of climate change. Furthermore, it showed that GAN technology can be used to evoke an emotional response, which offers interesting opportunities for future work. More specifically, the research suggests the potential for further development of playful experiences to make climate data accessible and engaging to the general public, as well as an opportunity to explore GANs that utilize emotional value as an input to generate targeted media that reinforces or counteracts the viewer’s emotional response.

Knowledge sharing brings overall value to organisations in the growth of the business and its people. For designers, sharing stimulates creativity and encourages personal development via collective sharing within the community, making happier individuals and by extension, a happier team. For knowledge sharing to happen, cultural change and the way people do things needs to be improved and this can be done through better management of knowledge within the organisation. The company context of this project, Accenture Interactive (Amsterdam), wants to explore ways to support its designers in their creative process and strengthen their culture of sharing by streamline the process of designing across the organisation through a digital knowledge platform. Within this research, knowledge sharing is defined as the information that was exchanged between people and knowledge material is defined as materials required for the sharing to occur. The goal of this research is to support the designer's process of designing, information of projects, people and materials should be made visible and easily accessible across the organisation. Therefore, an informative knowledge platform is designed to serve as a source of inspiration and foster sharing within the design team. This thesis documents both the findings of the research as well as the development of the knowledge platform. The challenge of this project is to understand what content do designers in Accenture Interactive share within the design team, what do they generate and how can sharing of knowledge be facilitated. The research will provide insights and define requirements that will contribute towards the building of the digital platform. To this end, the central aim of this research has been to explore ways for sharing of knowledge that supports designers in their creative work. A practice-based research methodology has been employed, with research methods including interviews, case studies and observations has been done with the design team across different departments and roles. The outcome of this project is a conceptual design of a digital knowledge platform, a common place where designers are able to access shared design resources, share tips across the design teams can be facilitated, thus fostering the sharing culture. The platform was designed in collaboration and evaluated with the designers in practice.

Artificial intelligence (AI) is expected to play a transformational role in health and wellbeing. Search (i.e. information retrieval) technologies already play a significant role in healthcare research and practice. Relevance feedback in Search is vital for system evaluation and improvements. However, in small user scale contexts, the exploitation of user behaviors may not infer valid relevance judgments. Therefore, engaging users to provide such feedback explicitly is essential for improving search performance (i.e. effectiveness). However, previous research has found that users are generally reluctant to provide explicit feedback in digital environments, and the willingness decreases overtime in some experiments. In collaboration with myTomorrows, an Amsterdam-based pharma-tech company, this Master thesis aims to find answers to the challenge mentioned above through a specific context of myTomororws AI-powered treatment Search which has the urgent need for engaging healthcare professionals (HCPs) in providing relevance feedback on search results (e.g. Clinical Trials and Expanded Access Programs) for system evaluation and improvements. Through Human Centered Design methods such as interviews, observations, and speed dates, the project yielded a future myTomorrows Search design enhanced with three relevance feedback collection concepts. As research materials, the concepts were tested and evaluated by nine HCPs from three countries (the Netherlands, China, and Brazil). The user study results indicate that embedding utility, as the motivator, in relevance feedback collection appeals to HCPs more than using motivators such as altruism or enjoyment. Moreover, the best point of user engagement is identified as the moment between users finishing the examination of information and starting the next ones. Additionally, this study generalized the project process and user study insights into a four-stage guide for designing explicit feedback collection in text-base Search. Although it remains unvalidated, this guide has the potential to apply to other small user scale contexts, guiding or inspiring user researchers and designers to design explicit user feedback collection in Search.

Deep learning models have achieved state-of-the-art performance on several image classification tasks over the past years. Several studies claim to approach or even surpass human-levels of performance when using such models to classify images. However, these architectures are notoriously complex, thus making their interpretation a challenge. This limited interpretability, in turn, leads to several issues, such as restricting their applicability to critical domains like health care and finance.

Several methods in literature attempt to address this issue by providing local explanations which describe individual predictions or global ones that explain the model behaviour for a specific class. When focusing on global methods, we notice that they are limited with respect to the interpretability queries that they answer. For instance, consider we want to query whether the simultaneous presence of two objects is associated with predicting a specific class. To the best of our knowledge, there is no existing method that can tackle such a query type due to their limited expressivity. In this thesis, we address this limitation by answering the following research question: to what extent can image classification models be interpreted by analysing semantic features extracted from groups of salient image pixels?

We begin our study by investigating existing research work to devise the ideal characteristics that an interpretability method should adhere to. Our analysis highlights the aforementioned gap regarding the query complexity that existing methods cover. To address this limitation, we propose a new global interpretability method called SEmantic Feature Analysis (SEFA). To elaborate, it combines explanations of individual image predictions with semantic descriptions provided by human annotators about them, thus extracting the aforementioned semantic features. We argue that by analysing a structured data representation extracted out of semantic features will allow us to answer a wider range of interpretability queries compared to existing methods. The proposed method poses several challenges, such as identifying the number of image annotations required to obtain reliable results at a reasonable annotation cost.

Our results show that SEFA provides its users with the flexibility to answer several types of interpretability queries, including the ones that we found existing methods to be lacking. Further experimentation on its hyperparameters using three separate image classification tasks provides us with a set of suggested settings that one should use on similar datasets. Finally, we showcase the ability of SEFA to output semantic features relevant to the model classification behaviour by fine-tuning existing model architectures on biased datasets and evaluating whether the salient semantic features output describe the previous bias.