Autism Spectrum Disorder (ASD) is a neuro-developmental disorder reported to affect around 58 million people across the world. Due to their struggles with social communication in real life, they prefer to use online mediums to form social connections and are better able to maintain meaningful relationships through their communications over these online mediums. However, empirical studies have revealed that autistic users are not as efficient in using these online mediums when they have to search for information. While studies in the past have resulted in a growing list of guidelines to improve the web search experience of autistic users, whether existing online ISTs adhere to these guidelines has not been studied extensively. Thus while we know what the autistic users need to efficiently look for information on the web, we do not know if and how the ISTs are catering to their accessibility needs.

To advance knowledge in the accessibility of popular online ISTs when catering to the information needs of autistic users, we conducted an empirical exploration focusing on the accessibility of the textual
information in the IST responses. We specifically examined the responses on three aspects of text accessibility for autistic users - (1) text structure, (2) text readability, and (3) text concreteness using
“accessibility indicators” i.e. quantifiable features created by us based on widely accepted web content accessibility guidelines for autistic users. Due to the lack of standard query datasets for autistic users, we had to generate synthetic queries representative of autistic users’ information needs to collect responses from four popular ISTs Google, Bing, Gemini, and ChatGPT. We examined the differences in the way ISTs respond to queries characteristic to autistic users in contrast to queries typically asked by the general public, as well as the differences in IST responses across different kinds of queries asked by autistic users. We also investigated if an autistic user were to reformulate their query and explicitly state that they are autistic in their query would nudge the IST to respond differently to their query.

Juxtaposing the IST responses generated for control and ASD group queries revealed that the ISTs not only responded differently to the queries asked by the two user groups, but the responses generated for the control group were surprisingly more accessible for autistic users than the IST responses generated for their queries. For queries typically asked by autistic users, none of the ISTs produced responses
suitable for autistic users. However, each IST had its strengths in the context of the accessibility which can employed in the other ISTs to improve the general accessibility of the responses generated by all ISTs. We also found that our query and prompt reformulation strategies affected the accessibility of each IST differently. In general, the reformulations did nudge the ISTs to generate responses that are more suitable for autistic users. This presents an optimistic solution to the issue of ISTs not catering to the accessibility needs of autistic users when they search for information on the web. Our results highlight areas where popular ISTs can be improved to make their responses more accessible to autistic users. Our systematic empirical investigation pipeline can also be extended to investigate IST responses on other factors that influence the accessibility of web content for autistic users.

This thesis investigates the enhancement of sentence decomposition in Large Language Models (LLMs) through the integration of linguistic features, including constituency parsing, dependency parsing, and abstract meaning representation. Traditional decomposition methods, which often rely on rule-based approaches, struggle with highly intricate sentences. By incorporating detailed linguistic features and employing reasoning steps and supervision in prompts, we aim to improve the comprehension and decomposition capabilities of LLMs. Experimental results show that integrating these linguistic features has the potential to improve decomposition performance. However, these enhancements also introduce new error types and increase computational costs. This study identifies several error types in the programs generated by LLMs, including format, decomposition, and conversion errors, emphasizing the need for further refinement in model training and prompt design. This research lays a step toward more accurate and efficient processing of complex sentences in LLMs, encouraging ongoing development and optimization in this field.

While most large language models (LLMs) are powerful, they are primarily designed for general purposes. Consequently, many enterprises and institutions have now focused on developing domain-specific models. In the realm of education, an expert LLM can significantly enhance students' ability to find information more effectively and reach their learning goals. Nevertheless, the training of such expert models in education remains largely unexplored. This study explores this research gap by developing a framework to transform semi-structured educational web data into structured datasets and perform instruction tuning on foundation models. Additionally, we conduct a comprehensive performance analysis to determine how various training factors affect model performance.

We first employed a systematic and cost-effective approach involving web data extraction, data cleaning, validation, task design based on student surveys, and automated instruction instance generation using LLMs. Human evaluations confirmed the quality, especially the relevance and accuracy of these datasets.

This study then investigates the impact of various training techniques on domain-specific educational large language models (LLMs) performance. Our experiments reveal that further pre-training enhances model performance, especially with domain-specific terminology, although the performance gains decrease as the dataset size increases. Furthermore, multi-task training also improves model relevance, accuracy, and clarity, but less correlated tasks and datasets can present challenges. These challenges include increased complexity and potential degradation in performance due to the model having to switch between diverse tasks. Lastly, this study conducts a comparative analysis of different models and it highlights trade-offs between computational resources and performance.

The findings demonstrate that a structured approach to dataset generation and strategic training can effectively develop domain-specific LLMs in education. This research benefits the development of educational LLMs and provides a foundation for future researchers to build more specialized models in various domains.

Recommender systems play a considerable role in the consumption of music, also for children. Children are easily influenced, inappropriate song lyrics can negatively impact children's behaviour and personality, by teaching inappropriate language or harmful biases. We argue for a need for recommenders to protect children from inappropriate content. Recommender systems should consider what content is and is not well-suited for children. To guide future design and development of recommender systems for children, this research probes the music lyrics of Pop, Rock, Country, Rap, and R&B for inappropriate content. We achieve this by an empirical analysis using four existing algorithms to scan different facets of inappropriate content in the lyrics in a dataset of 37,993 songs from the Genius database. The outcome of this empirical exploration reveals that Rap and R&B are the most inappropriate music genres and advises to be cautious when recommending these genres to children. Results also show that inappropriate content is highly prevalent in music. We address the need for a filter for recommender systems, to filter inappropriate songs for children.

Music recommender systems are increasingly present in our lives, and it is important to keep trying to improve recommendations in order to make them match the users preferences as well as possible. To achieve this, a vast amount of song and user data has to be analysed and taken into account. One of the approaches to do this, includes analyzing different audio features in order to find other songs with similar traits. The majority of the research and data in this sector is focused around adults, with little research surrounding children, which can result in worse recommendations for this demographic. In this paper, the focus is shifted towards children with the purpose of filling that gap. This is achieved by examining the prominence of specific song features among children of different age groups, expanding the knowledge on listening habits of a major demographic. More specifically, the research presented in this paper explores the prominence of various song features, aiming to find a connections between these features and the listening habits of children of specific age ranges from 8-18. This paper’s conclusions will offer potential enhancements, which can improve existing recommender systems by considering findings for their design. These findings will therefore allow for a more tailored experience for children of different age ranges, increasing overall user experience.

The exponential growth of online content and consumer options has increased the reliance on recommender systems. Children, as a distinct user group, require tailored recommender systems different from those for adults. However, research on recommendation models for children is limited. This study evaluates deep learning recommendation models according to several performance and beyond-performance metrics on data from underage users on the Last.fm streaming platform, offering insights into optimal recommendation strategies for this demographic. Traditional non-deep learning models are used as baselines.

The online presence of e-commerce platforms, publishers, and other businesses has been increasing with the rise of internet use. These companies regularly expose their (potential) customers to massive amounts of media and products. It is vital for these companies to elicit interactions from these individuals in order to sell products or subscriptions. Engagement with online items, such as articles or products, are often measured using clicks. A well-known and proven method for improving online interactions is the use of clickbait. However, this practice is known to degrade the reputation of businesses. Widespread use of obvious clickbait is, therefore, not a sustainable solution for reputable companies. Research into alternative solutions to foster engagement is mostly taken from the perspective of the consumer. These solutions mainly involve various recommender algorithms that match consumers with items and articles that interest them. However, research from the perspective of the producer is extremely limited. The producer of the content has different incentives than the consumer of the content. Their main goal is to elicit an interaction from the consumer, while the consumers' main goal is to find something that interests them.

In this thesis, inspired by the work on personalization for consumers, we make the producer the main stakeholder. We explore whether the personalization of subject lines for groups of users with similar behaviour patterns and interests improves the engagement of consumers on newsletters. With the digital marketing company Basedriver as the use case, we personalized the headlines of marketing e-mails in newsletters and performed experiments to analyse the effectiveness of our technique. We hypothesise that e-mail marketeers can improve engagement by personalizing subject lines to address the personal interests of consumers. By looking at the content of articles previously clicked by consumers, they can be grouped into user groups with similar interests. Information on the interests of users from these groups can then be leveraged to generate a subject line addressing the interests of the group. The historic customer behaviour data and platform used for the validation of our new personalization strategy was provided by Basedriver and Hearst, who assisted in the research. Basedriver also provides the platform for collecting the data from the experiment.

This thesis presents the design and evaluation of a comprehensive system for developing voice-based interfaces to support users in supermarkets. These interfaces enable customers to convey their needs across both generic and specific queries. While current state-of-the-art systems like GPTs by OpenAI are easily accessible and adaptable, featuring low-code deployment with options for functional integration, they still face challenges such as increased response times and limitations in strategic control for tailored use-case and cost optimisation. Motivated by the goal of crafting inclusive, personalised, and efficient conversational agents, this study advances on three fronts: 1) a comparative analysis of four popular off-the-shelf speech recognition technologies to identify the most accurate model for different genders (male/female) and languages (English/Dutch); 2) an assessment of the effects of personalised recommendations versus generic responses, using a blindfolded, counterbalanced within-subject experiment; and 3) the development and evaluation of a novel multi-LLM supermarket chatbot framework, comparing its performance with a specialized GPT model powered by the GPT-4 Turbo, using the Artificial Social Agent Questionnaire (ASAQ) in a counterbalanced within-subjects experiment and qualitative participant feedback. Our find-ings reveal that OpenAI’s Whisper leads in speech recognition accuracy across genders and languages, users significantly prefer personalised chatbots over the non-personalised counterparts and that our proposed multi-LLM chatbot architecture outperformed the benchmarked GPT model across all 13 measured criteria, including statistically significant improvements in four key areas: performance, user satisfaction, user-agent partnership, and self-image enhancement. The thesis concludes by presenting a simple method for supermarket robot navigation by mapping the final chatbot response to correct shelf numbers towards which the robot can plan sequential visits. This later enables effective use of low-level perception, motion planning, and control capabilities for product retrieval and collection. We hope this work encourages more efforts into using multiple, specialised smaller models instead of always relying on a single powerful model.

Scratch is a block-based programming language. It is designed to be simple and syntax error avoidant. This makes Scratch an accessible platform for cultivating coding skills. Many young learners are taught about different programming skills using various project types as examples. For instance, games are used as an engagement tool, and various games can motivate new learners to make their own. This influx of new published projects are manually classified into different types by using tags in the project descriptions. However, this manual classification only happens when the user adds the appropriate tags. This calls for an extension of that feature, which could help classify all projects published on the Scratch website. This has the goal of improving the browsing process, especially for new projects. To address this gap, in this initial iteration we leverage the fact that there are likely similarities, or even various project type defining features that would help improve the accuracy of classification through machine learning. Filling this gap also opens the possibility of automatic classification, depending on the accuracy of the results. Within this study, various machine learning models were tested with quantifiable project features as input data. The accuracy scores were compared to draw conclusions on how well various features extracted from Scratch projects performed for classification.

Although existing work demonstrates that the usage of Scratch strengthens programming skills, little is known about the effect of children encouraging or criticizing each other when programming on the development of their programming skills. To address this gap, we conducted a data analysis to reveal the correlation (or lack thereof) between the sentiment of comments on a Scratch project and the creator's inclination to continue producing projects.

In particular, using a known dataset of Scratch projects, which we enriched with new projects, we examined several independent variables that capture sentiment on comments from different perspectives, e.g. the absolute or relative number of comments with a particular sentiment score that a user receives, or the ratio of comments that fall in a certain category. We also examined several dependent variables that capture the creator's inclination to continue producing projects, e.g. the number of projects they create or the total time that they are active on the platform.

The results of our experiments reveal that the absolute and relative number of comments with a specific sentiment score given to a user have a low correlation to the time that a user is active on the platform. Additionally, the ratio of the number of positive and negative comments over the total number of comments given to a user has a low correlation to the time that a user is active on the platform. Finally, the number of comments that a user receives has a low correlation to the number of projects a user creates.

Scratch is a popular, visual programming language aimed at children, and is used by teachers and after school code clubs to teach their students about programming. Measuring whether they understand the underlying concepts, however, is a difficult task. In this research, we tried clustering Scratch projects by complexity to help students improve their programming skills. We did this by selecting an existing data set to extract features that indicate code complexity. Before, researchers attempted clustering on one metric that globalises the project’s complexity. Different researchers set out to measure the growth of the students by clustering the projects the students created.
With that in mind, we adopt a partition-based clustering algorithm to cluster the projects, as this method indicates outliers. We examine the quality of these clusters using the silhouette coefficient. We set up five experiments with different input vectors to make out the impact each input has on the clusters. We did not find a clear indication of the projects being clustered by the selected features. This could mean that Scratch projects are not suitable to measure a high-level understanding of programming concepts. Including the project name in the input vector had a negligible effect on the outcome of the experiments.

The spread of fake news has negatively impacted society. Prior efforts in Natural Language Processing (NLP) have employed machine learning models and Pre-trained Language Models (PLMs) like BERT to automate fake news detection with promising results. These models excel at text classification tasks, but their dependence on large context-specific datasets presents a hurdle in the dynamic and ever-evolving context of fake news. The recent emergence of Large Language Models (LLMs) offers a potentially transformative innovation. LLMs have demonstrated great promise in NLP tasks with little additional training data. Compared to PLMs, LLMs have broader knowledge and enhanced reasoning capabilities, suggesting their suitability for fake news detection. This study proposes an investigation which considers multiple perspectives to probe the applicability of the effectiveness, opportunities and challenges associated with leveraging LLMs for automated fake news detection. We leverage multiple state-of-the-art LMMs by using them at multiple levels of guidance to validate their accuracy and task-specific bias. The main contribution of this work is a better understanding of the role LLMs can play in automated fake news detection, the pitfalls that should be avoided when leveraging them, the most effective approaches when employing them, and the future challenges that need to be addressed. Our key contributions include a better understanding of how to employ LLMs for fake news detection, their strengths and weaknesses, and some practical recommendations for deploying them in the real world. Our work shows that while LLMs hold great potential for advancing automated fake news detection, thoughtful consideration of their limitations and careful application refinement are essential for their effective deployment in the fight against fake news.

Searching the web on debated topics, which are issues under active discussion and where individual opinions diverge, can be highly challenging. It requires users to approach their queries objectively, browse many resources, and accept a certain level of uncertainty, even if it conflicts with personal values. In particular, confirmation bias, the tendency to favor information that reinforces beliefs or attitudes, impacts how people gather and interpret information retrieved from search engines when searching on debated topics. It can have far-reaching effects, potentially leading to conflicts, extremism, and polarization.

In an innovative approach towards mitigating the negative effects of confirmation bias on web search on debated topics, we propose implementing a boosting intervention aimed at enhancing Intellectual Humility (IH) – an individual’s ability to acknowledge the fallibility of one’s own beliefs and the limits of one’s knowledge while remaining open to learning from others’ perspectives even if they differ from their own viewpoint. While previous research has highlighted the potential benefits of boosting IH as a means to mitigate confirmation bias, its impact on users’ search behavior has yet to be explored.

Our work bridges this gap through two randomized preregistered user studies, gaining valuable insights into the effectiveness of IH-boosting interventions in mitigating confirmation bias. In the first study, we assessed the effect of three boosting interventions with different levels of complexity on users’ context dependent IH. In the second study, we examined the effects of these interventions on web search behavior.

The first experiment successfully demonstrated the effectiveness of the interventions in boosting participants’ IH across all three treatment groups. However, applying these interventions to web search, no significant differences in search behavior were observed. Our exploratory findings reveal that both individual and environmental factors, including occupation, personal viewpoints, and search results order, shape the impact of IH-boosting interventions on online search behavior, with varying effects observed across different debated topics. We hope this study inspires, and is an initial basis for continued efforts to explore the multifaceted relationship between IH, information-seeking behavior, and responsible opinion formation, ultimately promoting a more informed and unbiased online discourse.

Transformer models have proven to be effective tools when used for determining the readability of texts. Models based on pre-trained architectures such as BERT, RoBERTa, and BART, as well as ReadNet, a transformer model which is dedicated to readability assessment, have shown some very promising results. However, there is a lack of research focused on comprehensively analyzing these models' performance at a more granular level. Moreover, GPT-2, a member of the very popular GPT transformer family, has never been adapted to and tested in readability assessment. The work presented in this paper fills these knowledge gaps by analyzing the behavior of the five aforementioned models and reflecting on their performance on separate classes of text difficulty. Seeing how they perform on texts of various complexity levels is vital to understanding their behavior and limitations, which will in turn further the knowledge of the situations in which each readability tool achieves the most optimal results.

Readability plays a vital role in the transfer of knowledge or information. This is especially true for children who are more than anyone gaining new knowledge every day, yet their reading abilities are still in active development. Although readability assessment of children's literature is an active topic of research in many different languages, research into the assessment capabilities of well-known Dutch traditional readability formulas on texts aimed at Dutch children remains in its infancy. In this paper, we explore the performance of four well-known Dutch traditional readability formulas when applied to different types of texts targeting Dutch children. More specifically, we examine the applicability of the Flesch-Douma, Leesindex A, CLIB and CILT formulas on excerpts of school materials, books, and media for children aged 6-14 years old. Outcomes from this empirical exploration reveal that the readability formulas do not perform similarly across reading materials. As a result, choosing a readability formula based on both the type of reading material and a child's age can improve the readability estimation, which in turn can help connect children to understandable resources.

As music video streaming occupies a significant market share in how people consume music, gaining an understanding of user behavioural patterns becomes increasingly crucial. This understanding can enable better music video streaming experiences by tailoring them towards more personalized and user-centric designs. Though prior works have emphasized user behaviour during solely listening to music, understanding user actions/clicks while consuming music videos remains largely unexplored. Given the unique experience offered by the combination of audio and visual elements, there is a need for focused research in this area.

Therefore this study attempts to bridge this research gap by collecting and analysing a large dataset of streaming sessions from a music video streaming company - XITE. In total, we analyzed 1.8 million sessions from approximately 270,000 unique users. The behaviour exhibited during those sessions is interpreted as a language and modelled using the Language Model - Doc2Vec. This facilitated the conversion of session action sequences into embeddings. Our findings suggest that music video streaming sessions exhibit cohesive user interaction patterns, which can be grouped into distinct clusters, thereby enabling the detection of distinct behavioural patterns across user sessions.


Furthermore, previous studies have indicated that user interactions with multimedia streaming platforms can be influenced by the context in which content is consumed. Extending these findings, our analysis of behavioural clusters revealed that certain user behaviours while consuming music videos are associated with specific music video genres and temporal factors. For instance, we discovered that passive sessions predominantly commence around 10 am, while sessions requiring more active engagement typically start in the evening. The insights derived from this study are valuable for improving user-centric design in music video streaming platforms and providing businesses with data-driven recommendations for strategic planning.

The cover of a children's book could play a crucial role in attracting the attention of young readers and influencing their decision to pick up and read the book. In this study, we aim to identify the factors that contribute to the appeal of children's book covers. Given that the target audience for children's books changes as children grow older, we divide our analysis into five age groups. We conduct a series of experiments in which we analyze three distinct aspects of covers: the visual attributes, visible objects, and implied story. Our results show that visual attributes play an important role in determining the appeal of a book. We also find that objects and themes present on a book cover vary between age groups. By understanding the elements that contribute to the appeal of children's book covers, we hope to improve recommendations for young readers and encourage a lifelong love of reading.

Recently, a few children-centered recommendation systems have been created and evaluated. How- ever, these systems required user interaction to cre- ate ground truth to evaluate the result. This research aims to compare some of the traditional recommen- dation models and explore which trait could impact the recommendation process most for different age group users. The result shows the children friendly model does not achieve higher accuracy than the traditional recommendation model. But the book length model and emotion analysis model shows the potential of a good RS that can help children choose books, and cover image recommendation models are only working with younger age

Reading is an essential skill for any child to learn, and finding enjoyment in it can greatly contribute to developing proper reading comprehension. Finding the books they like could prove to be difficult. Utilizing collaborative filtering recommender systems to recommend books to children is a tricky task, the lack of user feedback makes it difficult to accurately recommend books they would enjoy. Using content based recommender systems might be preferable, but what book features could a recommender system like this base recommendations on? This research explores the idea of utilizing the textual complexity of books and their descriptions as such a possible feature. By evaluating how accurate readability formulas can predict the age a book is intended for, how the variability and length of sentences vary per age and analysing the difficulty of words used, this paper finds that the descriptions of books intended for younger audiences might not be aimed at them, but instead at their parents. These findings imply that basing recommendations of the textual complexity of book descriptions might not be the most useful feature to base recommendations of.

This study examines traits that may be derived from book metadata and identifies statistically signifi- cant patterns and trends in order to assist recom- mender system designers for children’s books in selecting the most important traits to take into ac- count. This research focuses on descriptive such as publish year and structural metadata such as the chapters’ count. We rely on data from two different data sets, Goodreads and Wikisource.
The findings of this study, such as the fact that there is a relationship between the number of pages and the reader’s age, may have an impact on the design of recommender systems for children’s books.
This is all with the respect to the fact that children are a vulnerable audience, making it challenging to not infringe on their privacy while gathering infor- mation that can be used in statistical experiments in many fields such as education and health.