Hidden Intentions Behind Recommender Systems: Understanding Online Manipulation as Covert Influence

Master thesis (2024)

Authors

J. Mondragón Briseño Technology, Policy and Management

Contributors

M.B.O.T. Klenk (mentor)
C.N. van der Wal System Engineering - TPM (graduation committee member)
Faculty
Technology, Policy and Management
More Info
expand_more

Published Date

09-09-2024

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Technology, Policy and Management

Abstract

The emergence of social media and recommendation systems has profoundly transformed user interactions with digital content, bringing in both opportunities and ethical challenges. This thesis scrutinizes the online manipulation exerted by RS, which, while enhancing engagement and profitability, can guide user behavior subtly and without their awareness. This definition is based on covert influence, a particular account of manipulation.

A critical examination of existing literature reveals a significant gap: while the conceptual framework for online manipulation is well-discussed, empirical studies providing concrete evidence are scant. This research addresses this deficiency by employing an agent-based model to simulate interactions between users and recommendation systems, aiming to systematically analyze and quantify the effects of covert manipulation on user preferences.

This study contributes to the field by operationalizing the concept of manipulation within a controlled simulation of a book recommendation system, providing a clearer understanding of its mechanisms and effects. This approach not only offers insights into the ethical implications of RS but also aligns with current legislative movements, such as the European Union's Artificial Intelligence Act, aimed at regulating and mitigating harmful manipulative practices by intelligent systems. The findings are intended to guide the design and regulation of RS to ensure they serve the user's interests without compromising ethical standards.

The results show that book recommendation systems can modify user preferences by 9.79\% when prioritizing items covertly, while awareness of the intentions and social influence can diminish the effect of the manipulative algorithm's intention to 4.01\% and 3.64\%. When compared to the 2.58\% change in the case of a non-prioritized RS, the values provide a measurable estimation of the difference between manipulative and non-manipulative book RS for the change of user preferences after interacting with it for some time.