Exploring the effects of interactive interfaces on user search behaviour

Abstract

Interactive information retrieval (IIR) is a user-centered approach to information seeking and retrieval. In this paradigm, the search process is not confined to a single query and a static set of results. Instead, it emphasises the active involvement of users in refining their information needs, iteratively modifying queries, and exploring retrieved content. IIR studies research how to facilitate a more tailored and practical search experience, adapting to the evolving requirements and preferences of users. In this thesis, we focus on four distinct yet interrelated areas in the domain of IIR to have a better understanding of the interaction between the user and the information retrieval system.

How users interact with a search system depends on several things, including, but not limited to, the device on which they search, the interface, the task at hand, their prior expertise and so on. In Chapter 2, we explore the role of search interface layout and task complexity on user search behaviour and their task effectiveness. We aim to reproduce the setup of two IIR studies conducted a decade back that explored the effect of the search interface and task complexity on user behaviour. As search interfaces have kept on evolving, we ask the question of whether user search behaviour has remained the same. Our goal is to observe to what extent the findings from those two studies still hold today.

Next, we focus on a specific aspect of IIR, called Search as Learning (SAL), where users participate in learning-oriented search tasks. These search tasks are exploratory, involving multiple iterations that require cognitive processing and sensemaking. It often requires the searchers to spend time scanning, viewing, comparing and understanding documents. Prior studies have shown that, in offline classroom learning scenarios, active reading tools like highlighting and note-taking tools help learners better process what they read and consequently help their learning outcomes. In Chapter 3, we explore to what extent highlighting and note-taking tools, when we implement and incorporate them into the interface of a standard search engine, affect search behaviour and users’ learning outcomes. We intend to explore if they are also beneficial in the online SAL scenario.

While designing and incorporating widgets (e.g. a note-taking tool) in a search interface, researchers face numerous design decisions regarding where to place the widgets, what they should look like, what functionalities they must have and so on. Due to budget constraints, it is not feasible to run A/B tests on all possible options. Thus, next in Chapter 4, we build a user model leveraging Search Economic Theory (SET), where we, for the first time, incorporate positional information of widgets. SET is based on micro-economic theory that assumes that users are rational agents—they aim to maximise profit and minimise cost. Previous work has utilised SET to develop models for predicting user interaction under various circumstances where widgets on the SERP are typically considered fixed, and their position is not part of the user model definition. Thus, in this thesis, we explore if we can derive a sensible hypothesis of user behaviour using our user model that incorporates positional information of widgets.

Finally, having so far dealt with documents in text modality of presentation, in Chapter 5 we look into the voice modality of presentation in the context of collecting relevance judgments for building test collections by employing crowdworkers. Previous studies have explored to what extent various factors like document length, topic difficulty, cognitive aspects of crowdworkers, etc., affect their relevance judgement effectiveness. However, none of them considered the presentation modality of the documents to be judged. Audio-only devices are getting popular, and leveraging these devices can increase the scope of collecting relevance judgements. For example, crowdworkers can judge document on-the-go, those with visual disabilities can also participate in the judgement task and so on. Thus, we observe how the presentation modality of documents, that is, representing them as text or voice, affects the relevance judgement effectiveness of crowdworkers. We also explore to what extent there is an interplay of document length and cognitive aspects of crowdworkers with the presentation modality.
With the studies conducted in this thesis, we make scientific contributions to the field by providing novel insights covering a breadth of topics and advancing our understanding of the field. We hope our contributions pave the way for further research and exploration in the field of IIR with the ultimate goal of enhancing the web search experience and performance of users.