U.K. Gadiraju
73 records found
1
The State of Pilot Study Reporting in Crowdsourcing
A Reflection on Best Practices and Guidelines
Pilot studies are an essential cornerstone of the design of crowdsourcing campaigns, yet they are often only mentioned in passing in the scholarly literature. A lack of details surrounding pilot studies in crowdsourcing research hinders the replication of studies and the reproduc
...
The ability to make appropriate delegation decisions is an important prerequisite of effective human-AI collaboration. Recent work, however, has shown that people struggle to evaluate AI systems in the presence of forecasting errors, falling well short of relying on AI systems ap
...
"decisionTime"
A Configurable Framework for Reproducible Human-AI Decision-Making Studies
Empirical studies have extensively investigated human decision-making processes in various domains where AI systems are incorporated. However, comparing and replicating these studies can be challenging due to different experimental configurations. Moreover, the existing contexts
...
Web search has evolved into a platform people rely on for opinion formation on debated topics. Yet, pursuing this search intent can carry serious consequences for individuals and society and involves a high risk of biases. We argue that web search can and should empower users to
...
Human intelligence continues to be essential in building ground-truth data, training sets, and for evaluating a plethora of systems. The democratized and distributed nature of online crowd work - an attractive and accessible feature that has led to the proliferation of the paradi
...
Akal Badi ya Bias
An Exploratory Study of Gender Bias in Hindi Language Technology
Existing research in measuring and mitigating gender bias predominantly centers on English, overlooking the intricate challenges posed by non-English languages and the Global South. This paper presents the first comprehensive study delving into the nuanced landscape of gender bia
...
With the integration of AI systems into our daily lives, human-AI collaboration has become increasingly prevalent. Prior work in this realm has primarily explored the effectiveness and performance of individual human and AI systems in collaborative tasks. While much of decision-m
...
Disentangling Web Search on Debated Topics
A User-Centered Exploration
When using web search engines to conduct inquiries on debated topics, searchers' interactions with search results are commonly affected by a combination of searcher and system biases. While prior work has mainly investigated these biases in isolation, there is a lack of a compreh
...
Dealing with Uncertainty
Understanding the Impact of Prognostic Versus Diagnostic Tasks on Trust and Reliance in Human-AI Decision-Making
While existing literature has explored and revealed several insights pertaining to the role of human factors (e.g., prior experience, domain knowledge) and attributes of AI systems (e.g., accuracy, trustworthiness), there is a limited understanding around how the important task c
...
DECI
The 2ndTutorial on Designing Effective Conversational Interfaces
Conversational User Interfaces (CUIs) have been argued to have advantages over traditional GUIs due to having a more human-like interaction. The growing popularity of conversational agents has enabled humans to interact with machines more naturally. People are increasingly famili
...
To Err Is AI!
Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems
Powerful predictive AI systems have demonstrated great potential in augmenting human decision making. Recent empirical work has argued that the vision for optimal human-AI collaboration requires ‘appropriate reliance’ on AI systems. However, accurately estimating the trustworthin
...
DECI
A Tutorial on Designing Effective Conversational Interfaces
Conversational User Interfaces (CUIs) have been argued to have advantages over traditional GUIs due to having a more human-like interaction. The growing popularity of conversational agents has enabled humans to interact with machines more naturally. There is an increasing familia
...
ContextBot
Improving Response Consistency in Crowd-Powered Conversational Systems for Affective Support Tasks
Crowd-powered conversational systems (CPCS) solicit the wisdom of crowds to quickly respond to on-demand users' needs. The very factors that make this a viable solution - -such as the availability of diverse crowd workers on-demand - - also lead to great challenges. The ever-chan
...
AI systems are increasingly being used to support human decision making. It is important that AI advice is followed appropriately. However, according to existing literature, users typically under-rely or over-rely on AI systems, and this leads to sub-optimal team performance. In
...
Quality control is an, if not the, essential challenge in crowdsourcing. Unsatisfactory responses from crowd workers have been found to particularly result from ambiguous and incomplete task descriptions, often from inexperienced task requesters. However, creating clear task desc
...
Gender Choices of Conversational Agent
How Today’s Practice Can Shape Tomorrow’s Values
When creating conversational agents, designers have to make decisions about the way the agents present themselves. In this position paper, we identify and synthesize ethical dilemmas that conversational interface designers and researchers face around gender of conversational agen
...
A Missing Piece in the Puzzle
Considering the Role of Task Complexity in Human-AI Decision Making
Recent advances in the performance of machine learning algorithms have led to the adoption of AI models in decision making contexts across various domains such as healthcare, finance, and education.Different research communities have attempted to optimize and evaluate human-AI te
...
What You Show is What You Get!
Gestures for Microtask Crowdsourcing
Crowdsourcing is a valuable tool to gather human input which enables the development of reliable artificial intelligence systems. Microtask platforms like Prolific and Amazon's Mechanical Turk have flourished by creating environments where crowd workers can provide such human inp
...
Time constraints are commonplace in our daily lives. While literature in recent years from the Information Retrieval (IR) community has increased our understanding of the effects of time constraints on search, practical effects on search outcomes have rarely been evaluated. Littl
...