Crowdsourcing is a valuable tool to gather human input which enables the development of reliable artificial intelligence systems. Microtask platforms like Prolific and Amazon's Mechanical Turk have flourished by creating environments where crowd workers can provide such human input in a diverse and representative manner. Such marketplaces have evolved to support several hundreds of workers in earning their primary livelihood through crowd work. Crowd workers, however, often perform these tasks in sub-optimal work environments with poor ergonomics. Additionally, many of the various microtasks require input via the standard method of a mouse and keyboard and are repetitive in nature. As such, crowd workers who primarily earn their livelihoods in microtask marketplaces are at risk of injuries such as carpal tunnel syndrome. By changing the input modality from a mouse and keyboard to gesture-driven input, crowd workers can complete their work while simultaneously improving or safeguarding their physical health. Through three distinct microtasks, we constructed a dataset that enables the exploration of the physical and mental health of crowd workers while using gestures. In this work, we present the process of constructing this dataset, how we applied it, and the future applications we foresee.

Access to commonsense knowledge is receiving renewed interest for developing neuro-symbolic AI systems, or debugging deep learning models. Little is currently understood about the types of knowledge that can be gathered using existing knowledge elicitation methods. Moreover, these methods fall short of meeting the evolving requirements of several downstream AI tasks. To this end, collecting broad and tacit knowledge, in addition to negative or discriminative knowledge can be highly useful. Addressing this research gap, we developed a novel game with a purpose, 'FindItOut', to elicit different types of knowledge from human players through easily configurable game mechanics. We recruited 125 players from a crowdsourcing platform, who played 2430 rounds, resulting in the creation of more than 150k tuples of knowledge. Through an extensive evaluation of these tuples, we show that FindItOut can successfully result in the creation of plural knowledge with a good player experience. We evaluate the efficiency of the game (over 10 × higher than a reference baseline) and the usefulness of the resulting knowledge, through the lens of two downstream tasks - commonsense question answering and the identification of discriminative attributes. Finally, we present a rigorous qualitative analysis of the tuples' characteristics, that informs the future use of FindItOut across various researcher and practitioner communities.

Human input is pivotal in building reliable and robust artificial intelligence systems. By providing a means to gather diverse, high-quality, representative, and cost-effective human in put on demand, micro task crowdsourcing marketplace shave thrived. Despite the unmistakable benefits available from online crowd work, the lack of health provisions and safeguards, along with existing work practices threatens the sustainability of this paradigm. Prior work has investigated worker engagement and mental health, yet no such investigations into the effects of crowd work on the physical health of workers have been undertaken. Crowd workers complete their work in various sub-optimal work environments, often using a conventional input modality of a mouse and keyboard. The repetitive nature of micro task crowdsourcing can lead to stress-related injuries, such as the well-documented carpal tunnel syndrome. It is known that stretching exercise scan help reduce injuries and discomfort in office workers. Gestures, the act of using the body intentionally to affect the behavior of an intelligent system, can serve as both stretches and an alternative form of input for micro tasks. To better understand the usefulness of the dual-purpose in put modality of ergonomically-informed gestures across different crowd sourced micro tasks, we carried out a controlled 2 × 3 between-subjects study (N=294). Considering the potential benefits of gestures as an input modality, our results suggesta real trade-off between worker accuracy in exchange for potential short to long-term health benefits.