Accuracy of textual interfaces using comparative questions to elicit personal value-related information from the users for building responsible AI

Abstract

Individuals seeking a healthier lifestyle can benefit from behavior support agents. Customization and transparency are crucial for system effectiveness. This paper proposes using behavior trees as a user model, with a conversational agent extracting necessary information. The conversational interface enhances transparency, allowing users to understand how the system perceives them. Understanding comparative questions is vital to this approach's success. The objective is to investigate modeling personal values accurately using a conversational agent. Technologically literate participants engaged in iterative dialogue to elicit a personalized user model. Scenarios explored contextual factors' impact on value alignment. Results revealed decreased accuracy when more values were affected by contextual factors. Comparative questions were less effective than isolated questioning. System usability was rated poor but approaching acceptability. Larger sample sizes are needed for comprehensive conclusions. This research lays the foundation for conversational agents that model personal values within behavior trees, advancing behavior support systems.