Evaluation of feedback generated from agent-based social skills training systems

A qualitative analysis on the comprehensibility, usability, and improvement points of the generated feedback

Bachelor thesis (2023)

Authors

N. Ntasi Electrical Engineering, Mathematics and Computer Science

Contributors

W.P. Brinkman Interactive Intelligence - EEMCS (mentor)
M. Al Owayyed Interactive Intelligence - EEMCS (mentor)
E. Eisemann Computer Graphics and Visualisation - EEMCS (graduation committee member)
Faculty
Electrical Engineering, Mathematics and Computer Science, Electrical Engineering, Mathematics and Computer Science
Copyright: © 2023 Nikola Ntasi
More Info
expand_more

Published Date

03-07-2023

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

Chatbots are tools that can potentially be utilized in chat-based child helpline training. In this type of training, the quality of the feedback received is of vital importance. This paper aims to analyze the automated feedback generated by such a chatbot. The domains analyzed include user comprehension, usefulness, and potential improvement points. In a user study, a formative assessment and two interviews were conducted for each domain, respectively.
For comprehensibility, all participants could easily understand the feedback report. They found that the bot could be easier to work around after reading the feedback, with the table being of much guidance. They found the transcript to be a welcome addition, but missing constructive feedback. Regarding improvement points, two of them were tightly related to the limitations of the chatbot, rather than the report itself. Extra guidance and instructions were deemed necessary by the participants, alongside an easier-to-read transcript interface.

Files

Final_paper.pdf
(pdf | 1.18 Mb)