Print Email Facebook Twitter Using ChatGPT for human–computer interaction research: a primer Title Using ChatGPT for human–computer interaction research: a primer Author Tabone, W. (TU Delft Human-Robot Interaction) de Winter, J.C.F. (TU Delft Human-Robot Interaction) Date 2023 Abstract ChatGPT could serve as a tool for text analysis within the field of Human–Computer Interaction, though its validity requires investigation. This study applied ChatGPT to: (1) textbox questionnaire responses on nine augmented-reality interfaces, (2) interview data from participants who experienced these interfaces in a virtual simulator, and (3) transcribed think-aloud data of participants who viewed a real painting and its replica. Using a hierarchical approach, ChatGPT produced scores or summaries of text batches, which were then aggregated. Results showed that (1) ChatGPT generated sentiment scores of the interfaces that correlated extremely strongly (r > 0.99) with human rating scale outcomes and with a rule-based sentiment analysis method (criterion validity). Additionally, (2) by inputting automatically transcribed interviews to ChatGPT, it provided meaningful meta-summaries of the qualities of the interfaces (face validity). One meta-summary analysed in depth was found to have substantial but imperfect overlap with a content analysis conducted by an independent researcher (criterion validity). Finally, (3) ChatGPT's summary of the think-aloud data highlighted subtle differences between the real painting and the replica (face validity), a distinction corresponding with a keyword analysis (criterion validity). In conclusion, our research indicates that, with appropriate precautions, ChatGPT can be used as a valid tool for analysing text data. Subject application programming interface (API)reproducibilityprompt engineeringhuman-subject research To reference this document use: http://resolver.tudelft.nl/uuid:72baedb9-6a4d-48c8-95b7-10225db441ca DOI https://doi.org/10.1098/rsos.231053 ISSN 2054-5703 Source Royal Society Open Science, 10 (9) Part of collection Institutional Repository Document type journal article Rights © 2023 W. Tabone, J.C.F. de Winter Files PDF tabone_de_winter_2023_usi ... primer.pdf 1.71 MB Close viewer /islandora/object/uuid:72baedb9-6a4d-48c8-95b7-10225db441ca/datastream/OBJ/view