Abstract— Objective: The objective of this exploratory study is to investigate how AI speech and text technologies, specifically Whisper and ChatGPT-4, can help reduce the administrative burden in occupational health consultations, with a focus on accuracy, efficiency, and user satisfaction.
Methods: A quantitative research approach was employed, utilizing a controlled trial design. Fourteen occupational health doctors participated in simulations using Whisper for transcription and customized ChatGPT-4 modules for generating medical summaries (Medical Summarizer), letters (Letter Generator), and documents (Document Generator), and providing medical protocol-based replies on participants’ questions (Medical Protocol Assistant). The accuracy, efficiency, and user satisfaction of these technologies were compared against traditional administrative methods, with descriptive statistics and paired samples t-tests conducted to assess performance differences.
Findings: The study revealed that Whisper transcriptions had a word error rate (WER) of 13.1%, with an average transcription time of 18.4 minutes, which was 6.6 minutes faster
than human transcription. ChatGPT-4's Medical Summarizer generated summaries 29 times faster than human participants, with an average generation time of 0.5 minutes but had a 38.6%
error rate in element generation. The Letter Generator and Document Generator exhibited error rates of 90.4% and 17.5%, respectively, although both were significantly more efficient
than manual processes, with average generation times of 0.4 and 3.9 minutes, respectively. The Medical Protocol Assistant provided protocol-based replies with an 86.7% accuracy, achieving the highest user satisfaction score (4.4 out of 5) among all modules.
Conclusion: AI speech and text technologies show potential in reducing administrative tasks in occupational health settings, particularly in terms of efficiency. However, the moderate accuracy and varying satisfaction rates indicate that further refinement is necessary to enhance their applicability in clinical practice. Future research should focus on improving accuracy, evaluation of the technologies in actual patient-physician consultations and developing robust privacy safeguards.