Describing Images to Visually Impaired Users: a Requirement Elicitation Approach

Abstract

Visually impaired people should enjoy the same rights to acquire information as people with normal sight. Since visual contents become more and more pervasive in our daily life, image description becomes increasingly important to help visually impaired people to get equal access to the information contained in visual contents. However, how to produce image descriptions in a scalable and reliable way is still an unsolved problem. Therefore, researches on the requirements of image description from the perspective of visually impaired people are essential to approaching this problem. Based on a review of existing study results on this topic, this thesis investigates the possibilities of utilizing interactive image description as an approach to collect visually impaired people’s requirements on image description and the benefits of integrating interactive image description to the current image description production system. The existing one-shot statistical description requires descriptors to evaluate the importance of the image, make choices on what should be described, and organize the content so that necessary information can be effectively conveyed. Through literature review, it is found that the requirements of image description are highly context-dependent and influenced by plentiful factors. Therefore, existing guidelines are usually vague and require the describer to rely on experience and intuition while making a lot of subjective judgments, which increases the threshold for generating high-quality image descriptions. On the other hand, through field research and literature research it is found that VIPs hope to have more control over the presentation of image description (both its presence and content). Early explorations of interactive image description showed the possibility of this affordance. Since users are allowed to decide the description content actively, it is argued that the user's preference for image description can be collected through interactive image description. A design goal is proposed accordingly. A prototype is developed to verify this proposal. Through a comparative experiment, the systems’ function to collect user preferences and gradually improve the content of image description is confirmed. In addition, the qualitative research results also reveal the mental activities when users interacting with image description and the impact of interactive image description in this procedure, which is summarized as an image perception model. It is also argued that structured description and progressive description provide new perspectives to reduce the workload of describing images. A final design was developed as the demonstrator for the research findings and proposals.