From sight to insight: Eye contact and eye-tracking in the driver-pedestrian context

Abstract

A large number of traffic accidents occur worldwide each year, of which a sizable portion involve pedestrians, making them a vulnerable group on the road. Many of these accidents occur due to visual distraction, meaning drivers and pedestrians fail to look where they should be looking. In addition to this tendency for distraction, the eyes are a means of exchanging information between road users, via behaviors such as eye contact. However, the role and importance of eye contact in traffic in connection with traffic safety and the decisions of road users is not yet entirely clear. Further, with the advent of automated vehicles, the role of eye contact in traffic may change or disappear altogether, due to the absence of drivers. One promising way to shed light on this matter is to use eye-tracking, a technology which can measure the eye movements of road users, and which might allow the engineering of solutions to mitigate the frequency and severity of accidents.

This dissertation aims to investigate the role of eye contact between drivers and pedestrians, as well as its influence on pedestrians’ road crossing intentions. Another aim of this dissertation is to assess the accuracy of eye-tracking devices and to objectively detect and operationalize driver-pedestrian eye contact using eye-tracking. Finally, this thesis aims to develop safety systems based on eye-tracking that can automatically analyze and contextualize gaze in traffic and warn vulnerable road users of danger. This thesis consists of four independently readable and empirical research papers.

The first study examines the effect of drivers’ eye contact on pedestrians’ crossing decisions using an online crowdsourced experiment. It shows that, although a car’s kinematics have a dominant effect, a driver’s eye contact also makes pedestrians feel safer and more likely to cross the road, and that the timing of the driver’s eye contact has an influence as well. The second study benchmarks the accuracies of mobile eye-trackers under static and dynamic conditions, finding that eccentricity worsens accuracy, but dynamicity does not necessarily worsen it. The third study presents a method to objectively detect and operationalize driver-pedestrian eye contact using two synchronized eye-trackers and computer vision, defining eye contact as mutual gaze within a 4° threshold. The fourth study explores the integration of mobile eye-tracking, object detection, and a vision-language model in an attempt to develop a real-time, context-aware safety system that can assess risk in traffic and enhance the situational awareness of road users.

This dissertation concludes that while eye contact is neither as powerful a cue as kinematics nor essential for crossing, it is still a “should-have” in driver-pedestrian interactions as it can increase perceived safety and willingness to cross. This thesis also concludes that certain types of external Human Machine Interfaces (eHMIs) – substitutes for the missing eye contact between pedestrians and automated vehicles – would be beneficial to maintain existing levels of comfort in interactions. Finally, this thesis also highlights the potential of using mobile eye-tracking in combination with computer vision and AI for applications in the traffic, manufacturing, medical, education, and other domains, and recommends topics for further research into eye contact and eye-tracking.