Cars are increasingly capable of providing drivers with warnings and advice. However, whether drivers should be provided with ipsilateral warnings (signaling the direction to steer towards) or contralateral warnings (signaling the direction to avoid) is inconclusive. Furthermore, how auditory warnings and visual information from the driving environment together contribute to drivers’ responses is relatively unexplored. In this study, 34 participants were presented with animated video clips of traffic situations on a three-lane road, while their eye movements were recorded with an eye-tracker. The videos ended with a near collision in front after 1, 3, or 6 s, while either the left or the right lane was safe to swerve into. Participants were instructed to make safe lane-change decisions by pressing the left or right arrow key. Upon the start of each video, participants heard a warning: Go Left/Right (ipsilateral), Danger Left/Right (contralateral), and nondirectional beeps (Baseline), emitted from the spatially corresponding left and right speakers. The results showed no significant differences in response times and accuracy between ipsilateral and contralateral warnings, although participants rated ipsilateral warnings as more satisfactory. Ipsilateral and contralateral warnings both improved response times in situations in which the left/right hazard was not yet manifest or was poorly visible. Participants fixated on salient and relevant vehicles as quickly as 220 ms after the trial started, with no significant differences between the audio types. In conclusion, directional warnings can aid in making a correct left/right evasive decision while not affecting the visual attention distribution.

@en

A major question in human-automation interaction is whether tasks should be traded or shared between human and automation. This work presents reflections—which have evolved through classroom debates between the authors over the past 10 years—on these two forms of human-automation interaction, with a focus on the automated driving domain. As in the lectures, we start with a historically informed survey of six pitfalls of automation: (1) Loss of situation and mode awareness, (2) Deskilling, (3) Unbalanced mental workload, (4) Behavioural adaptation, (5) Misuse, and (6) Disuse. Next, one of the authors explains why he believes that haptic shared control may remedy the pitfalls. Next, another author rebuts these arguments, arguing that traded control is the most promising way to improve road safety. This article ends with a common ground, explaining that shared and traded control outperform each other at medium and low environmental complexity, respectively. Practitioner summary: Designers of automation systems will have to consider whether humans and automation should perform tasks alternately or simultaneously. The present article provides an in-depth reflection on this dilemma, which may prove insightful and help guide design. Abbreviations: ACC: Adaptive Cruise Control: A system that can automatically maintain a safe distance from the vehicle in front; AEB: Advanced Emergency Braking (also known as Autonomous Emergency Braking): A system that automatically brakes to a full stop in an emergency situation; AES: Automated Evasive Steering: A system that automatically steers the car back into safety in an emergency situation; ISA: Intelligent Speed Adaptation: A system that can limit engine power automatically so that the driving speed does not exceed a safe or allowed speed.

@en

Quantifying drivers’ perceived risk is important in the design and evaluation of the behaviour of automated vehicles (AVs) and in predicting takeovers by the driver. A ‘Driver's Risk Field’ (DRF) function has been previously shown to be able to predict manual driving behaviour in several simulated scenarios. In this paper, we tested if the DRF-based risk estimate (rˆ) could predict manual driving behaviour and the driver's perceived risk during automated driving. To ensure that the participants perceived realistic levels of risk, the experiment was conducted in a test vehicle. Eight participants drove five laps manually and experienced 12 different laps of automated driving on a test track. The test track consisted of three sections (which were sub-divided into 12 sectors): curve driving (9 sectors), parked car (1 sector), and 90-degree intersections (2 sectors). If the driver verbally expressed risk or performed a takeover, that particular sector was labelled as risky. The results show that the DRF risk estimate (rˆ) predicted manual driving behaviour (ρ_steering=0.69, ρ_speed=0.64), as well as correlated with the driver's perceived risk in curve driving (r² = 0.98) and while negotiating a car parked outside the lane boundary (r²=0.59). In conclusion, the DRF-based risk estimate (rˆ) is predictive of manual driving behaviour and perceived risk in automated driving. Future research should include tactical and strategic components to the driving task.

@en

Much psychological research uses pupil diameter measurements to investigate the cognitive and emotional effects of visual stimuli. A potential problem is that accommodating at a nearby point causes the pupil to constrict. This study examined to what extent accommodation is a confounder in pupillometry research. Participants solved multiplication problems at different distances (Experiment 1) and looked at line drawings with different monocular depth cues (Experiment 2) while their pupil diameter, refraction, and vergence angle were recorded using a photorefractor. Experiment 1 showed that the pupils dilated while performing the multiplications, for all presentation distances. Pupillary constriction due to accommodation was not strong enough to override pupil dilation due to cognitive load. Experiment 2 showed that monocular depth cues caused a small shift in refraction in the expected direction. We conclude that, for the young student sample we used, pupil diameter measurements are not substantially affected by accommodation.

@en

Several papers by Eckhard Hess from the 1960s and 1970s report that the pupils dilate or constrict according to the interest value, arousing content, or mental demands of visual stimuli. However, Hess mostly used small sample sizes and undocumented luminance control. In a first experiment (N = 182) and a second preregistered experiment (N = 147), we replicated five studies of Hess using modern equipment. Our experiments (1) did not support the hypothesis of gender differences in pupil diameter change with respect to baseline (PC) when viewing stimuli of different interest value, (2) showed that solving more difficult multiplications yields a larger PC in the seconds before providing an answer and a larger maximum PC, but a smaller PC at a fixed time after the onset of the multiplication, (3) did not support the hypothesis that participants’ PC mimics the pupil diameter in a pair of schematic eyes but not in single-eyed or three-eyed stimuli, (4) did not support the hypothesis of gender differences in PC when watching a video of a male trying to escape a mob, and (5) supported the hypothesis that arousing words yield a higher PC than non-arousing words. Although we did not observe consistent gender differences in PC, additional analyses showed gender differences in eye movements towards erogenous zones. Furthermore, PC strongly correlated with the luminance of the locations where participants looked. Overall, our replications confirm Hess's findings that pupils dilate in response to mental demands and stimuli of an arousing nature. Hess's hypotheses regarding pupil mimicry and gender differences in pupil dilation did not replicate.

@en

The arrival of highly automated vehicles introduces a new interaction between the vehicle and driver. System limitations during highly automated driving require the driver to be ready to take back control at request. Previous studies on the take-over process concluded that the driver requires a transition period to stabilize vehicle control after resuming manual control. These studies used traded control to instantaneously transfer control back to the driver, causing an abrupt switch in control authority. This study explores Haptic Shared Control as a potential approach to mitigate these stabilization issues, and assist the driver to make a lane change. We expected that Haptic Shared Control improves the take-over performance compared to the traded control approach. A total of 30 participants drove two trials in a driving simulator, one for each transition approach. Each trial consisted of 10 takeover scenarios, where driver had to avoid a stationary car in the lane ahead. The take-over scenarios had either a time-to-collision (TTC) of 5 or 7 seconds. During autonomous driving the participants were engaged in a non-driving related task. The take-over performance was assessed based on safety margins, steering wheel input, and subjective measures. Results showed that with the HSC approach drivers adopted larger safety margins in longitudinal direction, but smaller margins in lateral direction. The HSC approach yielded lower steering wheel velocities. The subjective measures did not yield any significant differences. These results suggests that haptic shared control can assist the driver in stabilizing lateral vehicle control after resuming manual control. On the other hand, HSC seems to slightly hinder the driver to execute her/his preferred trajectory. Future research should focus on designing an adaptable human compatible reference in order to mitigate conflicts during take-over scenarios.

@en

Objective: This study aims to compare the effectiveness and subjective acceptance of three designs for haptic lane-keeping assistance in truck driving. Background: Haptic lane-keeping assistance provides steering torques toward a reference trajectory, either continuously or only when exceeding a bandwidth. These approaches have been previously investigated in driving simulators, but it is unclear how these generalize toward real-life truck driving. Method: Three haptic lane-keeping algorithms to assist truck drivers were evaluated on a 6.3-km-long oval-shaped test track: (1) a single-bandwidth (SB) algorithm, which activated assistance torques when the predicted lateral deviation from lane center exceeded 0.4 m; (2) a double-bandwidth (DB) algorithm, which activated as SB, but deactivated after returning within 0.15 m lateral deviation; and (3) an algorithm providing assistance torques continuously (Cont) toward the lane center. Fifteen participants drove four trials each, one trial without and one for each haptic assistance design. Furthermore, participants drove with and without a concurrent visually distracting task. Results: Compared to unsupported driving, all three assistance systems provided similar safety benefits in terms of decreased absolute lateral position and number of lane departures. Participants reported higher satisfaction and usability for Cont compared to SB. Conclusion: The continuous assistance was better accepted than bandwidth assistance, a finding consistent with prior driving simulator research. Research is still needed to investigate the long-term effects of haptic assistance on reliance and after-effects. Application: The present results are useful for designers of haptic lane-keeping assistance, as driver acceptance and performance are determinants of reliance and safety, respectively.

@en

Im Zusammenspiel mit dem automatisierten Fahren muss das menschliche Fahrverhalten genau untersucht werden. Forscher der Technischen Universität Delft arbeiten mit dem Simulatorhersteller Cruden daran, dieses Verhalten bei Spurwechseln besser zu verstehen, mit dem Ziel, die Interaktion zwischen dem Fahrer und dem (teil-)automatisierten Fahrzeug zu verbessern.@en

This paper assessed four types of human–machine interfaces (HMIs), classified according to the stages of automation proposed by Parasuraman et al. [“A model for types and levels of human interaction with automation,” IEEE Trans. Syst. Man, Cybern. A, Syst. Humans, vol. 30, no. 3, pp. 286–297, May 2000]. We hypothesized that drivers would implement decisions (lane changing or braking) faster and more correctly when receiving support at a higher automation stage during transitions from conditionally automated driving to manual driving. In total, 25 participants with a mean age of 25.7 years (range 19–36 years) drove four trials in a driving simulator, experiencing four HMIs having the following different stages of automation: baseline (information acquisition—low), sphere (information acquisition—high), carpet (information analysis), and arrow (decision selection), presented as visual overlays on the surroundings. The HMIs provided information during two scenarios, namely a lane change and a braking scenario. Results showed that the HMIs did not significantly affect the drivers’ initial reaction to the take-over request. Improvements were found, however, in the decision-making process: When drivers experienced the carpet or arrow interface, an improvement in correct decisions (i.e., to brake or change lane) occurred. It is concluded that visual HMIs can assist drivers in making a correct braking or lane change maneuver in a take-over scenario. Future research could be directed toward misuse, disuse, errors of omission, and errors of commission.@en

Toward automated driving, the human driving behavior has to be studied exactly. Researchers in the section Human-Robot Interaction of the Delft University of Technology are working together with the driving simulator manufacturer Cruden to achieve an understanding of this behavior during lane changes, with the aim of improving the interaction between the driver and the (semi)-automated vehicle.@en

Traditional driver-automation interaction trades control over the vehicle back and forth between driver and automation. Haptic shared control offers an alternative by continuously sharing the control through torques on the steering wheel and pedals. When designing additional feedback torques, part of the design choice lies in the stiffness around the neutral steering point: also called the Level of Haptic Authority (LoHA), which is usually static and tuned to balance safety benefits (better at high LoHA) with conflicts torques in case of different intentions between automation and driver (higher conflict torques with increased LoHA). In this paper we explore the idea of situation-adaptive LoHA: in this case during lane-keeping by changing the LoHA based on time to lane crossing (TLC). Consequently, when safety margins are high (e.g., when driving on a wide road) the LoHA is low, but the LoHA would only increase when safety margins decrease. We propose two alternative design approaches to apply the LoHA: symmetrically and asymmetrically (i.e., only increase of LoHA in the direction of the low TLC). We compared these design in an explorative driving simulator study (n=14) to driving with two static LoHA designs (low and high). We found that compared to the high LoHA controller, both adaptive LoHA controllers designs resulted in similar safety margins, but at decreased conflict torques. Hence, a TLC-based adaptive LoHA controller seems to be an effective approach to mitigate conflicts while maintaining the safety benefits associated with HSC.@en

For automated vehicles (SAE Level 2-3) part of the challenge lies in communicating to the driver what control actions the automation is taking and will take, and what its capabilities are. A promising approach is haptic shared control (HSC), which uses continuous torques on the steering wheel to communicate the automation’s current control actions. However, torques on the steering wheel cannot communicate future spatiotemporal constraints, that might be required to judge appropriate overtaking or obstacle avoidance. A visualisation of predicted vehicle trajectory, along with velocity-dependent constraints with respect to achievable trajectories is proposed. The goal of this paper is to experimentally compare obstacle avoidance behaviour while driving with the designed visualisation against driving with a previously designed HSC, as well as the two support systems combined. It is expected that adding visual feedback improves obstacle avoidance and user acceptance, and reduces control effort with respect to HSC only. In a driving simulator experiment, 26 participants drove three trials with each feedback condition (visual, HSC, and combination) and had to avoid obstacles that appeared with a Time to collision of either 1.85 s (critical) or 4.7 s (non-criticall). Results showed that, compared to HSC only, the HSC and visual combination yielded slightly smaller safety margins to the obstacle, a significant reduction of control activity on straights, and increased subjective acceptance rating. Visual and HSC offered a beneficial synergy, as it seemed the visual feedback allowed drivers to anticipate the effect of their steering actions on the car’s trajectory more accurately, and the HSC reduced the intra-subject variability. Future research should investigate the effects of added visual feedback in more detail, specifically in terms of the effectiveness to communicate automation capabilities and driver gaze behavior.@en

Current haptic control systems provide feedback torques based on a lateral deviation with respect to a reference trajectory (i.e., centre of the lane), which do not capture the satisficing behaviour human beings typically adopt during a lane keeping task. As such, a novel time-to-lane-crossing-based controller is proposed, which is expected to provide more human-like guidance. The aim of this study is to describe a novel time-to-lane-crossing-based controller and investigate its potential as an alternative to previous reference-trajectory-based guidance. In a simulator study twenty-four participants drove three trials through a single-lane, 10.8 km long road (width: 3 m), receiving three types of guidance, namely 1) none (manual), 2) reference based controller, 3) TLC-based controller. Results showed that both the reference-based, as well as the TLC-based guidance provided significant safety benefits, in terms of more centred and less varying lane position, and higher safety margins. Moreover, no significant differences were revealed between the two guidance approaches. In conclusion, the TLC-based guidance is a potential alternative to reference trajectory-based guidance. Nevertheless, a more detailed analysis is warranted to investigate the two approaches in different driving conditions, like road width, straights, and curves.@en

Across the automotive industry, manufacturers have recently released various Partial Automation systems (SAE Level 2) which allow simultaneous/combined execution of both lateral and longitudinal vehicle control at the same time, yet still require active human supervision/engagement. Current reactive trends will be reviewed across major automotive players regarding differences in terminology, HMI input/outputs, and escalation intervals. Scholarly research is also reviewed pertaining to proactive strategies for driver engagement. Additionally, human factors research and findings will be presented regarding recommendations for situation awareness, human machine interfaces, TOR, as well as shared control concepts. The tutorial will conclude with discussion and brainstorming around outlook toward tele-operated remote driving services (Tele-Driving); what they have to offer beyond assisted/automated driving, autonomous vehicles, and ride-hailing/car-sharing paradigms; as well as the design/conduct of human factors research regarding Tele-Driving.

@en

An important research question in the domain of highly automated driving is how to aid drivers in transitions between manual and automated control. Until highly automated cars are available, knowledge on this topic has to be obtained via simulators and self-report questionnaires. Using crowdsourcing, we surveyed 1692 people on auditory, visual, and vibrotactile take-over requests (TORs) in highly automated driving. The survey presented recordings of auditory messages and illustrations of visual and vibrational messages in traffic scenarios of various urgency levels. Multimodal TORs were the most preferred option in high-urgency scenarios. Auditory TORs were the most preferred option in low-urgency scenarios and as a confirmation message that the system is ready to switch from manual to automated mode. For low-urgency scenarios, visual-only TORs were more preferred than vibration-only TORs. Beeps with shorter interpulse intervals were perceived as more urgent, with Stevens’ power law yielding an accurate fit to the data. Spoken messages were more accepted than abstract sounds, and the female voice was more preferred than the male voice. Preferences and perceived urgency ratings were similar in middle- and high-income countries. In summary, this international survey showed that people's preferences for TOR types in highly automated driving depend on the urgency of the situation.

@en

Conditionally automated driving systems may soon be available on the market. Even though these systems exempt drivers from the driving task for extended periods of time, drivers are expected to take back control when the automation issues a so-called take-over request. This study investigated the interaction between take-over request modality and type of non-driving task, regarding the driver's reaction time. It was hypothesized that reaction times are higher when the non-driving task and the take-over request use the same modality. For example, auditory take-over requests were expected to be relatively ineffective in situations in which the driver is making a phone call. 101 participants, divided into three groups, performed one of three non-driving tasks, namely reading (i.e., visual task), calling (auditory task), or watching a video (visual/auditory task). Results showed that auditory and tactile take-over requests yielded overall faster reactions than visual take-over requests. The expected interaction between takeover modality and the dominant modality of the non-driving task was not found. As for self-reported usefulness, auditory and tactile take-over requests yielded higher scores than visual ones. In conclusion, it seems that auditory and tactile stimuli are equally effective as take-over requests, regardless of the non-driving task. Further study into the effects of realistic non-driving tasks is needed to identify which non-driving tasks are detrimental to safety in automated driving.@en

The driver of a conditionally automated car is not required to permanently monitor the outside environment, but needs to take over control whenever the automation issues a “request to intervene” (i.e., take-over request). If the driver misses the take-over request or does not respond in a timely and correct manner, a take-over could result in a safety-critical scenario. Traditionally, warnings in vehicles are conveyed by visual and auditory displays, though recently it has been argued that vibrotactile stimuli could also be a viable approach to present a takeover request to the driver. In this paper, we present a vibrotactile seat designed to convey dynamic vibration patterns to the driver. The seat incorporates 48 vibration motors (eccentric mass rotation) that can be individually controlled. One 6 × 4 matrix, with an average inter-motor distance of approximately 4 cm, is located in the seat back and one in the seat bottom. The DC-voltage to the motors is controlled by three Pulse Width Modulation (PWM) drivers, which in turn are controlled by an Arduino microcontroller. A study with 12 participants was conducted to investigate (1) at which vibration intensity participants find a vibratory stimulus annoying and whether this threshold changes over time, (2) how well participants are able to discriminate vibratory stimuli as a function of spatial separation, and (3) which of six dynamic vibration patterns are regarded as most satisfying. Results showed that participants’ annoyance threshold reduced when they were repeatedly exposed to vibrotactile stimuli. Second, the percentage of correct responses in the two-point discrimination test increased significantly with increasing inter-stimuli distance (i.e., from 4 to 20 cm). Third, participants seemed to be more satisfied when more motors were activated simultaneously (i.e., more spatial overlap). Overall, the results suggest that participants are well able to perceive vibrotractile stimuli in the driver seat. However, the results suggest that repetitive exposure to vibrotactile stimuli may evoke annoyance, a finding that should be taken into account in future designs of vibrotactile displays. Future studies should investigate the possibility to convey complex messages via the vibration seat.

@en

Vibrotactile stimuli can be effective as warning signals, but their effectiveness as directional take-over requests in automated driving is yet unknown. This study aimed to investigate the correct response rate, reaction times, and eye and head orientation for static versus dynamic directional take-over requests presented via vibrating motors in the driver seat. In a driving simulator, eighteen participants performed three sessions: 1) a session involving no driving (Baseline), 2) driving a highly automated car without additional task (HAD), and 3) driving a highly automated car while performing a mentally demanding task (N-Back). Per session, participants received four directional static (in the left or right part of the seat) and four dynamic (moving from one side towards the opposite left or right of the seat) take-over requests via two 6 × 4 motor matrices embedded in the seat back and bottom. In the Baseline condition, participants reported whether the cue was left or right, and in the HAD and N-Back conditions participants had to change lanes to the left or to the right according to the directional cue. The correct response rate was operationalized as the accuracy of the self-reported direction (Baseline session) and the accuracy of the lane change direction (HAD & N-Back sessions). The results showed that the correct response rate ranged between 94% for static patterns in the Baseline session and 74% for dynamic patterns in the N-Back session, although these effects were not statistically significant. Steering wheel touch and steering input reaction times were approximately 200 ms faster for static patterns than for dynamic ones. Eye tracking results revealed a correspondence between head/eye-gaze direction and lane change direction, and showed that head and eye-gaze movements where initiated faster for static vibrations than for dynamic ones. In conclusion, vibrotactile stimuli presented via the driver seat are effective as warnings, but their effectiveness as directional take-over requests may be limited. The present study may encourage further investigation into how to get drivers safely back into the loop.

@en

This paper summarizes our results from survey research and driving simulator experiments on auditory, vibrotactile, and visual take-over requests in highly automated driving. Our review shows that vibrotactile takeover requests in the driver’s seat yielded relatively high ratings of self-reported usefulness and satisfaction. Auditory take-over requests in the form of beeping sound were regarded as useful but not satisfactory, and it was found that the beep rate corresponds to perceived urgency. Visual-only feedback (LEDs) was regarded by participants as neither useful nor satisfactory. Augmented visual feedback was found to support effective steering and braking actions, and may be a useful compliment to vibrotactile take-over requests. The present findings may be used in the design of take-over requests.@en

Highly automated driving can potentially provide enormous benefits to society. However, it is unclear what types of interfaces should be used for takeover requests during highly automated driving, in which a driver is asked to switch back to manual driving. In this paper, a proposal for a driving simulator study on the use of six auditory signals during such takeover requests is outlined. The auditory signals to be tested in the experiment are based on the results of an online international survey previously conducted by the authors. The experiment will involve 24 participants performing a secondary task, and the takeover scenario will be represented by an accident in the middle lane of a three-lane freeway. The time margin prior to takeover will be 7 s. The driving time between subsequent takeover requests will be 2 to 3 min. The application of the results of the proposed study as well as plans for future studies are presented in the last section.@en