As vehicles transition between driving automation levels, drivers need to be continually aware of the automation mode and the resulting driver responsibilities. This study investigates the impact of visual user interfaces (UIs) on drivers’ mode awareness in SAE Level 2 automated vehicles. It focuses on their understanding of speed and distance control, steering control, and the hands-on steering wheel requirement presented through UIs. Forty-five UIs were generated, presenting the activation of Lane Keeping Assist (LKA) and Adaptive Cruise Control (ACC) and the hands-on steering wheel requirement. Through an online questionnaire with 1080 respondents with experience of SAE Level 2, the study evaluated how these visual UIs influenced users’ understanding of control responsibilities, information usability, and trust in automated vehicles. The results show a limited role of UI in shaping users’ understanding of control. ACC UIs and LKA UIs had no significant effects, and apparently, the understanding of speed and distance control and steering control was independent of the ACC UI and LKA UI. A large variance in responses regarding the understanding of steering control and speed and distance control indicates confusion caused by mode ambiguity, suggesting that drivers do not well understand how the speed and distance control and steering control task is shared between the driver and the automation. However, the hands-on steering wheel UIs significantly improved the understanding of the hands-on steering wheel requirement. The hands-on steering wheel UI combining the hands on the wheel icon and the text “Keep hands on steering wheel” yielded 94.4% correct understanding and outperformed the UI with hands but without text (87.8% correct) or no UI (82.5% correct). In addition, the variation of visual UI did not affect trust. This study contributes to the understanding and design of visual UIs for effective communication of driver responsibilities in automated vehicles.

@en

As the development of Generative AI technology continues to progress, the opportunity for innovation with AI in the form of user interfaces, products and services within vehicles is expanding. Furthermore, automobiles are undergoing major transformations in design due to changes in the underlying technology resulting in evolved user needs, behaviors, activities and aspirations. This workshop is aimed at providing the participants hands-on experience of designing novel Generative AI interfaces for vehicles. While working on the design challenge as the connecting thread, we will introduce and weave together modules of knowledge domains focusing on Human-centered design, Ethical and Responsible behavior, and Autonomy in vehicles. Participants will learn about and engage collaboratively in employing design methods such as Co-creation using Activity Canvases, Enactment, Wizard of Oz, Bodystorming and inter-group discussion. As the outcome, we aim to publish participant’s design concepts as a booklet and a research paper, and seek new research collaborations.@en

Trust and perceived safety are pivotal in the acceptance of automated vehicles and can be enhanced by providing users with automation information on the (safe) operation of the vehicle. This study aims to identify how user interfaces (UI) can enhance drivers' trust and acceptance and reduce perceived risk in partially automated vehicles. Four interfaces were designed with different levels of complexity. These levels were achieved by combining automation information (surrounding information vs surrounding and manoeuvre information) and modality (visual vs visual and auditory). These interfaces were evaluated in a driving simulator in which a partially automated vehicle reacted to an event of a merging and braking vehicle in its front. The criticality of the events was manipulated by the factors merging gap (in meters) and deceleration (m/s²) of the vehicle in front. The reaction of the automation was either to brake or to change lanes. The results show that an optimal combination of automation information and modality enhances drivers' trust and acceptance. More specifically, the most advanced UI, which provided surrounding and manoeuvre information via the visual and auditory modalities, was associated with the highest trust and acceptance ranking and the lowest perceived risk. Manoeuvre information delivered through the auditory modality was particularly effective in enhancing trust and acceptance. The benefits of the UIs were consistent over events. However, in the most critical events, drivers did not feel entirely safe and did not trust the automation completely. This study suggests that the design of UIs for partially automated vehicles shall include automation information via visual and auditory modalities.

@en

The emergence of automated vehicles promises a revolution in urban mobility. To benefit from a new mobility system, women who have specific mobility considerations necessitate inclusion in designing automated vehicles. This study explores women's perspectives and the potential impact of automated vehicles through focus group discussions and in-depth interviews. Results demonstrate concerns among women about safety in current mobility systems, vulnerabilities regarding personal safety, and stereotypes about female drivers. Additionally, mothers face additional challenges managing items for children and their demands during travel, and senior women consider safety issues and declining capabilities when contemplating driving cessation. Current experience with mobility is reflected in concerns and visions regarding automated vehicles. The absence of a driver is expressed as improved safety in driverless taxis, while it is perceived as a safety concern in automated public transportation. Mothers with children anticipate convenience in travel, whereas senior women expect enhanced mobility and social participation. These findings underscore the importance of safety in women's mobility experiences and provide insights into addressing safety and interaction issues in the design of automated vehicles. Researchers, transportation authorities, and vehicle manufacturers can leverage these results to understand women's needs better and consider them in future designs and policy developments for automated vehicles. Prioritising women's perspectives in automated vehicle research is essential to realising the innovative potential of this technology and fostering a more inclusive and accessible future in urban mobility.

@en

Automated vehicles are anticipated to not only improve safety and comfort but also redefine the nature of driver-vehicle interactions. With these advancements, drivers in automated vehicles share the monitoring and supervising role with the system, creating a unique human-machine team. In this relation, beyond technical excellence, it is important for drivers to comprehend and trust the system for its effective and safe utilisation. Trust evolves during interactions, influenced by the system’s performance, emphasising the significance of provided information and interaction format. Design features of driving automation, particularly user interfaces, impact trust and shape perceptions of system performance, forming a crucial element for human-machine interaction.

This dissertation addresses three critical research gaps. First, an understanding of driver-vehicle interaction is crucial for designing effective interfaces. Current studies lack a clear comprehension of interactions during driving and transitions in complex systems, emphasising the need for an integrated view that considers various contexts and levels of driving automation. In this dissertation, I address this gap by investigating how drivers perceive complex interactions and identifying necessary interactions for drivers. Second, interactions extend beyond isolated take-over events, forming a sequence of interconnected behaviours that shape the overall driving experience. While take-over situations are undeniably critical, they represent just one facet of a broader continuum of scenarios. In-depth research on interactions during automated driving, where the human driver transitions between passive monitoring and active engagement, is lacking. Therefore, I explore the interactions in various situations, including automated driving and mode transitions in automated vehicles. Third, my research emphasises the design phase of user interfaces, beyond the conventional focus on identifying the effects of interfaces on drivers in specific scenarios. It stresses the importance of considering interface design comprehensively, incorporating factors such as cognitive load, situational awareness, and driver experience. Specifically, I look into the design of soundscapes to guide the driver back to control in take-over situations, creating a novel transition experience while prioritising safety.

Aligned with the goal of designing human-machine interaction for trustworthy automated vehicles, the research objectives are delineated into two objectives and six studies. The first objective aims to understand the driver-automated vehicle interaction, focusing on identifying the effects of interfaces, investigating human-machine interaction, and understanding drivers’ mode transition logic. The second objective aims to contribute to the development of interaction design guidelines, designing and evaluating user interfaces and developing an approach to soundscape design in automated vehicles. Finally, I consolidate the findings, discuss the research’s contribution, and provide an outlook on future work in the evolving landscape of automated vehicle technology.

Six studies employing diverse methods yield following results. A literature review and an on-road study unveil that the interaction between the driver and the automated vehicle through user interfaces significantly influences drivers’ performance and trust. Specifically, the literature review indicates that this interaction during automated driving has an impact extending from take-over situations to overall performance. The on-road study, using a vehicle with multiple levels of driving automation, reveals mode confusion related to mode transitions. Further exploration of driver’s understanding of mode transition logic, through an online survey demonstrates that, despite the driver performing certain interventions related to driving functions, there is no dominant mental representation of mode-transition logic under specific scenarios. Simulator experiments in partial and conditional automated driving scenarios illustrate that providing automation information during driving enhances the driver’s trust and acceptance. In particular, delivering the manoeuvre of driving automation to the driver through auditory modality is effective in enhancing trust and acceptance. The proposed soundscape design for take-over requests and spatial sound delivering manoeuvre information while automated driving showcases the potential of sound design to elevate the driver’s experience beyond a simple beep.

Throughout this dissertation, I investigate interactions in automated vehicles, addressing interaction challenges and designing and evaluating user interfaces. By conducting a literature review, on-road observations, interviews, online surveys, and simulator experiments, it navigates the complexities of trust, acceptance, and overall user experience. The contributions extend to understanding driver trust, performance, and the impact of system complexities on human-machine interaction, enriching the field with empirical evidence and practical guidelines for interaction design. @en

In conditionally automated driving (SAE level 3), drivers may take their eyes off the road but will still need to be ready to take control and will, therefore, benefit from information on automation. This study aims to investigate the effectiveness of automation manoeuvre information provided through spatial sound, traditional notification sound (beep), and a visual interface. Spatial sounds were designed differentiating four distinct driving manoeuvres: overtaking a leading car, slowing down, turning right, and passing a roundabout. The notification sound consisted of one beep being identical for all manoeuvres. The visual interface showed the automation mode with an image and manoeuvre information with text and images. The impact of these interfaces on trust, workload, acceptance, situation awareness, and sense of control was evaluated with questionnaires and visual attention was evaluated with eye tracking while participants engaged in a visual-motor secondary task in a driving simulator. The results indicate that, with all interfaces tested, manoeuvre information enhances trust, acceptance, situation awareness, and sense of control, without significantly affecting the overall workload. These benefits were more profound, adding auditory information and differed marginally between the traditional notification and the spatial sound, as the effectiveness of the different auditory interface types varied depending on the specific manoeuvre. Findings highlight the importance of designing user interfaces for automation manoeuvre information using auditory cues to improve the user experience in automated driving.@en

Mode awareness is important for the safe use of automated vehicles, yet drivers' understanding of mode transitions has not been sufficiently investigated. In this study, we administered an online survey to 838 respondents to examine their understanding of control responsibilities in partial and conditional driving automation with four types of interventions (brake pedal, steering wheel, gas pedal, and take-over request). Results show that most drivers understand that they are responsible for speed and distance control after brake pedal interventions and steering control after steering wheel interventions. However, drivers have mixed responses regarding the responsibility for speed and distance control after steering wheel interventions and the responsibility for steering control after gas pedal interventions. With a higher automation level (conditional driving automation), drivers expect automation to remain responsible more often compared to a lower automation level (partial driving automation). Regarding Hands-on requirements, more than 99% of respondents answered that drivers would keep their hands on the steering wheel after all intervention types in partial automation, while 60–95% would place their hands on the wheel after various intervention types in conditional automation. A misalignment between actual logic and drivers' expectations regarding control responsibilities is observed by comparing survey responses to the mode transition logic of commercial partially automated vehicles. To resolve confusion about control responsibilities and ensure consistent expectations, we propose implementing a consistent mode design and providing enhanced information to drivers.

@en

As the adoption of automated vehicles becomes more prevalent, there is a need for systematic approaches to understanding, communicating, and analyzing the design aspects of the technology. Designing a taxonomy - a hierarchical framework that classifies and organizes design elements and features - is a critical tool in this pursuit. A well-defined taxonomy provides an effective communication tool and enables stakeholders to understand, discuss, and compare system designs. However, current taxonomies often fail to consider technological advancements and the needs of stakeholders. This gap limits the understanding of vehicle automation and hinders addressing critical aspects such as safety and social impact. This workshop aims to propose a stakeholder-centered taxonomy for automated vehicles, taking into consideration the perspectives and requirements of different stakeholder groups, such as technology professionals, policymakers, and end-users. The workshop aims to identify relevant considerations for taxonomy design that cater to different stakeholders, incorporate diverse perspectives, and highlight opportunities and challenges for effective communication and decision-making in vehicle automation. @en

Introduction: SAE Level 3 is known as conditional driving automation. As long as certain conditions are met, there is no need to supervise the technology and the driver can engage in non-driving related tasks (NDRTs). However, a human driver must be present and alert to take over when the automation is facing its system limits. When such an emergency takes place, the automation uses the human machine interface (HMI) to send a take-over request (TOR) to the driver.

Methods: We investigated the influence of a color themed HMI on the trust and take-over performance in automated vehicles. Using a driving simulator, we tested 45 participants divided in three groups with a baseline auditory HMI and two advanced color themed HMIs consisting of a display and ambient lighting with the colors red and blue. Trust in automation was assessed using questionnaires while take-over performance was assessed through response time and success rate.

Results: Compared to the baseline HMI, the color themed HMI is more trustworthy, and participants understood their driving tasks better. Results show that the color themed HMI is perceived as more pleasant compared to the baseline HMI and leads to shorter reaction times. Red ambient lighting is seen as more urging than blue, but HMI color did not significantly affect the general HMI perception and TOR performance.

Discussion: Further research can explore the use of color and other modalities to express varying urgency levels and validate findings in complex on road driving conditions.@en

Automation of vehicles not only provides greater safety, but also many previously unimagined opportunities, such as less inequality, less stress, and more meaningful activities while driving. However, the uptake and implementation of automated driving have been falling short of its promise, due to challenges in identifying safe and acceptable ways for humans to interact with automated vehicles. The human-machine interface (HMI) in vehicles plays a more critical role today than ever before. The main research question addrssed in this study, as part of the EU-funded HADRIAN project, is: What steps need to be taken to achieve effective fluid HMI (f-HMI) design for improved driver-vehicle dialogue, especially in critical scenarios?.

@en

In the future, automated vehicles can allow much longer journeys to be undertaken without a driver. This could involve travel across countries with different road systems, traffic rules, and regulations. Disparate cultures also have different expectations of how vehicles should behave. These need to be taken into account when designing HMIs to convey vehicle behaviour and intent. In this work, we presented four scenarios where culturally-sensitive HMIs could benefit both drivers and other road users in a cross-cultural setting. This video will hopefully provoke further discussion and research interest in cross-culture human-vehicle interaction.

@en

The design of take-over requests in automated vehicles traditionally focuses on safety and reaction time. We are interested in how take-over requests can be designed from a broader user experience perspective while ensuring safety. This paper proposes designs for ambient sound (i.e., soundscape) and driving noise to inform the driver of transition situations. Drivers must take-over control within the time budget, the time from the take-over request to the automation system limit. The time required for a safe transition depends on the complexity of the driving environment. In a scheduled take-over, which is not an emergency, there is an opportunity for an interaction that gradually introduces the driver into the transition process. Ambient sound is expected to lead the driver back to the loop with comfort, creating a novel transition experience as well as safety.

@en

Human-Machine Interfaces (HMIs) aim to support the interaction between automated vehicles and drivers to improve safety and driver experience. With the development of automated vehicles, drivers interact with vehicles in new scenarios. In addition to visual modality, sound is the other modality often used in vehicles. Previously, sounds were mainly used for alarms, but they can be used in other ways in automated vehicles. Therefore, a new approach to sound design is needed. We proposed an interactive approach for sound design to improve driver safety and user experience in automated vehicles. In this study, we suggested that the driver's interaction with automated vehicles should be analyzed based on the user and contextual understanding, and the sound should be designed to consider the appropriateness of situation matching and alert levels. This study showed that the approach supports designing sounds that enhance vehicle and driver interaction.@en

In automated vehicles, the driver and the vehicle make a decision on the driving. However, there is no guarantee that drivers always agree or follow the system's decision. Drivers can reject the system's proposal or regain control, and it reduces the usefulness of automated vehicles. When a decision conflict happens, the vehicle can negotiate with the driver. Human-human communication depends on the individual's attitude and situation. Similarly, the negotiation style needs to differ depending on the context of conflict and the cause of disagreement. In this workshop, we address the negotiation approach to designing HMI and discuss considerations for applying the human-human negotiation style to human-automated vehicle interaction design. HMI design using a negotiation approach can address the decision conflict between humans and automation and expect enhancing trust and acceptance.

@en

In automated driving, the user interface plays an essential role in guiding transitions between automated and manual driving. This literature review identified 25 studies that explicitly studied the effectiveness of user interfaces in automated driving. Our main selection criterion was how the user interface (UI) affected take-over performance in higher automation levels allowing drivers to take their eyes off the road (SAE3 and SAE4). We categorized user interface (UI) factors from an automated vehicle-related information perspective. Short take-over times are consistently associated with take-over requests (TORs) initiated by the auditory modality with high urgency levels. On the other hand, take-over requests directly displayed on non-driving-related task devices and augmented reality do not affect take-over time. Additional explanations of take-over situation, surrounding and vehicle information while driving, and take-over guiding information were found to improve situational awareness. Hence, we conclude that advanced user interfaces can enhance the safety and acceptance of automated driving. Most studies showed positive effects of advanced UI, but a number of studies showed no significant benefits, and a few studies showed negative effects of advanced UI, which may be associated with information overload. The occurrence of positive and negative results of similar UI concepts in different studies highlights the need for systematic UI testing across driving conditions and driver characteristics. Our findings propose future UI studies of automated vehicle focusing on trust calibration and enhancing situation awareness in various scenarios. @en