Human-Machine Interfaces and Vehicle Automation: A Review of the Literature and Recommendations for System Design, Feedback, and Alerts

Abstract

Introduction

The effectiveness of the human–machine interface (HMI) in a driving automation system is based, in part, on how it issues alerts and requests to the driver. That is, it must provide clear information about the state of the world in a manner that helps quickly and sufficiently orient the driver to the driving task when required.

The purpose of the current study was (a) to review and synthesize existing research and guidance on HMI design regarding requests to intervene for driving automation systems and (b) to propose a clear and comprehensive set of recommendations that could inform future system development and implementation.

Key Findings

Based on the literature search, nearly 100 relevant articles were identified and further examined. The majority of HMIs in the articles were evaluated in a driving simulator or laboratory setting and focused on Level 3 automated systems. The focus of these studies varied significantly, yielding a wide array of independent variables, including but not limited to differences in alert types, alert modalities, presentation of alerts, timing of takeover requests, request wording, request urgency, use case scenarios, and various driver characteristics. Likewise, there was variability in the underlying outcome measures; however, these were largely grounded in driving performance or behavioral measures (e.g., eye glance metrics). Several articles were also identified which provided HMI design recommendations for vehicle automation. The report provides detailed information about the literature search and outcomes.

Recommendations

Based on outcomes from studies identified in the literature search as well as from guidelines from existing sources, a list of 10 recommended design guidelines was proposed and organized into three broad themes:

Modality

1. Systems should be designed to be multimodal.
2. Visual displays should be used to support continuous status information, as well as other content-rich information. Visual interfaces should also prioritize pictographic information and standardized symbology over text-based messages. Text should be used to supplement non-standard symbols, preferably in non-time critical situations.
3. Auditory and/or tactile displays should complement visual information and be used to help reorient driver attention in critical situations. Sustained attention to HMI should not be required in time critical situations.

Information Content and Control

4. System status (e.g., on, off, activation, deactivation/disengagement, availability) should be presented clearly and continuously. In doing so, display elements for a common system should be grouped together.
5. Alerts should help orient the driver to the source of danger or provide some information about the traffic context. High priority information should be presented close to the driver’s line of sight.
6. Feedback about RTIs or failure modes should be provided. For example, providing reasons for a takeover or conveying information about system limitations in situ.
7. Driver decision-making and responses should be supported by providing them with information to support situation awareness and required actions. Systems should be designed to minimize or prevent unintentional actions (e.g., accidental activation or deactivation of the system).

Timing and Stages

8. Alert should give sufficient time to drivers to regain control safely and effectively.
9. The intensity of the alert should reflect the urgency of the situation, without being a hindrance, distraction, or annoyance to driver. The intensity of an alert should increase as the available time (i.e., response window) decreases.
10. Gradient or multi-staged alerts (e.g., first visual, then auditory) should be used to help convey urgency and to counter non-responses to earlier alerts.

Methodology

A literature review of HMIs in driving automation systems was conducted. The resulting articles were organized and categorized into top-down and bottom-up guidelines. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The Web of Science (WoS) and the Transportation Research International Documentation (TRID) databases were targeted for queries related to vehicle automation, control transitions, and HMI. All articles were accepted for publication in conference proceedings, journals, or technical reports and were published between January 2011 until April 2021.

An initial list of 13,899 unique articles was obtained from the WoS and TRID databases. Through an iterative process of reviewing titles, abstracts and, eventually, full papers, a final set of 96 articles was identified. Key study features and outcomes were distilled from each study in the set.