An Examination of How Longer-Term Exposure and User Experiences Affect Drivers’ Mental Models of ADAS Technology

Abstract

Introduction

New advanced vehicle technology offers new safety and convenience features to drivers; however, it also stands to change the nature of the driving task as the systems take on more of the driving responsibilities. Driver understanding of advanced driver assistance systems (ADAS)—referred to as a mental model—is important in regards to safe and appropriate system usage. If drivers do not understand the technology, especially its limitations, they might use it in situations that it was not designed for, possibly compromising safety.

Although past research has examined how mental models affect performance and safety, it is less well understood how mental models develop and evolve over time. The goal of this project, based on a cooperative research program between the AAA Foundation for Traffic Safety and the SAFER-SIM University Transportation Center, was to assess the mental models of naïve users of adaptive cruise control (ACC) and evaluate how their mental models change with greater exposure to the systems (i.e., time and experience).

Key Findings

Over the course of the first 6 months of vehicle ownership, drivers’ understanding of ACC improved (as measured by a mental model assessment survey). These improvements were largely due to increased understanding of the technology’s limitations, as opposed to improvements in knowledge about system function.

Different sub-groups of drivers emerged based on their knowledge and confidence, including drivers who were: (i) high in knowledge and highly confident, (ii) high in knowledge but low confidence, (iii) low in knowledge and low confidence, and (iv) low in knowledge but highly confident. Drivers who were high in knowledge (i and ii) showed improvement in their system understanding from the first to last session, whereas drivers in the low knowledge groups (iii and iv) did not show improvement and, some  exhibited lower understanding in the final test session compared to the first.

With respect to driving performance in the simulator session held at the end of the 6-month period, the mental model scores were not predictive of responses to the edge case situations, likely due to the small sample size (which was reduced due to COVID-19 limitations). However, a comparison of the current mental model scores against previous results revealed some additional insight about the role and effectiveness of exposure in mental model development. For both mental model accuracy and driving performance, the current group of drivers fell in between the strong and weak mental model benchmarks established in a previous study. The combined data further support the linkage between mental model strength and driving performance.

Implications

Overall, the combination of questionnaire, simulation, and naturalistic data used in the current study offers some important insight into how mental models develop in new owners of vehicles equipped with advanced technology.

The current results suggest that 6 months of exposure led to some improvements in knowledge of the ACC system. However, while the level of understanding in this group of drivers was superior to a group of drivers who received only nominal training, they did not achieve the robust knowledge observed in a highly trained group of drivers. This suggests that there is room for improvement in how drivers gain understanding about driver support features.

These results also revealed that some participants were well-calibrated in their confidence in relation to their knowledge, while others were not. Perhaps most concerning was the emergence of a sub-group of drivers who demonstrated relatively low understanding of the system yet were highly confident in their knowledge. Additional research is needed to understand how different approaches (e.g., instruction materials, in-vehicle training) might be used to augment the development of mental models, especially considering individual differences and driver sub-groups.

The results also suggest that training methods should focus on understanding the limitations of particular driver support features, as this is often the weaker component of mental models but also more important for appropriate responses in edge case situations.

Methodology

Thirty-nine experienced drivers between the ages of 25 and 65 (M=43.0, SD=12.1) were recruited. Each potential participant was screened for eligibility. They were required to have a valid driver’s license, have at least three years of driving experience, and drive at least 2,000 miles per year. Potential participants were also required to have purchased a vehicle (new or used) within the previous 6 weeks that was equipped with adaptive cruise control (ACC), and ACC was not present on any vehicle they previously owned.

The quality of the participants’ mental models regarding a typical ACC system was assessed at the start of the study. This mental model assessment was given again after 2 weeks, 4 weeks, 8 weeks, 16 weeks and at the end of the study (i.e., 6 months after start). They were also asked to report, on a weekly basis, the miles driven using the vehicle, miles driven using ACC, as well as any experiences they encountered in which they were confused by the actions of the ACC system.

After 6 months of enrollment in the study, drivers participated in a simulator session at the National Advanced Driving Simulator at the University of Iowa. Upon arrival, participants completed their sixth and final mental model assessment. Next, they completed the simulated, experimental drives while interacting with the ACC. The driving environments were designed to mimic the range of operational design domains for ACC, including several safety-critical or edge-case events representing situations that possibly exceeded the capability of the ACC system.