Exponent : Evaluating L2 Advanced Driver Assistance System Performance

Leveraging crash and non-crash Level 2 ADAS data approaches to understand safety benefits

Advanced driver assistance systems (ADAS) are becoming more common in the auto industry - in fact, they are poised to transform driving and automotive safety. It's estimated that half of all cars in use globally by 2030 will feature ADAS technologies such as lane-keeping assistance and automatic emergency braking - up from approximately 10% in 2020.

Deploying new and innovative ADAS technologies, especially L2 systems (see fig. 1), has the potential to change the nature of driving. Recognizable in the marketplace as Tesla Autopilot, GM Super Cruise, Ford BlueCruise, Toyota Teammate Advanced Drive, Mercedes-Benz Drive Pilot, Honda Sensing, BMW Highway Assistant, Audi Traffic Jam Assist, and Nissan ProPILOT Assist, L2 ADAS systems integrate L1 systems, such as adaptive cruise control and lane-centering assistance, to provide drivers with continuous assistance in both lateral (steering) and longitudinal (braking and acceleration) vehicle control. By some estimates, adoption of L2 systems could reach an installation rate of 20% by 2025 in new vehicles in some regions.

Level of Automation - A Quick Guide

Fig. 1: Adapted from National Highway Traffic Safety Administration's Levels of Automation

Frequently marketed as convenience features (similar to cruise control), L2 systems have the potential to also improve safety by reducing crash rates. However, compared to simpler systems like automatic emergency braking, which can slow or stop a car nearing collision, performance evaluations of L2 systems are complicated by several factors, including the fact that drivers can activate and deactivate L2 systems at their discretion. As a result, it's critical for OEMs and other stakeholders to understand the strengths and limitations of how L2 ADAS safety is being assessed.

Learn more about the methods used to study ADAS systems in Exponent's article "Assessing the Impact of Driver Assistance Technology: A Review of Non-Crash and Crash Studies," presented at the 15^th International Conference on Applied Human Factors and Ergonomics.

Non-crash assessments of L2 systems

Researchers have frequently turned to non-crash studies to assess emerging technologies before the availability of extensive crash data. These involve alternative forms of data collection and analysis to estimate and predict the real-world effectiveness of L2 features based on a wealth of variables and measurements, such as driver attentiveness, speed, hands-on-wheel status, secondary task activity, and ADAS feature use.

While non-crash studies can provide valuable insights as proxies for real-world driving scenarios, the results are not guaranteed to reflect real-world performance.

The findings from such non-crash studies for L2 systems are, thus far, mixed. Some have indicated slower driver response times to unexpected traffic events, decreased levels of arousal (measured by heart rate and other factors, including observed yawning and frequency of eye closure), and longer maximum total-eyes-off-road time (TEORT) when L2 systems are engaged. Notably, one recent study, "Disengagement from driving when using automation during a 4-week field trial," concluded that the longer drivers used L2 systems, "the more likely they were to become disengaged, with a significant increase in the odds of observing participants with both hands off the steering wheel or manipulating a cell phone relative to manual control."

Other research, in contrast, suggests that L2 systems may increase situational awareness and decrease driver workload - which might be presumed to improve crash risk compared to manual driving - and that when drivers use L2 systems, they keep their eyes on the forward roadway at similar rates to manual driving, operate L2 systems as intended, and engage in secondary tasks at a similar rate to manual driving.

While non-crash studies can provide valuable insights as proxies for real-world driving scenarios, the results are not guaranteed to reflect real-world performance. Given their mixed results, it is even more important to understand how driver use of this technology impacts overall crash rates. From a human factors perspective - the scientific study of how humans interact with emerging vehicle technologies - it is also critical to examine the mechanisms behind when and why drivers may use L2 technologies in unanticipated ways, including whether driver behavior reflects over-trust or perhaps whether drivers are simply adapting their behavior to the driving context at hand.

Manufacturer telemetry systems, which can track both crash events and ADAS usage, are likely to play a pivotal role in understanding the performance of driving automation technology.

Crash assessments of L2 technology

In addition to driver-centric, non-crash studies, the growing number of on-road miles for L2 technologies means real-world crash data are becoming increasingly viable as a means of assessing safety performance. These data are playing a key role in helping industry understand the potential implications of L2 ADAS, with important takeaways for how they influence crash rates.

Crash studies for L2 ADAS fall into two categories:

1. Use/non-use studies, which compare crash rates for the same vehicles when they are or are not using L2 technology
2. Equip/non-equip studies, which compare crash rates for vehicles with and without L2 systems installed

Equip/non-equip studies do not include any information regarding the actual use of L2 systems, which may limit their usefulness in estimating the magnitude of beneficial effects for relevant technologies. In the case of use/non-use studies, publicly available crash databases generally do not contain sufficient information to determine mileage with the systems engaged or whether an L2 system was in use at the time of a crash, in addition to other important contextual information.

Use/non-use field analyses of L2 systems to date have depended on manufacturer-provided data, particularly miles driven and crash events with the systems engaged. Studies have been published using telemetry data from Tesla and GM to assess the effect of their L2 systems on crash rates. One of the challenges addressed by both studies is that the "road mix" (e.g., highway versus surface street driving) for L2 systems differs from the typical driving mix of the general public. Consequently, these studies attempt to develop comparable crash rates, either through mathematical adjustment or by only considering mileage and crashes occurring on particular roads.

Future ADAS research and development

Undoubtedly, L2 technologies and emergent L3 systems - including Mercedes-Benz's Drive Pilot and BMW's Personal Pilot, which allow for hands-free and eyes-free driving in specific circumstances - have the potential to influence roadway safety.

Non-crash assessments represent innovative ways to augment driver behavior data related to L2 systems. However, to date, these studies have not produced conclusive results and may limit insights into real-world system performance. While non-crash ADAS assessments will have an ongoing role, an emphasis on non-crash study quality will be important in analyses of the relationships between this data and real-world driving scenarios, performance, and safety.

Crash assessments have the advantage of directly demonstrating real-world performance, but they are "lagging" indicators that rely on extensive deployment before providing conclusive results. L2 systems also pose new challenges in crash assessment because neither system engagement nor miles with the system engaged are observable using traditional data collection methods such as police reports. In this case, manufacturer telemetry systems, which can track both crash events and ADAS usage, are likely to play a pivotal role in understanding the performance of driving automation technology.

Pairing insightful approaches to data analysis with equally sophisticated methods of measuring and mitigating risks related to how drivers engage with ADAS features can help OEMs and regulators improve road safety through the deployment of these emerging systems.