Automotive cabin sensing is a rapidly evolving area with a range of applications using a combination of sensors and intelligent algorithms.

Today there are two key aspects driving the development of in-cabin sensing technologies.

Firstly, regulations to improve occupant safety are being realized. Both Euro NCAP standards and European Commission (EC) regulations mandate the use of driver monitoring systems (DMS) by 2022, and the U.S. National Transportation Safety Board (NTSB) recommends DMS in semi-autonomous cars. A hands on/off detection (HOD) system is already required in 2021 to meet UN legislation (R79) for lane keeping assist systems (LKAS). Also, child presence detection systems will become mandatory in the U.S. by 2022. With this forthcoming legislation, DMS and occupant sensing capabilities will become a standard feature of L2+ and above driving assist platforms.

The second key aspect is comfort enhancement for passengers—for example, through emotional sensing. OEMs have detected this is a differentiation factor and will use leading-edge innovation technologies to address it.

How DMS Can Reduce EU Traffic Fatalities

Safety

The European Transport Safety Council (ETSC) has calculated that 51 people per million inhabitants in the EU were killed in road accidents in 2019, with 95% of the cases caused by a human factor: driver error, distraction, drowsiness, stress, or fatigue.¹ And the European Commission has announced a major package of transport safety initiatives with the “target of a 50 per cent cut in road deaths and serious injuries by 2030, with Vision Zero the aim by 2050.”²

DMS today already support other advanced driver assistance systems (ADAS). As an example, an LKAS is usually combined with an HOD system to detect the hands of the driver on the steering wheel. The system informs the driver to take back control when required. In addition, DMS offer new features to detect the driver’s health and ensure adequate attention levels.

The introduction of vital sign monitoring (VSM) technology, such as electrocardiography (ECG) or electrodermal activity (EDA → skin impedance), offers the ability to sense the driver’s health and stress level in order to avoid potential issues before they even arise. For example, if the VSM system detects the driver is unable to control the car due to drowsiness or fatigue, ADAS can bring the car to a safe state (slow down, steer to an emergency lane, or stop the car).

Comfort

Ensuring an excellent driver and passenger experience is more critical than ever. It offers the opportunity to innovate increasing brand recognition and ensure customer loyalty. The ever-increasing expectations of customers is putting an onus on OEMs to add features that offer this enhanced experience. Personalization and advanced human machine interface (HMI) concepts (intuitive vehicles) are becoming key components of the user experience. OEMs can strengthen their brands by implementing unique comfort features. These could be automatic climate control or ambient light adjustment based on the driver’s mood and mental condition. Touch-free HMI systems allow driver interaction without requiring drivers to move their hands from the steering wheel. Along with voice-operated systems, more precise DMS using eye-tracking technologies continue to close the gap with traditional touch-based interfaces. Differentiation and customer loyalty drive a need for an enhanced user experience.

Applications

A wide range of applications are covered by in-cabin sensing. Figures 1 and 2 show application examples.

Driver monitoring addresses the legislation requirements, with features like HOD systems, and is becoming a mandatory solution deployed as a standard in future cars.

Vital sign monitoring is also an emerging feature addressing not only health conscious drivers but also the aging population trends by monitoring the driver’s health, stress level, well-being, and fitness to operate the vehicle.

Biometric authentication not only provides support for that personalized experience, such as seat and steering wheel position and preferred infotainment setup, but also provides security with the ability to authenticate the driver of the vehicle.

Another vital safety application is occupant sensing. This ensures that should a passenger be accidentally locked inside the vehicle, the driver will be immediately notified with the ability to detect the number and age of the passengers. Emergency responders can be provided with additional information before their arrival to an accident when this data is combined with the emergency call (eCall) system.

A special use case of occupant sensing is child presence detection, which will become (dependent on the region) a mandatory requirement.

Advanced HMI and gesture control provide enhanced user experience (see Figure 2). Instead of searching for and pushing a button on the main control area, a gesture can perform simple functions such as skipping an audio track.

How to Address These Applications

When we look at these applications, be it from their physical location (see Figure 3), the comfort vs. safety aspect, or the solution they need to provide, it becomes evident that a single sensing solution cannot address them all. Sensor fusion with different sensing modalities is required. The combination of safety and comfort applications needs to be addressed. Several technologies need to be combined and work together to reach the goals of the increasing safety and comfort requirements.

Technologies

Time of Flight (ToF) Technology

A time of flight (ToF) camera is one of the key technologies that can address several of the previously described applications, providing both image and depth data.

A ToF camera located in the dashboard or roof can be used for DMS such as eye tracking. ToF is already in production at premium OEMs for advanced HMI and gesture control, with the optimum location being in the roof console. ToF provides an excellent solution not just for detecting the presence of a child but also their body position. You can see the need for a different location of the sensor to enable line of sight to the rear seat (as shown in purple in Figure 3).

With the advances in image sensing technology, ToF is the depth sensing method of choice for its smaller form factor, wide dynamic range of sensing, and ability to operate in direct sunlight. The combination of high resolution distance measurement and medium resolution intensity image (ambient light insensitive 2D active brightness image) is unique to ToF. ADI’s ToF sensor has the highest resolution (1 megapixel) in the market, which enables wider field-of-view cameras. Although many vision applications can be realized using 2D cameras, the 3D (depth) information offers an additional level of robustness. For comfort applications this translates to better user experience, and for safety applications it is the key differentiator.

Since there will be multiple cameras in the cabin to support different use cases, it is necessary that cameras support interference cancellation to reduce errors in depth measurement. Such challenges must be solved at the system level, and ADI is actively pursuing this area with a smart mix of hardware and software elements.

For biometric authentication ToF can provide a very secure solution, making it very difficult to fool the system, as has been proven possible with other implementations.

ADI technology in the ToF area includes ToF imagers (ADSD3100), laser drivers (ADSD3000), and power regulators (ADP5071).

Car Camera Bus (C²B)

For connecting different cameras regardless of 2D imagers or 3D ToF, ADI also provides a dedicated automotive camera link technology, the Car Camera Bus (C²B™). C²B is a low cost solution to transport up to 2 megapixel camera data along with control information from cameras installed inside or outside the car.

Impedance Sensing

ADI offers several VSM solutions and works closely with software partners to provide an overall system solution for features including heart rate variability, stress, etc. The AD5941W is an integrated solution for both HOD and for measuring EDA.

Impedance sensing provides a robust and reliable solution to not only detect hands on the steering wheel but also the quality of the grip, which is a key requirement. The location of the hand on the steering wheel can also be provided. EDA information can be used to detect the driver’s current stress level.

ADI has significant expertise in this area from working on this technology across multiple industries. The AD5933 high precision impedance converter is used in cars today for HOD. The AD5941W supports multiple HOD zones in a single device.

Electrocardiography

Combining impedance sensing components with the AD8232W, a high precision amplifier for electrocardiography, ADI offers a complete VSM solution consisting of only two components. ECG along with EDA can monitor the driver’s health state, but ECG can also be used to address biometric authentication applications.

Conclusion

The automotive industry is adopting advanced in-cabin sensing applications to increase driver and passenger safety. Consumers demand continuous innovation in user experience and personalized comfort. Sensor fusion is required for accurate and robust L2+/L3 driver assist solutions. ADI is strongly positioned to address the sensing needs of this market and to support these requirements with an ecosystem of hardware and software algorithm partners for ToF algorithms (gesture control, eye tracking, etc.) and VSM (for example, ECG analysis).

Go to analog.com/automotive for more information.