The Biggest Change in Automobiles in the Last Decade: Perception Hardware

The biggest change in automobiles over the past decade has been the addition of perception hardware. The use of perception hardware has given cars a sort of all-seeing eye, allowing for a substantial increase in safety capabilities.

Just a decade ago, AEB (automatic emergency braking) and 360° panoramic imaging were considered luxury features in B-class cars. Nowadays, cameras, ultrasonic sensors, and even laser radar are becoming increasingly common, with vehicles like the XPeng P5, NIO ET7, and Ideal L9 leading the trend. Remarkably, discussions have even started to surface about whether it’s possible to extend the range of laser radar for detecting distances from a few dozen meters to two or three hundred meters.

As significant as external perception hardware is, internal safety considerations are also vital. Ignoring internal safety risks detracts from a car’s full potential.

The Catalyst for Developing Internal Perception Hardware: Children

Data shows that if a car is left in the sun for 15 minutes at a temperature of 35℃, the internal temperature can reach 65℃. If left for 20 minutes, the heat can cause brain and kidney damage to a child, and if left for more than half an hour, it can cause unconsciousness or even death.

Everyone wants to protect their children, no matter how tight finances are. Incomplete statistics show that 70% of families that buy cameras do so because they have children. Modern cameras are extremely advanced; for instance, the Xiaomi camera can monitor for any movement and send remote alarms if the situation requires.

If children are at the highest risk for safety accidents due to stuffiness in the car, many parents would be willing to purchase devices with similar functions to protect them. Therefore, CPD (Child Presence Detection) is gaining attention.

CPD has many derivative versions, such as the Latte DHT-PHEV rear-row life sign monitoring device. After locking, the ultrasonic sensor is activated to detect any living signs within 90 seconds. If there are any signs of life, the device will sound an alarm, send a text message and notify the owner through the app, and simultaneously open the skylight to prevent high temperatures in the car.“`markdown

The Sihao QX CSS child safety monitoring system uses ultrasound and facial recognition technology to monitor vital signs in the vehicle and promptly send alarm messages to the owner to prevent unexpected incidents. Therefore, the current mainstream methods for monitoring children in the industry include breathing and heart rate recognition based on 60GHz radar, RCS recognition based on 4D imaging radar, and child recognition based on cameras, among others.

Tesla has also introduced a variant of Cabin Overheat Protect. After the car is locked, the temperature inside the vehicle can be maintained at a safe temperature for several hours, and the central control screen will display the current temperature inside the vehicle, reassuring passers-by that they need not worry about children and pets.

However, the sensitivity and accuracy of CPD have not yet reached absolute maturity, largely due to the lack of strong industry standards for its widespread application. The good news is that CPD is beginning to draw the attention of safety agencies. Starting in 2022, E-NCAP announced that it will award a score of 4 points through “direct sensing” technology, including alarm strategies. C-NCAP has also added a CPD testing item in the 2025 version of its regulations, and vehicles equipped with this function will earn points.

According to E-NCAP’s requirements, the targets to be detected are infants, one-year-olds, three-year-olds, and six-year-olds. The movement patterns of the targets are classified into two categories: limb movement and respiratory movements when covered by a blanket or in a shadow.

However, there is still much work to be done to ensure its sensitivity and accuracy. For example, I have participated in the verification and R&D of a domestic CPD product, where after being subjected to a “105℃ high-temperature storage test” for 48 hours or a “random vibration test” for 24 hours, the sensitivity of the product would decrease. The purpose of these tests is to use laboratory tests to accelerate the verification of product functionality and reliability at its limits or after its aging. This indicates that there should be no problems with the daily use of new cars, but ensuring the stability of the product under extreme conditions or after years of use will still pose a challenge.
“`## “Danger” to Adult Drivers, Next Step Is Imperative

The essence of the ADAS advanced driving assistance system, which guarantees the functionality through external perception hardware, is “assistance,” so the driver is mainly responsible for the system. An investigation by the National Highway Traffic Safety Administration(NHTSA) shows that drivers are responsible for more than 90% of road traffic accidents, with driver distraction being the key reason.

Therefore, it is necessary to add perception hardware to the vehicle to understand the driver’s state in a timely manner. In recent years, brands that emphasize the ADAS advanced driving assistance function mostly feature DMS driver fatigue monitoring system (Driver Monitor System), which is based on the driver’s physiological image response, composed of two major modules, ECU and camera, and uses the driver’s facial features, eye signals, and head movements to infer the driver’s fatigue state and provide warning prompts and corresponding measures. It offers active intelligent security to drivers and passengers.

However, when monitoring techniques are applied to adults, there are more concerns about its popularity.

Driver monitoring systems (DMS) used to detect driver fatigue generally use traditional 2D cameras, supplemented by infrared illumination technology, which can work properly under dim lighting conditions.

There are privacy concerns with CPD cameras, so with regard to camera solutions, it is specifically noted that they are for the rear row. When it comes to monitoring adults, it is difficult to avoid visualization because it is not just a matter of detecting non-living things, but of analyzing human states.

Compared with safety, privacy concerns are now believed to be secondary. For the sake of public transportation safety, the EU and China have both issued laws and regulations, and China has already made it mandatory to install DMS systems for commercial vehicles such as “two passengers and one risk,” and is also promoting the requirement for passenger vehicles.

Going further, the EX90, the latest safety technology representative of Volvo, is equipped with as many as two cameras, in addition to the steering wheel sensor. These two cameras will always capture the driver’s gaze to monitor whether the driver’s mental state is suitable for driving. When it detects that the driver’s driving state is not good, it will take measures such as voice reminders and gradually strengthen or weaken the reminders according to the feedback. If the driver has no response to the reminders, then the system will intervene in the vehicle’s control.

Of course, automakers are also avoiding unnecessary disputes and making it clear that video information will not be used for other purposes or provided in a voluntary open mode. Taking NIO as an example, in the privacy policy, it introduces the scope and purpose of the functions and information collection specifically for the in-car camera, and the owner’s manual clearly describes the location, monitoring function, and shutdown method of the in-car camera.

In addition, automakers can also use the 3D ToF scheme. The ToF system presents a virtual image of the object’s three-dimensional contour in different colors representing different distances, rather than an RGB color image, which achieves acquiring the outline and behavior of the person without causing privacy issues.

The world’s first 3D ToF-based system scheme entered mass production for front loading in 2015, mainly used for gesture recognition for information and entertainment control. For example, Melexis and BMW’s gesture system is this kind. Alternatively, the 3D ToF sensor mounted on the Ideal L9 also works well in sentry mode.According to industry predictions, due to the fact that DMS has become a necessary system for L2 and L2+ level autonomous vehicles, the global market is exploding. Approximately 1 million vehicles were sold in 2020, with an expected increase to 22 million vehicles by 2025. Therefore, detection technology for adults will also be a popular configuration in the future.

Ultimate Mode is a Cabin Monitoring System

The cabin monitoring system uses camera sensors to understand human behavior and monitor passenger movements. In the event of an accident, the strength and pop-up time of the airbag can also be adjusted according to the position and body shape of each passenger.

The working principle of the airbag is as follows: the signal is received outside the car→SRS ECU (airbag controller) processes the signal→the airbag works. From the principle explanation, the airbag sensor judges whether to pop open the airbag through the induction collision acceleration outside the car.

As for the collision intensity, high-priced airbags now use a staged and layered inflating device, generally using sodium azide as a propellant. The propellant is ignited in graded layers by sensors, and the first layer produces about 40% of the gas volume, while the second layer is filled with gas, reaching maximum pressure, and has a buffering effect on the human body.

However, this process still lacks an important link inside the car, which assumes that the passenger is in an appropriate sitting position. However, everyone has a different body shape and posture, and airbags cannot guarantee maximum effectiveness. If adjustments can be made based on sitting posture and body shape, with cooperation between inside and outside the car, safety will be better guaranteed.

In addition, the most advanced method of intelligent communication now is voice or gesture communication, but it has not yet reached the level of “eye contact.” A real person can tell what kind of approach can make everyone relax simply by looking at the other person’s state.

How can an intelligent butler go further? The cabin monitoring system would be very useful. The vehicle analyzes user characteristics, such as their posture, to adjust the internal environment, and then adjust the temperature or driving mode, implement entertainment features, such as taking selfies and sharing experiences while traveling, and control the vehicle, start the engine, and make face payments by logging in with facial recognition accounts, among other application scenarios. Therefore, if the cabin monitoring system can be popularized, it will improve in multiple areas.### Summary

Safety is the key to the car, with both internal and external factors contributing to its security. Compared to various perception hardware outside the car that greatly enhances the car’s perception ability, the development of the car’s interior is not growing as fast. There are many reasons that constrain its development, including privacy and surveillance issues that have not been well resolved. However, in terms of safety and convenience, there are many benefits to developing interior perception hardware. It can be foreseen that this technology will have great potential in the future. If you had the choice, would you adopt it?

This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.