This article is reprinted from the public account “新车一讲”, author’s Weibo: @不是郑小康
On April 29, 2020, Simon Fürst, BMW’s chief expert in autonomous driving, publicly released the BMW autonomous driving roadmap covering SAE Level 1-5 levels at an event. On May 4th, BMW submitted the “BMW Autonomous Driving Safety Test Evaluation Report” to the National Highway Traffic Safety Administration (NHTSA) of the United States.
With the support of the above two materials, we were able to learn about the L3 autonomous driving system on the new intelligent electric SUV iNEXT from BMW a year in advance.
The controversy over L3 autonomous driving systems has been around for a long time. Prior to BMW, major manufacturers such as General Motors and Ford, and new manufacturers such as NIO and Ideanomics, announced the direct development of L4 autonomous driving systems by skipping L3. On the other hand, Mercedes-Benz, BMW, Audi, and XPeng all invested heavily in L3.
In any case, the L3 system carried by BMW iNEXT is the most complete system architecture and comprehensive L3 autonomous driving system in recent years. Especially after the fierce competition of L2.5, L2.99 and L3 autonomous driving systems of a number of domestic independent brands, BMW iNEXT stands out even more.
Hardware Solution hPAD
Hardware is the foundation for realizing the upper-level experience of autonomous driving. We have seen some advanced driving assistance systems fall behind in terms of functional implementation due to limitations in hardware capabilities. However, in the short term at least, BMW’s L3 autonomous driving hardware solution hPAD does not have this worry.
Starting with sensors, the BMW iNEXT is equipped with four types of sensors: ultrasonic sensors, millimeter-wave radar, cameras, and lidar. They are listed as follows:
Ultrasonic sensors: 12 (6 front and 6 rear), an industry-standard configuration for short-range sensing.
Millimeter-wave radar: 5 (1 forward radar, 2 short-range front radar, 2 side rear radar), with different sizes and specifications and completing millimeter-wave radar’s 360° sensing coverage for the entire vehicle.
Cameras: 10 (4 surround-view cameras, 6 perception cameras), with a front-facing triple camera, a left and right side rear camera, and one rear view camera for perception.# Laser Radar: The Supplier is Unknown, but in 2021, Essentially Valeo ScaLa 2 that Combines Cost, Regulations, and Performance Advantages.
In July 2018, Audi launched the world’s first Level 3 autonomous driving hardware-equipped car, Audi A8, in Spain, which premiered the Valeo ScaLa 1 laser radar. In the following years, due to hardware architecture constraints and immature high-precision maps, laws and regulations, the L3 autonomous driving system on this car ultimately became obsolete.
After 5 years, BMW iNEXT might take over Audi A8’s baton, carrying the upgraded performance Valeo ScaLa 2 laser radar.
Notably, compared to ScaLa 1, ScaLa 2’s line count has increased four times and point cloud count has increased six times. The comprehensively upgraded performance will endow the BMW iNEXT’s L3 autonomous driving system with more powerful perception capability and a wider speed operating range.
Next is the computing platform. Unlike Tesla and XPeng, which adopt centralized computing platforms, BMW chose a distributed computing platform that combines multiple chips. These chips include two Infineon Aurix MCUs, two Intel Denveton 8-core CPUs, two world’s first Mobileye EyeQ5 chips, and Renesas V3H and H3 chips.
Are you confused? Don’t worry, let’s analyze them one by one.
Starting with the most significant one, Mobileye EyeQ5 is the world’s first vehicle-grade chip based on 7-nanometer FinFET process.
The more advanced process brings higher energy efficiency: while EyeQ5 has a computing power of up to 24 Tops, its energy consumption is controlled at 10W. Compared to EyeQ4 with 2.5 Tops of computing power, EyeQ5’s energy efficiency has increased by nearly three times, and computing power has increased by nearly 10 times.
The BMW iNEXT is equipped with two EyeQ5 chips, one of which carries Mobileye’s algorithm, while the other EyeQ5 open version is based on BMW’s proprietary algorithm written into the chip.
In its collaboration with automakers, Mobileye has always been dominant. Prior to EyeQ5, Mobileye’s front-loading business model was to supply chips and algorithms as a package, with carmakers having no knowledge of the specific principles behind the Mobileye algorithm.
This “stumbling block” has been completely changed by EyeQ5: Mobileye provides a complete software development kit (SDK) for EyeQ5, along with an integrated hardware security module, allowing carmakers to independently research relevant algorithms and deep neural networks, and safely deploy them to fleets through secure FOTA updates.
The BMW iNEXT is the first vehicle to benefit from this business strategy adjustment.
Specifically, the EyeQ5, along with two Intel Denveton 8-core CPUs, is responsible for the fusion processing and decision planning of the camera, millimeter wave radar, lidar, and high precision map information for L3 level autonomous driving.
Next are two Infineon Aurix MCUs, long-tested chips in the automotive industry. We have seen their presence in the Audi A8’s zFAS and Tesla Autopilot HW 2.0/1/5 version hardware.
In particular, one of the Aurix chips in the iNEXT plays a role similar to that in Tesla Autopilot 2.x: processing ultrasonic data to achieve automated parking.
The other Aurix chip continuously monitors system operation status, ensuring that the entire system meets the highest ASIL-D level vehicle regulatory standard, which is crucial for L3 level autonomous driving systems.
Lastly, the Renesas V3H and H3 chips are responsible for 360-degree surround view and visual automatic parking in the BMW iNEXT.
According to previous information released by BMW, the power consumption of the hPAD is 200W, with liquid cooling.
Overall, whether it’s the high-performance Mobileye EyeQ5 or the Aurix, whose safety performance has been tested over a long period of time, the BMW hPAD has the meaning of “matching good horses with good saddles,” balancing high performance with vehicle regulatory certification while matching the sensor architecture.
HMI Human-Machine Interaction
At the beginning of the article, we mentioned that the controversy over Level 3 autonomous driving systems has been going on for a long time. Ideal Auto founder Li Xiang openly stated that L3 is an anti-human product in practical use.
This is because, according to the definition of SAE, L3 level autonomous driving system is a conditionally autonomous driving system. When the operating conditions (ODD) are met, the system can perform all dynamic driving tasks, and when the operating conditions are not met, the human driver needs to respond to the system’s request and take over the vehicle in a timely manner.
This product setting determines that the system needs to interact frequently with the driver. How to maximize the travel experience while ensuring driving safety, is a true test of the skills of the human-machine engineering department of car companies.
Returning to the BMW iNEXT, it has two driving modes: Boost mode and Ease mode.
In Boost mode, the driver can retain traditional controls and experience the joy of driving without restriction (similar to Tesla’s track mode); in Ease mode, the driver can activate the autonomous driving system and focus on other activities with eyes off the road.
Here we discuss the human-machine interaction of the L3 autonomous driving system in Ease mode.
First, let’s explain the ODD of L3 on iNEXT: the system can only run under the premise that high-speed passing roads covered by high-precision maps, weather and environmental conditions allow all sensors of the vehicle to operate normally, and the maximum speed does not exceed 85 miles per hour (approximately 137 km/h).
Reflected on the dashboard, the status display of the system’s operability, start-up and exit is as follows:
- If the current status does not support system operation, the dashboard will display an icon with a slash;
- If the current state supports system operation, the dashboard will display a READY icon;
- If the system is started to take over the vehicle normally, the icon will turn blue, indicating that the system is in operation;
- If the driver takes over the vehicle and turns on the driving assistance, the icon will change the A in the lower left corner to a steering wheel.
Compared with Tesla’s Autopilot, the status display of the system’s operability, start-up and exit is as follows:
- If the current state does not support system operation/driver takes over the vehicle, the dashboard will not display anything.- If the current status supports the operation of the system, the dashboard will display a grey steering wheel;
- If the system is started successfully, the steering wheel icon will turn blue, indicating that the system is running.
In comparison, although BMW’s scene division is more detailed compared to Tesla, the presentation of the status is more complicated. Ultimately, consumers will judge which is better.
In addition, according to the definition of SAE, Autopilot is only a level 2 assisted driving system, while iNEXT is a level 3 autonomous driving system. BMW stated in the report that to avoid the high-risk scenario of “the system has exited, but the driver still thinks the system is running normally”, BMW strengthened the display of the system’s operation and exit status:
- If the system is in operation, the light strip on the steering wheel will display blue;
- If the system does not support operation and triggers TOR (TakeOver Request, issuing a takeover request), the light strip on the steering wheel will turn yellow for continuous warnings;
- If the driver fails to take over in time, the system will exit operation, and the light strip on the steering wheel will display red
This visual interaction system for the steering wheel of iNEXT has similarities with General Motors’ SuperCruise, and the main difference may be that the light strip on the Cadillac is located at 12 o’clock, while the BMW is located at 3 o’clock and 9 o’clock.
In the 2018 Consumer Reports’ test of assisted driving systems, General Motors’ SuperCruise surpassed Tesla’s Autopilot and won first place, with the key scoring item being “Clear When Safe to Use”. In other words, accurately conveying the system status to the driver.
From this perspective, BMW iNEXT’s HMI design is somewhat inspiring.
Dashboard icons, steering wheel light strips, and sound prompts all accurately and clearly convey the system status to the driver. So how does the system accurately perceive the driver’s status?The driver monitoring system of iNEXT combines the gripping force sensor, torque sensor, pedal sensor, seat sensor (to prevent the driver from moving to the back seat after L3 activation…) and the seatbelt fastening sensor to confirm that the driver is in a take-overable state.
To sum up, similar to the hPAD’s autonomous driving sensor scheme, BMW has done a very thorough job in the hardware support of HMI human-machine interaction. This is not only a manifestation of the maturity of the supply chain, but also a requirement for the positioning of iNEXT as a luxury intelligent electric SUV.
Functional Safety and Extreme Scenario Deduction
As one of the world’s most successful luxury brands, BMW undoubtedly pays high attention to the safety performance of L3 level autonomous driving. The redundant fault-tolerant design of hPAD and system performance under extreme scenarios are therefore crucial.
Specifically, BMW designs functional safety from the following 14 aspects:
- Design architecture;
- Sensors;
- Execution mechanisms (brakes, acceleration, steering, etc.);
- Communication failure;
- Potential software errors;
- Reliability;
- Potential control deficiency;
- Poor control execution;
- Potential collision with obstacles and other road users;
- Potential collision caused by system execution action;
- Lane deviation;
- Loss of traction or stability;
- Violation of traffic rules;
- Deviation from expected/normal driving habits.
After integrating the relevant provisions of ISO 26262 and IEC 61508, BMW’s hPAD redundant design is as follows: the main channel and the sub-channel are redundant to each other (corresponding to the two EyeQ5 chips mentioned earlier), and each has its own independent diagnostic unit, which can detect faults and transfer control to the other channel in a timely manner.
In case of faults that affect both channels at the same time, the diagnostic unit will transfer control to the third basic channel to help the vehicle reach a minimum risk state. The “third channel” here is an L2 level auxiliary driving system, with a sensor architecture of one front monocular camera and five millimeter-wave radars, a calculation platform consisting of a single Mobileye EyeQ5 chip, an Intel Denverton dual-core chip, and an essential Infineon Aurix chip.
Simply put, the L3 autonomous driving system operates on dual platforms with redundant backup. In extreme scenarios where both platforms cannot function simultaneously, the system will downgrade to an L2 assisted driving system, and promptly alert the driver to take over.
This is arguably the most complete functionally-safe design for an L3 autonomous driving system that we have ever seen.
Now, let’s analyze some extreme scenarios. As mentioned earlier, iNEXT’s L3 autonomous driving system requires multiple conditions to be met before it can safely operate. In situations where these conditions are not met, the system may not be able to safely control the vehicle. Examples of such situations include environmental conditions that are close to the operational design domain (ODD) limits, detection of system faults during continuous monitoring or sensor self-diagnosis, which may require a downgrade in operation.
While waiting for the driver to take over, iNEXT’s system will continue to perform driving tasks, but its functionality will begin to be limited (e.g. the lane change function will no longer function when the rear side sensors fail).
In scenarios where the system is unable to operate, the vehicle will trigger a TOR, requesting that the driver takes control. If the driver fails to take over within the defined time frame (10 seconds as stipulated by the NHTSA), the system will execute a risk-minimization strategy.
The risk-minimization strategy will vary according to the situation, and the vehicle may eventually come to a stop by the roadside or in the current lane (e.g. in traffic conditions where it is unsafe to change lanes). Once iNEXT has minimized risks to other road users and maneuvered the vehicle to a safe position, the warning flasher will immediately light up, and the system will automatically dial the BMW emergency call for rescue.
In the most extreme scenario, if the system detects an imminent collision with insufficient time to transfer control to the driver, the system will slow down or execute automatic evasive action without additional input from the driver.
In summary, BMW has accounted for almost all potential high-risk scenarios, answering questions such as “What should the system do if the autonomous driver falls asleep and cannot respond?”.
Other
From the iNEXT, we can feel that BMW’s R&D strategy is changing.
As we all know, the latest BMW X5 already supports full vehicle OTA updates, but BMW has not made any official announcements, with the actual iteration frequency being surprisingly low.As iNEXT becomes more intelligent and fully online, BMW has explicitly stated that, subject to compliance with the two most stringent privacy protection regulations, the European Union’s General Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA), BMW will collect global fleet data to validate functional logic security and continuously improve system algorithm robustness.
Using crowdsourced data for algorithm iteration and improvement has been a continuous research and development strategy for Tesla in the past five years.
Maintaining leadership while ensuring the highest safety standards has never been an easy task.
On this issue, BMW has gradually relaxed and achieved its goals in a roundabout way. For example, to meet the needs of high performance, information security, and continuous updates, the Intel Denverton chip runs on the Apdative Autosar heterogeneous software platform, which can only support up to ASIL-B level standard.
However, the Infineon Aurix MCU that monitors the entire system operation operates on a classic AutoSAR embedded operating system to ensure strong real-time performance and achieve ASIL-D level highest vehicle regulations standard.
In addition, the Denverton chip is not a regulation-level chip, but considering that its highest operating temperature is 85 ℃, BMW has solved the problem with liquid cooling and multiple redundancies.
Why BMW?
Let’s try to answer this question: From BMW’s autonomous driving roadmap, iNEXT’s attack is a long-planned action by BMW in collaboration with Intel and Mobileye over the past five years.
This system includes both BMW’s long-term technology accumulation (such as upper-layer algorithms, the L2-level advanced driver assistance system on BMW X5 before iNEXT was highly praised; and lower-layer support, BMW has always been one of the traditional automakers that has invested the most in automobile network security and electrical architecture among brands) and supply chain maturity, such as Mobileye EyeQ5 being mass supplied in March 2021, and Valeo’s ScaLa 2 laser radar being supplied at almost the same time. In contrast, the aggressively exited Audi A8 has become a martyr.
This reminds me of the L4-level autonomous driving planning of Ideal Auto before. To a certain extent, iNEXT in the L3-level autonomous driving market is like Ideal X01 in the L4-level market. It requires automakers to conduct years of research and development in advance and “accumulate energy,” and it also requires precise control of supply chain maturity. Only with both, can accurate and differentiated competitive advantages be formed at the appropriate node. The difference is that BMW has already reached the halfway point, while Ideal has just set off.
Of course, whether it is BMW or Ideal, they still need to continue to persist until the product is delivered in bulk and undergo consumer testing.In the five years when all car companies snubbed it, Tesla quickly filled its software and hardware capabilities through continuous iteration (in addition to redundant design, the latest FSD chip has passed the North American AECQ100 automotive regulation certification). Chinese new car companies are also ready to go.
At the time when it seemed impossible for traditional car companies to resist it, iNEXT emerged, which maintained the traditional nobility and dignity of BMW. So, will Tesla, who acts differently from others, launch a bigger counterattack?
We look forward to it, no matter how the competition changes, the ultimate winner is always the consumer.
This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.