Author: Tian Xi
In 2022, in the field of intelligent automobiles, almost everyone is talking about “integrated driving and parking”.
“Driving and parking share the same domain controller, achieving deep sensor reuse and shared computing resources. While improving the user’s intelligent driving experience, it also helps the host factory reduce costs and increase efficiency, greatly improving the latter’s development efficiency…” This is almost the common parlance of companies that do “integrated driving and parking” autonomous driving.
However, in reality, the performance of such solutions varies greatly. Most so-called integrated driving and parking solutions simply integrate hardware into a “box”, without any change in functionality and no significant improvement in user experience.
In the face of such varying effectiveness, it even makes one wonder whether different companies’ “integrated driving and parking” refers to the same thing.
Therefore, some industry insiders have proposed different forms of integrated driving and parking based on long-term thinking and practice, using 1.0, 2.0, and even 3.0 to distinguish the huge differences in essence among different solutions, and explaining why each company’s integrated driving and parking differs greatly in terms of functionality, experience, hardware, and cost.
In this regard, leading companies in the production of autonomous driving may have the most say.
Recently, BitAuto interviewed Nullmax founder and CEO Xu Lei at the delivery site of the first single TDA4VM integrated driving and parking project in China, and discussed with him the current progress of integrated driving and parking in mass production.
As an entrepreneur with a Silicon Valley background, he has the usual rigorousness of technology enthusiasts. He divides “integrated driving and parking” into four forms, with the highest-level 4.0 form consisting of an integrated domain controller, a single SOC system, and continuous deep reuse of chips and sensors, with no restrictions on NOA functionality.
Currently, Nullmax is one of the few companies in the industry that can provide integrated driving and parking 4.0 solutions. It has developed MaxDrive, a platform-based integrated driving and parking solution that covers different computing power, and will be the first to complete a series of mass deliveries on the TDA4VM chip.## From 1.0 to 4.0, the integration of driving and parking systems suffered mixed success.
Currently, most car models on the market still use a separated domain controller, where the systems for driving and parking are separated. The driving function can only call on the chips and sensors responsible for driving, like front camera and millimeter wave radar, while the parking function can only call on the chips and sensors responsible for parking, like fisheye camera and ultrasonic radar.
This means that when one system is running, the hardware for the other system will be idle.
This situation is referred to as the 1.0 form of driving and parking integration, where not only are the number of electronic control units (ECUs) greatest, their cost is also very expensive, according to Xu Lei.
The essence of driving and parking integration in 1.0 form is just that the functions of driving and parking exist on the same vehicle.
The 2.0 form, on the other hand, is what has been mentioned before, which integrates the SoCs (driving SoC and parking SoC) responsible for different functions into one domain controller in a “carnival in appearance but confusion in reality” manner, such as the dual chip combination of “1 EyeQ4 + 1 TDA4,” which truly reduces one domain controller and cuts the hardware cost by around 30%. However, it belongs to a pseudo driving and parking integration architecture, where the computing resources are not shared truly and communication delay is easy to occur, which is not conducive to data fusion processing and the improvement of autonomous driving performance.
The driving and parking integration in 3.0 form goes further by reducing one chip besides reducing the domain controller and directly using a single SoC system. This reduces the hardware cost by 50%, but a new problem arises:
The sensors cannot be deeply reused, and their functions are limited to basic ADAS.
That is to say, in the 3.0 form, only part of the sensors can be activated, while the rest of the sensors are still idle on a time-sharing basis, which means that more performance space can be explored under the same sensors.
The reason for this is that some autonomous driving companies cannot process a massive amount of data from various sensors that are being input simultaneously on a single SoC with limited computing power.”This leads to the use of only front cameras and other sensors during driving, and only fisheye cameras for perception during parking. While they may appear combined physically, they are still separate in functionality. A true integration of driving and parking not only reduces overlapping hardware and saves costs, but also achieves higher performance.
According to Xu Lei, while driving on the road, the driving system should use fisheye cameras to improve the close-range perception of the surrounding 360° area, enhancing driving performance in congested following and cut-in situations. For example, fisheye cameras can be used to detect changes in the adjacent driving area, so that the system can avoid collision in advance when a truck passes by.
When driving in a parking lot, the system can directly call the front camera to enhance the forward perception of the vehicle, recognize obstacles on the road, and improve the driving speed during the parking process, “such as reaching more than 10 kilometers per hour.”
These functions can only be achieved in a single SoC, in order to truly achieve the integration of driving and parking in the 4.0 form.
The challenge is obvious, as it requires autonomous driving companies to deeply understand the entire process of autonomous driving, build efficient and high-performance overall architecture, possess strong software algorithm technology, and especially engineering capabilities to complete the actual development and implementation of the entire solution.
This means that 4.0 form of integrated driving and parking is very rare.
From the delivery progress perspective, companies that can achieve the landing of self-developed integrated driving and parking solutions like Nullmax are indeed few and far between.”
Xu Lei had worked for Tesla before he started his own business. He was involved in the development and implementation of the TeslaVision deep learning network deployed in Autopilot 2.0 system, along with another founder, Song Xinyu, who participated in the research and development of Autopilot 2.0 system in both software and hardware aspects.
Therefore, Nullmax has had a strong software and hardware gene since its inception.
For example, in the automatic driving SoC chip, multiple modules are usually integrated, such as CPU, AI chip (GPU/FPGA/ASIC) and deep learning acceleration unit (NPU). How to match perception, fusion, planning, control, map, positioning and other functions with the most suitable computing unit is a challenging problem.
Based on the rich development experience on the heterogeneous platform, Nullmax efficiently deploys deep learning models and assigns different tasks to enable each module to perform optimally.
“Multiple heterogeneous computing resources can be overlapped and processed in parallel, which improves resource utilization, throughput, acceleration ratio, and frame rate of system data processing.”
In addition, it is also necessary to consider specific issues such as how signals and data are transmitted between modules, and how to respond to module or system failures.
“In short, how the entire architecture should communicate, schedule, and synchronize.” To this end, Nullmax has designed an efficient system framework that can connect different modules to achieve the most efficient collaboration, scheduling, and fusion.
It can be said that the actual mass production difficulty of integrated parking and driving, especially the engineering deployment of modules and module-to-module communication, has been underestimated in the past. After all, the 4.0 solution of integrated parking and driving that achieves cross-level experience with the same level of configuration cannot be achieved overnight.
Software platformization, Nullmax targets million-unit vehicle salesUsing Nullmax’s single TDA4VM solution as an example, with only 8 TOPS of AI computing power, it can provide a series of driving and parking functions such as navigation assistance, high-speed driving assistance, congestion following assistance, and memory parking.
In 2021, this solution won a fixed-point project of a well-known automaker, using a sensor configuration of 2 forward-facing cameras, 4 fisheye cameras, 5 millimeter-wave radars, and 12 ultrasonic radars, deployed on the TDA4 autonomous driving domain controller. It can not only provide rich and safe functional experiences but also take hardware costs into account.
Now, the Nullmax single TDA4VM solution for driving and parking is about to be finally delivered, and the platformization of mass-produced models is expected to reach hundreds of thousands.
Based on this, Nullmax has also developed a standard version of driving and parking based on dual TDA4VM chips, which is the industry’s first exclusive fixed-point mass-produced solution for driving and parking with dual TDA4VM chips.
With 16 TOPS of computing power, it can handle up to 11 camera groups with large-scale visual perception such as front, side, and surround view, and perform sensor fusion of millimeter-wave radars, ultrasonic radars, and other sensors to complete various tasks required for intelligent driving functions such as high-speed navigation, congestion following, autonomous parking, and memory parking, further optimizing the driving experience.
In addition to the low and medium computing power layout, Nullmax is also developing a high computing power driving and parking smart driving solution based on the NVIDIA Orin chip platform.
The “Integrated Parking” project, based on the standard version Orin chip platform, is the first domestic mass-production application that covers highway, urban, and parking scenarios, providing all mainstream functions including navigation assisted driving, congestion follow-up, parking assistance, and routine ADAS. It is expected that the first car model of this platform project will be launched and delivered in 2023, and the final total production scale will be close to one million vehicles. It is understood that Nullmax will use its fully self-owned intellectual property closed-loop data toolchain to help create an automated closed-loop data platform that supports the addition of more innovative features and automatic driving function OTA upgrades. Nullmax can launch differentiated integrated parking solutions based on different chip platforms such as Texas Instruments TDA4 and NVIDIA Orin, and achieve different levels of intelligent driving experience. In addition, many production projects are also platform projects involving multiple vehicle models. “If doing integrated parking on one platform is considered difficult, expanding to multiple platforms is even more difficult,” an industry insider pointed out. The architectures are vastly different between different platforms, and it is difficult to simply migrate and replicate algorithms. It is precisely for this reason that many autonomous driving companies either spend a lot of energy on adapting to different platforms with low efficiency and high investment, or only focus on one platform with limited scalability. Nullmax does not have a “multiple choice” mindset, but rather focuses on having “all that is good”. Its secret lies in platforming software and launching the MaxDrive platform that covers the entire set of automatic driving application layer algorithm modules and middleware systems. Xu Lei explained the concept of “platforming” using a universal chassis in hardware.
Polishing the universal chassis and developing it into pick-ups, SUVs, and box trucks according to various needs is platformization in hardware.
“Software platformization is similar, and in intelligent driving, it is shown as using a set of software to adapt to different numbers of sensors and chips with different computing capabilities.”
Xu Lei likened Nullmax’s MaxDrive platform to a “trunk”. Whether development is based on what kind of computing power, the “trunk” remains the same, with only differences in the “branches”.
“For example, to detect a car, under the condition of high computing power, we may use deep learning, Transformer, and other technologies to extract image features to determine if it is a car, while under low computing power, there are not so many ‘branches’ to do this.”
The advantages of “software platformization” are obvious.
Firstly, it greatly improves business efficiency, without the need to start from scratch every time for adaptation, avoiding fragmented development;
Secondly, from a commercial perspective, it greatly saves the time and cost of implementing solutions for host factories; more importantly, the experience of development on the same platform can be accumulated and iterated, which can promote technological breakthroughs.
Based on this platformization capability, Nullmax not only created the Xiaoboyitiji intelligent driving solution based on Texas Instruments TDA4 chip and NVIDIA Orin chip platform, and completed staged delivery, but also reached strategic cooperation in autonomous driving with Horizon Robotics, Black Sesame Technologies, and others. It is hoped to upload the Xiaoboyitiji function to more models in the future.
Behind large-scale production, cost-effectiveness is the key.The advantage of Nullmax lies not only in its software platform with excellent scalability and ultimate performance, but also in its ability to provide highly cost-effective solutions, which is particularly appealing to OEMs.
This means a great appeal to OEMs, especially in some mid-to-low-end models where cost control is highly valued. The emergence of the parking and cruising integrated function is largely driven by the OEMs’ demand for cost reduction and efficiency improvement.
“With the same cost, we can achieve better performance; with the same performance, we can achieve lower cost.” Xu Lei revealed that in the visual perception algorithm solution delivered to a top-notch car manufacturer, Nullmax, based on a single TDA4VM platform with 8 TOPS, connected 4 surround-view cameras and allocated 20k computational power on the CPU, but actually used less than 4k computational power to achieve the same performance.
It is also worth mentioning that the 4 cameras can not only achieve a detection speed of 20 FPS without frame skipping or loss, but also detect every frame, with a delay of less than 100 milliseconds.
Under the configuration of dual TDA4VM, coupled with the sensor combination of 5R11V, Nullmax can achieve the effect that other solution providers can only achieve under 30+ TOPS of computational power with 16 TOPS of computational power.
At present, among the many mass-produced projects of Nullmax, a part of them provide L2+ intelligent driving solutions for economical models, and the algorithm is carried on the TDA4 of the third intelligent driving domain controller IPU02 of Desay SV.
IPU01/02 is a high-cost-effective solution under Desay SV, which provides certain functions of ADAS applications within a limited cost range to help automakers quickly realize function integration.The IPU01 was the first mass-produced solution with annual shipment reaching millions, while the IPU02, launched in 2021, is the current focus of development and promotion for Desay SV.
Desay SV Vice President Li Lele has claimed that the IPU02 has a wider range of customers than the IPU01. Apart from the existing clients, IPU02 has entered the supporting system of automakers such as SAIC, Great Wall, GAC, GM and new forces in automobile manufacturing. Similarly to the status of IPU01 in 2017, with the development of integrated parking, more projects are expected to be implemented. By 2022, the projects will begin to go into trial production, and by 2023, many of these projects will be able to be produced on a large scale.
This is also one of Nullmax’s plans, with the aim of occupying as much of the market as possible by leveraging the rapidly-rising TDA4 platform domain control. “On cars with a price range of 100,000 to 150,000 yuan, I hope that our integrated parking solution will become a standard.” Xu Lei straightforwardly revealed Nullmax’s goal.
Moreover, the deeper purpose is to accumulate a large amount of data through being carried on more car models and iterating algorithms, eventually reaching the capability of Level 4 autonomous driving technology.
Similar to the mainstream view in the industry, Xu Lei prefers a progressive development route rather than a giant leap. “Ultimately, autonomous driving is data-driven. Only by having more cars on the road, can technology be tested and matured, and generalization capabilities be continuously improved.”
Except for the low-power row parking integrated in the layout, Nullmax is also implementing higher-level functions based on large computing power, such as providing full-scene intelligent driving experience covering urban roads like Tesla FSD with more than 100 TOPS of computing power, without relying on high-precision maps. Nullmax calls it USD, and this feature will be mass-produced in 2023.
As we can see, the parking and cruising integration is gradually integrating into people’s daily lives in different sub-markets.
Meanwhile, Nullmax, as a representative of the autonomous driving industry, is making secret efforts to compete for the entrance to intelligent driving.
“The window period is just these two or three years, and time won’t be too long. We are very confident,” said Xu Lei with a relaxed expression.
This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.