Author: Tian Xi
Since its debut at the Shanghai Auto Show in April 2021, DJI Car has finally come into mass production after a year-long wait.
This is an intelligent driving system called “Lingxi Smart Driving,” jointly developed by DJI and SAIC-GM-Wuling, covering driving functions such as adaptive cruise control, advanced lane keeping, close-range merging response, intelligent deceleration on turns, and in parking scenarios, it provides four types of applications: assisted parking, memory parking, autonomous parking, and intelligent summoning.
According to the industry’s claim, it has already fully possessed L2+ capabilities.
At first glance, this should be a configuration that only luxury cars have.
In fact, the first model to be equipped with DJI Car System is an A0-grade electric vehicle – KiWi EV, launched by Baojun in August 2021, with a starting price of 77,800 to 86,800 yuan.
And this Lingxi Smart Driving system will be installed on the KiWi EV 2023 model.
Equipping a smart driving system on a car at such a price range is really surprising. Leaving aside the user’s demand for intelligentization, the real issue is how to make these price-sensitive people pay for the “high premium” of intelligent driving?
DJI has given another answer: self-oppression, using lower cost to achieve better functions and experiences.
A set of forward dual-camera, 4 surround fisheye cameras, 1 forward millimeter-wave radar, and 12 ultrasonic radars – these are the main sensor configurations of Lingxi Smart Driving System. Someone calculated that the cost should not exceed 6,000 yuan.
Xie Xiandi, the PR director of DJI Car, said that this will be an excellent starting point for popularising intelligent driving functions to everyone.
How did DJI achieve “both excellent performance and price-competitiveness”? What is the research and product development philosophy behind it? What is the next mass-produced model after KiWi EV?
DJI Car, surpassing the “old driver” of L2
Although DJI Car has already announced different intelligent solutions for driving scenarios such as D80/D80+, D130/D130+, and intelligent parking at the 2021 Shanghai Auto Show, it is still not as real as it is after a test drive.
The car tested by Autohome this time is a KiWi EV equipped with the D80 program. After a long 1.5-hour experience, over 80 kilometers in total, the following points were summarised:
1. Intelligent following, flexible obstacle avoidance
Intelligent following, also known as adaptive cruise control (ACC), is the basic function of L2, which DJI Car can easily and coherently execute:
- When the front car is running, KiWi EV can drive at a stable speed and keep a certain distance from it.- When the car decelerates, stops, and restarts, KiWi EV can react synchronously to ensure safe and efficient driving.
Obstacle avoidance is an essential function of the intelligent driving system. When driving on a city expressway, a small section of the lane on the right front is obstructed by a traffic barrier. KiWi EV will advance to the left in advance to bypass the obstacle and pass smoothly.
It is worth mentioning that DJI’s onboard system can recognize and avoid any static or dynamic obstacles without the need for prior learning, thereby eliminating driving safety threats.
(2) Lane keeping: as steady as an experienced driver, calmly and efficiently handles sharp curves
Lane keeping is also a common feature of L2, but it is not easy to do it well. It requires high demands for lane recognition, route planning, vehicle control, etc.
After experiencing DJI’s onboard system, the first impression it gives people is “stable”. Not only will it not have a “wandering” phenomenon, but it will also automatically decelerate when passing through sharp curves, keeping the vehicle in the center of the lane and avoiding speed reduction and rollover.
It is understood that KiWi EV can recognize and pass through sharp curves with a minimum radius of 75 meters, and it won the highest score in the continuous curve centering control among all models at the 2022 WIDC Intelligent Driving Challenge held in Tianjin.
(3) Fearless and meticulous when changing lanes, stable and flexible when dealing with cut-ins
Changing lanes by shifting the lever has already surpassed the ability range of general L2, and KiWi EV’s changing lane by shifting the lever is even more advanced than the intelligent changing lane level of emerging car-making forces.
It can accelerate or decelerate by 15km/h according to the merging lane’s situation. If the lane to be merged into is spacious both before and after, it will respond quickly to the lane change instructions and implement overtaking and lane change strategies, giving people a feeling of “bold with skill”.
Of course, the vehicle passing conditions are not always suitable after the lane change instructions are issued. KiWi EV will “carefully” observe the traffic situation, decelerate, and wait for the trailing car to pass before quickly changing lanes.
KiWi EV’s ability to deal with cut-ins is also impressive. The system can predict the motion trajectory of surrounding vehicles in advance, and it can handle cut-ins within 1.5 meters calmly and smoothly without sudden braking or stopping. On the contrary, it can be handled very smoothly.
The DJI test personnel put a cup of water in the car, and the water did not spill throughout the driving process, which is something some Robotaxis have not done well.
(4) Autonomous parking, saving time and effort“`markdown
The parking function is the core highlight of DJI’s KiWi EV, which is seamless and of high completion rate.
After activating the parking function, KiWi EV can autonomously detect and search for parking spaces, no matter from which angle it is opened, whether the target is a lateral or vertical parking space, and even if the space is relatively narrow, it can make adjustments in various directions and complete the parking smoothly.
Another amazing point is the parking out function, which is not a straight forward exit, but will tilt at a certain angle towards the co-pilot direction to reserve space for the driver to get in the car. This is very practical and convenient when there are obstacles on the left side of the garage, making it difficult to open the door.
The Invaluable Hardware Configuration: Exclusive Dual-Camera System
As mentioned earlier, all the functions above are achieved relying on just a dozen or so conventional sensors, even without the popular laser radar that has swept the industry. If it were not for firsthand experience, DJI’s explanation may be met with disbelief, just like most people’s reaction.
The reason why KiWi EV accomplishes all these functions lies in DJI’s two major advantages: a wealth of accumulation and precipitation in intelligent system development over the years, with the ability to develop and produce millions of products in R&D; and a highly vertical integration capability in software and hardware supply chains.
In fact, the sensors used by DJI may seem conventional, but they are not ordinary. The most representative one is the 2 million-pixel stereo perception camera used on KiWi EV.
It has the ability to perceive distance and depth, allowing it to accurately identify and locate moving and static targets and roadway elements, and obtain key point cloud depth information of any type of obstacle to effectively reduce the rate of omission. This also confirms DJI’s statement that:
The reason why the Lingxi intelligent driving system can recognize all obstacles is that it does not need to know what the “obstacle” actually is, it only needs to know that this is an “obstacle,” and then avoid it.
In contrast, monocular solutions are “stubborn,” and must rely on massive data training to recognize obstacles’ types. If there are scenes that are not in the database, such as overturned vehicles on the real road, monocular solutions may fail to recognize and cause traffic accidents, as seen in the frequent accidents of colliding with white box trucks.To achieve accurate and effective depth perception in automotive industry, it is not as simple as installing a binocular camera on the vehicle. There are two prominent issues with binocular vision: difficult high-precision calibration and high computational requirements.
The reason behind this is that the principle of binocular cameras is to deduce the three-dimensional information of an object by comparing the two-dimensional images captured by each camera. Environmental factors, such as thermal changes, vibrations, etc., can cause changes in the baseline and lead to a decrease in performance. Therefore, it is necessary to align the relationship between the pixel coordinate system and the world coordinate system, which is the calibration process.
The reason for high computational requirements of binocular vision is straightforward, as real-time recording of the road and surrounding environment produces thousands of gigabytes of data per second at the receiving end of the vehicle.
However, according to automakers, the KiWi EV’s computing power is only around 20T, which is far from the computer power of hundreds or even thousands of Terabytes used in the industry.
For DJI, the solution to the above problems lies in its accumulated advantages in the unmanned aerial vehicle field.
Those who have used a DJI drone know that it needs to autonomously complete flight tasks such as obstacle avoidance, hovering, and follow through its camera. This is similar to the perception system of smart cars.
DJI has developed an online self-calibration technology for unmanned aerial products, which can recalculate the correct position relationship between left and right cameras by using natural features and semantic features from real-time taken images.
“Part of this is calibration on the production line,” Xie Kuan Di added, “offline calibration on the production line affects the performance of camera hardware, which is easy to overlook.”
DJI’s self-developed binocular camera not only has rich and vertically integrated product delivery capabilities but also has mature production and manufacturing experience. For example, DJI’s unmanned aerial vehicle visual perception system has experience of producing millions of units, ensuring product consistency.
In terms of computational power, DJI optimizes the algorithms and utilizes parallel computing units to solve this problem.
It’s worth mentioning that the application of omnidirectional fisheye cameras also builds DJI’s unique advantage in automotive cameras. The industry often uses “omnidirectional fisheye” for parking functions, but DJI has developed a high-precision 3D object detection algorithm that applies to driving scenarios, supporting fisheye and pinhole camera models, meeting the detection needs of lateral vehicles and vulnerable road users, and supporting lateral functions such as change lane and overtake.## Summary
In general, based on binocular vision, online strong local perception includes Visual inertial fusion localization technology VINS, binocular BEV lane detection technology, binocular 3D object detection technology, and drivable area/obstacle detection (dense depth estimation and online reconstruction combining deep learning and geometry).
With these technologies, it is possible to accurately estimate the vehicle’s position, orientation, and shape, and to detect fine components such as wheels; it supports non-standard vehicle identification such as taxis, irregularly-shaped cars, occluded vehicles, and truncated vehicles.
In addition, it can detect large curvature corners and uphill/downhill scenes with higher precision, resulting in lower lateral error and stronger lane-keeping ability; it also supports rich semantic element detection for complex urban road structures, such as fork/merge, curbs, and safety islands.
In addition to “doing things” in terms of perception, relying on DJI’s vehicle-grade domain controller and intention prediction with multimodal decision-making control technology, the “Lingxi intelligent driving system” has high software and hardware coordination optimization ability and rich peripheral expansion and fusion ability, and can predict complex road conditions and self-learning of various driving styles.
Currently, DJI has also cooperated with SAIC-GM Wuling to build a basic infrastructure for processing massive data, and has implemented a data closed-loop system. With the continuous accumulation of data for model training, future OTA upgrades will unlock more functions for KiWi EV.
DJI’s “soul”, priced at less than 10,000?
Of course, DJI’s in-vehicle system is not perfect. In fact, no manufacturer currently dares to guarantee that their intelligent driving system is capable of handling all scenarios.
“We are still in an era of human-machine co-driving.” Xie Kandi said, which is why it is necessary to set boundaries for intelligent driving systems: what to give to machines, and what needs human intervention.
DJI proposes a set of hierarchical degradation strategies. When potential dangerous situations are automatically identified or when the system does not support certain situations, the functions will be degraded according to the degree of danger, and users will be reminded to take over.
It is understood that the lightest level is to display information and use text to signal the driver, the second level is interactive warnings and disabling some functions, and the most severe is the third level. At this level, if the driver does not respond, the vehicle will automatically slow down and come to a stop, and the hazard lights will be turned on, and the handbrake will be applied to ensure safety.
For example, in light rain, it is appropriate to increase the distance between vehicles; in moderate rain, it is appropriate to increase the distance between vehicles and turn off the lateral lane change function; in heavy rain or in scenes where the camera is dirty, affecting the imaging quality, the user will be reminded to take full control.
In DJI’s view, when the automatic driving system has not been developed or it is difficult to cover certain scenarios, the user should be truthfully informed to ensure driving safety.
It should be noted that DJI’s human-machine co-driving concept not only reflects its focus on safety, but also its focus on the driving experience.For example, in the process of changing lanes, the intelligent driving system allows the driver to intervene in the operation of stepping on the accelerator and turning the steering wheel while the function is enabled, and the system will not exit. After the lane change is completed, the system will take over again and complete posture adjustment and lane keeping.
Autocar has experienced this during the test drive of KiWi EV:
After manual intervention, KiWi EV not only does not “grab the steering wheel” with the driver, causing a “man-machine fight” situation, but also adapts to the driver’s control and autonomously takes over after the lane change is completed.
It can be said that DJI has chosen a method that can ensure automatic driving functions, continuous user experience, and safety.
This set of intelligent driving system is loaded on KiWi EV, but the final selling price does not exceed 100,000 yuan. According to this calculation, the DJI vehicle selling price is about 10,000 yuan.
Although DJI did not directly respond at this communication meeting, it indirectly indicated that it was indeed “less expensive in solutions with equivalent functions and experiences.”
The reason lies in what was mentioned earlier, for the same intelligent driving experience, DJI’s intelligent driving solution can reduce its dependence on high-cost sensors, infrastructure, and external data sources, and achieve it with lower-threshold hardware.
In DJI’s view, sensors, algorithms, computing power, and data are the “four major mountains” of the industry. To achieve the improvement and leap of intelligent driving, it is necessary to advance in these four aspects together. Only a single breakthrough, such as only solving the problem of sensors, or blindly pursuing a doubling of computing power, cannot accelerate the arrival of the era of automatic driving.
On the contrary, DJI emphasizes polishing its basics. Without pursuing hardware stacking, it uses reasonably applicable hardware configurations to create a user-friendly basic ADAS function. Without relying on external data sources, it relies solely on online perception to achieve core capabilities. Relying on highly integrated system architecture, it integrates driving and parking into one, making the functions and scenarios of intelligent driving system continuous and continuously learning and upgrading.
Of course, this does not mean that DJI has given up on the blessing of magical weapons. The LINC intelligent driving system fully supports higher-level and more expensive sensors or external data sources, but they must bring users a higher-level experience, which is what a good intelligent driving system should do.
It is understood that in addition to the D80/D80+ program applied to KiWi EV, DJI Carrying also launched the D130/D130+ intelligent driving program, covering a speed domain of 0 to 130km/h, upgrading the front dual-lens camera to 8 million pixels, and adding a rear view camera with 1 channel of 2 million pixels, which can be equipped with LIDAR and DMS.
In addition, the computing power of DJI domain controller can be expanded from 20T to 100T.For the collaboration between Wuling and DJI in the automotive industry, Zhang Dawei, Planning and Product Director of SAIC-GM-Wuling, said that the collaboration with DJI is based on scenario redefinition, unified coordination, and continuous iteration.
Specifically, Wuling acts as a super product manager and super user representative, proposing how to drive like a “veteran driver”, and discussing with DJI how to achieve it, and finding and continuously polishing solutions.
Regarding the price of this set of solutions finally applied to the KiWi EV, Zhang Dawei also mentioned that it will definitely not make consumers feel difficult to accept, “Wuling will always be worth everyone’s expectation… We will launch an intelligent driving solution that everyone can use and love.”
In addition to SAIC-GM-Wuling, Volkswagen Group is also considered as another potential customer of DJI.
Previously, CEO of Volkswagen China, Stephan Wöllenstein, said that the company is collaborating with DJI in the field of visual information processing to develop automated driving technology based on various scenarios, and plans to be installed on vehicles from 2023 to 2024.
However, for a long time, news between DJI and Volkswagen has fallen silent, which inevitably arouses people’s imagination: where will the DJI D130/D130+ solution be used on which model?
Nevertheless, the truth should be revealed soon: after all, the first car with DJI’s intelligent driving system for mass production has arrived, would the second one be far away?
This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.