On March 9th, NIO held its first user meeting with the theme of NIO Pilot at the Beijing Delivery Center. Jamie Carlson, Vice President of Autonomous Driving at NIO responsible for NIO Pilot’s development, traveled from Silicon Valley to attend the event.
After the meeting, NIO also held NIO Pilot ACC test driving experience, and many users participated in the test drive. The interested NIO users can further understand by visiting NIO App.
Yesterday’s topic focused on the largest challenge recognized inside and outside of NIO: software issues. More specifically, NIO Pilot, which concerns over 10,000 ES8 owners and a larger scale of ES6 owners who are about to be delivered.
On December 18th, 2017, the NIO ES8, which was officially released, was equipped with the “world’s first installation of the Mobileye EyeQ4 chip.” 15 months later, the world’s first installation of the Mobileye EyeQ4 is still displayed on the official website. “Continuous learning, rapid upgrading” is also ambiguously written there. However, only basic active safety functions such as forward collision warning (FCW), lane departure warning (LDW), and automatic emergency braking (AEB) are open to NIO Pilot.
Yesterday’s simple workshop had the following attendees:
-
CEO William Li
-
President Qin Lihong
-
CIO & North America Director Ganesh V. Iyer
-
Vice President of Autonomous Driving Jamie Carlson
-
Vice President of AI Algorithm Chris Pouliot
Clearly, the relevant management team attended collectively, indicating that there is progress to share in the next stage:
- April 2019: Adaptive Cruise Control (ACC) push.* June 2019: High-Speed Automatic Assistance Driving (HP), Congestion Automatic Assistance Driving (TJP), and Automatic Parking Assistance (APA) are launched.
However, NIO ES8 car owners who have been waiting for nearly a year want to know why NIO Pilot’s ACC R&D cycle is as long as 18 months while L2 autonomous driving systems are exploring the 100-150k price range today.
Why Self-developing?
To some extent, Jamie Carlson is responsible 🙂
In August 2016, NIO was still promoting the use of Bosch L2 autonomous driving suite in the ES8. In October 2016, Jamie Carlson left the Apple Autonomous Driving Project Project Titan to join NIO as Senior Technical Director.
Earlier, Jamie was the firmware manager at Tesla Autopilot and one of the founding members of the Autopilot department.
After Jamie joined NIO, Li Bin asked him a question: NIO expects to start delivering ES8 in March-April 2018. Starting from now, can we independently develop a set of automatic driving assist systems from scratch in time?
Jamie’s reply was: OK.
But as we have all seen, there has been a long delay in the delivery of ES8, which is much later than internal planning. The first batch of ten ES8s was not delivered until May 31. The actual volume delivery was not until Q3 2018. Even so, NIO Pilot has not yet released the L2-level feature push.
The real challenge lies in the difficulty of self-development.
Jamie shared a PPT page specifically on why self-development.
In fact, the autonomous driving assistance systems from top-tier players such as Bosch have undergone extensive testing and have strong reliability and stability. The only problem is that the entire development of this system from bottom to top is led by Bosch. Only the final adaptation and road testing are done by OEM.If NIO had stuck with Bosch’s solution without knowing anything about the know-how related to the system, it would have posed significant challenges for adding user-oriented functions and iterative development after delivery. A system that cannot be iteratively improved will completely lose its vitality, contradicting the principle of intelligent cars evolving continuously based on OTA updates, and will not be able to provide differentiated advantages as compared to “non-smart cars.”
Even if Bosch still provides complete post-technical support and maintenance, NIO’s subsequent feature updates will still have to rely on Bosch’s development progress, and it will be difficult to show differentiated product strength in the face of market competition.
If the supplier’s solution is inconvenient in many ways, why do carmakers not choose to develop their own solutions? In reality, among the leading new car manufacturers, only NIO, XPeng, and C&J are still insisting on independently researching and developing automatic driving assistance systems.
This is because independent research and development has a high threshold, involving many cross-disciplinary technological challenges. This is also the main reason why the development of NIO Pilot has been repeatedly postponed.
Specifically, the development of NIO Pilot involves multiple technological domains, such as perception, path planning, decision-making, control, simulation, cloud, high-precision maps, and hardware. Choosing independent research and development essentially means taking the technological discourse power back from suppliers to car manufacturers. NIO independently developed many of the underlying drive components of NIO Pilot, and many basic software codes were written line by line from scratch by Jamie’s team.
Tesla started this trend, and today NIO is the smart electric car manufacturer that has gone the furthest in independent research and development after Tesla.
According to NIO’s progress, in June 2019, NIO Pilot had the ability to support L2-level automatic driving assistance system, but from a user experience perspective, this is not a point to be particularly proud of. However, NIO has a profound understanding of the entire system and has complete know-how about the technical details. In other words, Jamie led the NIO Pilot team to rebuild an Autopilot from scratch in 18 months, and Autopilot in itself has a high technical threshold.
Jamie said in the Q&A session that due to the first-mover advantage, Tesla did indeed do better, but beyond Tesla, NIO is confident in achieving a very high level of performance.
Mobileye EyeQ4?A typical argument is that Tesla kicked out Mobileye and still built Autopilot 2.0, while NIO used the Mobileye solution and made it so slow.
This is also what we are curious about. What is the proportion of Mobileye in the automatic assistance driving system? Why did NIO not adopt the technology route with visual perception ability as the main? In Jamie’s view, choosing EyeQ4 is the best choice for NIO.
Tesla Autopilot 2.0 has built up core visual perception capability from scratch, and the Autopilot team has indeed made good achievements. However, we must know that the Tesla Vision machine vision tool is built by a continuous stream of top-level technology talents who are proficient in computer vision or artificial intelligence. Over the past 18 months, many experts have joined the Autopilot team, and then left because of the enormous technical challenges.
According to Jamie, for a considerable period after the release of Autopilot 2.0, Tesla was struggling to build up visual perception capability. The path planning and decision control team also had no progress for a long time because of the stagnation in perception.
If NIO imitates Tesla to establish completely autonomous visual perception capability, the progress speed is undoubtedly slower, and this is not the best choice for NIO owners who are eagerly anticipating it.
The second reason why NIO chose EyeQ4 is that there are differences in perception technology routes between NIO and Tesla. Elon Musk believes in first principles, insisting that full coverage cameras, ultrasonic sensors, and a front radar can achieve Level 4 autonomous driving technology. The NIO Pilot team has arranged for complete coverage of cameras, ultrasonic waves and radar sensors for the ES8 and ES6. In NIO Pilot 2.0, we will also see the presence of Lidar.
In short, camera-driven visual perception capability is more important in Autopilot than in NIO Pilot.This directly leads to Tesla insisting on mastering visual perception capability itself. However, for NIO, visual perception is only a small part of the technology. NIO cooperates with the top supplier Mobileye in this field and can focus more on the fusion processing of radar data and ultrasonic sensor data.
Mobileye EyeQ4 is a black box for car companies. It supports recognition of cars, trucks, motorcycles, pedestrians, and even smaller obstacles. But car companies do not know how the specific recognition algorithm works. They can only obtain the signal output by Mobileye.
However, this does not mean that NIO’s visual perception capability is entirely provided by Mobileye. For example, although EyeQ4 can recognize pedestrians, it does not support recognition of the pedestrians’ direction or posture. The image signal transmitted by the camera is divided into two signals, which enter the EyeQ4 chip and the central controller chip of NIO Pilot, respectively. The latter will combine the output result from EyeQ4 with NIO’s computer vision recognition capability to jointly analyze and predict the next action of the pedestrian.
This is just a small example of the perception differentiation. Considering the broad scope of technology involved in NIO Pilot, NIO dominates almost all of the software building except for the visual perception. Choosing Mobileye does not affect the development of NIO’s core competitiveness.
For NIO, Mobileye is a very important partner. NIO is the first car company in the world to adopt the EyeQ4 chip, and will also strive for the launch of the EyeQ5 chip in the future. It can also be seen that NIO is tracking and enthusiastic about the latest chips.
Automatic driving is the key to NIO’s future competitiveness. Li Bin confirmed that NIO will skip Level 3 autonomous driving technology and directly develop Level 4 autonomous driving technology. The research and development of NIO’s second-generation autonomous driving platform, NIO Platform Two, has also been launched.
When it comes to the delay of NIO Pilot, it is difficult to avoid discussing the previous adjustments of NIO’s North America management. In that adjustment, NIO co-founder, Chief Development Officer, and North America CEO Padmasree Warrior resigned. Has the adjustment of senior management to some extent affected the progress of research and development?In fact, Jamie himself was recruited by Padmasree Warrior to join NIO, but he maintained a clear attitude towards Warrior’s departure: sad emotionally, but continuing to work professionally. Jamie stated that NIO autonomous driving team has developed into a mature team with 360 people. Anyone can decide whether to stay or leave, but it won’t affect the team’s efficiency.
Another issue is whether the NIO Pilot team led by Base in North America is not as efficient in product iteration as other new car companies rooted in China.
Jamie mentioned his colleague, NIO’s Intelligent Driving CTO, Jianyong Zhang. The team led by Zhang is based in Shanghai and is responsible for the development, integration and validation of NIO Pilot feature software. However, Silicon Valley is a truly top talent gathering place, and NIO needs to absorb top talent. The only problem to overcome is the time difference. Currently, NIO Pilot team operates in a Silicon Valley + Shanghai mode, crossing global time zones and running all the time.
Jamie mentioned in today’s speech that Tesla has already achieved a mode of collecting road test data through globally delivered vehicles, and NIO has made plans. NIO is continuously tracking the impact of the shadow mode, which greatly affects user experience and algorithm improvement, as well as upgrading the next generation of chip sensor hardware.
But currently, NIO needs to first solve the software problem and needs strong software capabilities.
- NIO ES6 debuts, let the bullets fly for a while | 42HOW on-site report* Is Termination of the Construction of NIO’s Shanghai Factory Bad News?
This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.