Q&A Section after Nissan’s ProPilot Launch Event

On Monday of this week, Nissan officially released the new generation of ProPilot perception hardware and demonstrated the system’s capabilities. This Q&A session with the domestic media, after the launch event, covers questions ranging from the system’s timeline for entering the Chinese market to differences with other brands’ autonomous driving technologies and understanding of cost and system stability issues. The following is the complete content.

Interviewees:

• Takao Asami: Senior Vice President of Nissan Motor Corporation
• Toru Futami: Head of AD/ADAS (Advanced Driver-Assistance/Advanced Driver Assistance System) Technology Development Division of Nissan Motor Corporation

1. Is Nissan’s advanced driving assistance system mainly used in city roads or highways? What is the earliest release time and installation time for this new technology?

Takao Asami: Thank you for your question. Nissan has sold over one million vehicles equipped with driving assistance system. ProPILOT, Nissan’s ultra-smart driving technology launched in 1996, is mainly applied to highways. The driver needs to hold the steering wheel, and the system won’t work if the driver leaves it. In 2019, ProPILOT 2.0 was released, which supports drivers to drive with hands off the steering wheel on highways.

This time, we are introducing automated emergency avoidance technology, which will significantly improve safety performance whether it’s assisted or automated driving. As for future commercialization, this technology is suitable for both autonomous and assisted driving vehicles, or for both. We hope to complete the development of ground truth perception technology by the mid-2020s, which will first be installed on some new models and will eventually be equipped on all new models by the 2030 financial year. At that time, this technology can significantly improve safety performance, so we are also considering applying it to our driving assistance system.

2. What are the differences between Nissan’s autonomous driving technology and commonly used technologies on the market (such as Tesla) in terms of hardware and software? What is Nissan’s future technology roadmap, and which products will use this technology?

Takao Asami: Thank you for your question. From an appearance standpoint, sensor configurations vary among different automakers, and the system’s ability to handle different traffic scenarios will also be different. As for Nissan, when it comes to highways, for example, we design appropriate sensor algorithms and control system driving structures based on vehicle speed and whether the driver holds the steering wheel during technology development.Many companies use similar parts, and I believe the differences lie in the design concepts or structural aspects. Nissan has introduced over one million vehicles equipped with driving assistance technology to the global market and has achieved a certain degree of market awareness and penetration.
Toru Futami: I think the driving force behind the development of this new system is some traffic accidents involving driving assistance systems that have occurred today. Many of them are due to the fact that the accident scene exceeds the system’s capabilities and the system cannot provide assistance in these scenarios. Therefore, how can we cover and resolve accidents in these scenarios with technology? We believe that safety must be improved, so we have developed this technology.

3. In Nissan’s LIDAR scheme, which partner was chosen? We saw in the information that Nissan’s LIDAR arrangement is placed on the top, which looks like a LIDAR. How many LIDARs are actually used? In China, some manufacturers choose to install two or even four LIDARs. What is the logic or idea behind Nissan’s choice of this scheme? Thank you.

Takao Asami: Thank you for your question. We will use one LIDAR, and our partner is Luminar, but Luminar has not yet been confirmed as Nissan’s mass production supplier, but as a co-development partner.

Toru Futami: Nissan will only install one LIDAR on the top of the vehicle. The LIDAR detects the space around the vehicle and the vehicles driving on the road, i.e., moving objects. The road structure and surrounding buildings are essentially stationary, and the surroundings change when the vehicle is moving. The LIDAR can perceive the changes in the scene and make three-dimensional measurements to convey to the vehicle.

Therefore, installing a LIDAR on top of the vehicle can basically grasp the spatial structure around it. The radar and camera are used to understand the surrounding traffic information. The camera is used to understand the scene’s meaning (context) and identify the specific target. Neural networks are often used to understand the scene’s meaning, and to look around to understand its meaning.

The radar can accurately grasp the speed and direction of moving objects around it. The LIDAR can accurately capture spatial structure, and then the camera can understand the meaning of obstacles ahead. The radar provides accurate motion status. Through the combination of these three, it ultimately achieves similar human cognitive ability.

4. Currently, millimeter-wave radar and vision sensors cannot accurately identify stationary objects, so when adaptive cruise control is turned on while driving on highways, it is difficult to accurately identify stationary objects, which can lead to traffic accidents. In the face of this situation, does Nissan have any solutions? Thank you.### Q&A

浅见孝雄: Thank you for your question. Currently, Nissan’s driving assistance technology is at level L2, where the driver needs to be in control of the vehicle. For vehicles that require steering, if the driver removes both hands from the wheel, the system will issue a warning, and in severe cases, the system will automatically judge and brake the vehicle. For systems that allow the driver to remove both hands from the wheel, the driver monitoring system will monitor whether the driver is looking ahead. If the driver is not participating in the driving, the system will stop such behavior.

In the future, the ground truth perception technology introduced here can accurately identify the position of static objects and avoid collisions while driving.

饭岛彻也: First, I’d like to mention the identification of static objects. For currently available production cameras and radar, the detection range is relatively limited, and they can detect stationary objects within a range of about 60 meters. However, if it is at a distance of 100 meters, 150 meters, or even 300 meters, it is difficult for cameras and radar to identify stationary objects. Therefore, when driving on highways, cameras and radar cannot identify stationary objects, which can lead to accidents.

The laser radar introduced this time aims to be able to detect stationary vehicles at a speed of 130 km/h and avoid them safely, thus capturing stationary vehicles and objects beyond 300 meters.

Unlike radar and cameras, laser radar cannot recognize the meaning of the target object. However, if there are raised objects on the ground, laser radar can accurately determine the shape of the object and determine whether the vehicle can cross the obstacle. If the laser radar detects an obstacle in the far distance and cannot cross it, in this dangerous situation, the risk can be reduced by changing lanes. By taking these combined measures, higher safety performance than now can be achieved.

5. Regarding the positioning of laser radar in the auxiliary driving system, does Nissan tend to use laser radar as the main perception source or as a perception correction method for the auxiliary driving system? In addition, depending on the different perception schemes of the auxiliary driving system, there are roughly two factions: the pure visual perception route and the multi-sensor fusion route. For the multi-sensor fusion route, in special cases, different sensors may feedback different, even contradictory information to the system. How does Nissan handle this?

浅见孝雄: Thank you for your question. As far as I understand, laser radar plays a major role in accurately measuring the distance between vehicles and objects. However, it also needs to make correct judgments and controls based on sensor standards combined with cameras and radar.# Translation

Tetsuya Iijima: The relationship among cameras, radars, and lidars is not about which one is the main sensor and which one is the auxiliary, but rather they complement each other by combining their most reliable parts to enhance their collaborative effect. The purpose is to improve the performance that cannot be achieved by a single sensor. Therefore, the main function of lidar is to accurately grasp the spatial structure and shape of objects, that is, to accurately draw the coordinate system around a vehicle.

However, the coordinate system accurately drawn by lidar is monochromatic and has no meaning in a black and white world. Hence, it is necessary to use cameras with neural networks to enhance it and add color. In addition, radar is added to capture the moving, speed, and distance information it excels in. Combining the strengths of these three sensors achieves perception capability that is very close to that of humans, which cannot be achieved by a single sensor. This is our goal.

6. Previously, it was understood that Nissan planned to equip all vehicles with lidar by around 2030. How will Nissan reduce the cost of lidar? Lidar is vulnerable to snow or foggy weather; how will Nissan ensure that the functions announced today work in all weather conditions?

Takao Asami：Thank you for your question. Indeed, cost is a significant challenge. Currently, we are researching the automation of emergency avoidance operations and evaluating the performance of lidar and developing algorithms. Regarding cost reduction, I think lidar may undergo several upgrades and iterations until it becomes popular in the market and its cost can be significantly reduced. I think it is possible to achieve cost reduction by 2030.

Although Nissan does not produce lidar, considering the development process of new technologies so far, I think that is the time horizon. Regarding weather, it is certain that it can be affected by adverse weather such as rain. However, if this technology cannot be used in a practical range, it cannot be commercialized; therefore, we are also trying to solve this problem.

Tetsuya Iijima: Regarding the decrease in lidar performance in bad weather, it is an optical sensor, so if there is any dirt, bird droppings, etc., in front of it, it cannot function correctly. Therefore, rain wipers, cleaners, or washing devices are necessary.

However, for cloudy weather, since the lidar uses a very thin laser beam for scanning, it does not mean that every single water drop that vertically falls into space will block all laser beams every time. It fills the scene in front of it over several cycles, meaning it flexibly renders the images behind rain droplets. Therefore, currently, lidar is stronger than radar and cameras in dealing with performance degradation in bad weather conditions.7. Nissan plans to equip its next-generation laser radar system on all new models by the FY 2030. What technical means will Nissan use to maintain its competitive advantage in assisted driving over the next eight years? Can Nissan’s existing assisted driving technology cope with the impact from new Chinese car manufacturers? Thank you.

Takao Asami: Thank you for your question. We have always been paying attention to the developments of other companies. The technology we have just released is still in the R&D stage and will enter the mass production stage later. Of course, in two, three, or four years, there may be new technologies that become mainstream. Therefore, eight years from now, there may not be only this mature technology.

Nissan has been developing assisted or automatic driving technology since 1990 and has maintained a leading position in technology for nearly 30 years. We are also accelerating our R&D efforts and will continue to do so in the future.

Toshiyuki Shiga: This time, we integrated our laser radar technology with Ground Truth Perception to achieve real-time road sensing. Nissan plans to complete the development work in the mid-2020s. We have already achieved hands-free driving on highways. In common applications, I believe we have reached a very high level.

I think our competitors will continue to develop these technologies. Technologies such as high-definition mapping HD-Map, localized GPS navigation systems, cameras, and radar will gradually mature, and performance will gradually improve. We will continue to improve and provide systems with high safety performance and ease of use.

8. Chinese car companies have already achieved mass production of laser radar-equipped vehicles. What does this mean for the entire industry, and will this have any impact on Nissan’s smart driving technology R&D strategy?

Takao Asami: Thank you for your question. Regarding the current use of laser radar, I think this is valuable. As various companies begin to adopt laser radars, this will greatly promote the popularization of laser radars for suppliers. Therefore, I think this is a very welcome move. In this way, the performance of laser radars will improve, production will increase, and they will be further popularized, and can be applied to various IoT scenarios, which we are also very happy to see.

Toshiyuki Shiga: The prototype car we introduced this time is mainly developed for emergency evasion performance. But I think laser radar has the potential to improve various performance. I think companies will use their own wisdom to think about how to use laser radar, such as further improving ease of use in common applications. While referring to this information, we will further improve emergency evasion performance and the performance of common applications.9. The road conditions in China are relatively complex, and there is a big difference between the technical theory and the actual road conditions. From a technical perspective, I am very interested in knowing whether a large amount of data collection and calibration is needed in China. In terms of ensuring the stability of the technology and the collection of the Chinese road environment, how does Nissan adapt to the Chinese traffic environment? Thank you.

Response from Takeshi Asakawa: Thank you for your question. In terms of the possibility of collision with objects, it may be similar all over the world, but the road conditions in China do have its own special characteristics. Nissan has established a research and development base in Shanghai, and the team there is currently putting in a considerable amount of effort to carry out simulated analysis based on the existing Chinese traffic environment, in order to verify whether this technology can be applied to Chinese road conditions. This is very necessary, and relying only on theory is not feasible; a large amount of data must be collected.

Moreover, I also believe that it is necessary to carry out verification experiments on site, so I hope that the cutting-edge team in Shanghai can carry out development in an exploratory manner.

10. How does Nissan view the current and future situations, such as the impact of the policy environment in China and Japan on the application of Nissan’s autonomous driving technology within the next three to five years? Will Nissan be restricted by policies of China and Japan? Thank you.

Response from Takeshi Asakawa: Thank you for your question. I think that future product development or product launches may be affected by regulations. However, as far as I know, the situation in different countries or regions is very different. For example, in Japan and the United States, the function of drivers’ hands leaving the steering wheel has been legally recognized, but there is no such regulation in Europe or China.

As for the development of L4 autonomous driving technology, the Chinese government is taking the lead in actively carrying out research and development, including the gradual improvement of related infrastructure. However, in Japan, there has not yet been any specific policy for the implementation of autonomous driving and the importation of vehicles, including infrastructure.

Considering the policies of each country and the different technical standards in different regions, I believe that it is necessary to balance these two factors to promote the development of technology.

This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.