This article is the third in a series of popular science articles on FMCW LiDAR by Su Qingtao. Interested readers can click hyperlinks to check out the first two pieces: FMCW LiDAR popular science (Part 1): basic concepts, technical routes, advantages, controversies and misunderstandings; FMCW LiDAR popular science (Part 2): lasers, detectors and scanning.
Main Challenges
Higher Requirements for Silicon Optoelectronic Chips than for Optical Communication Products
Despite the fact that the technological principle of FMCW LiDAR is similar to that of optical communication products, its requirements are much higher.
Yao Jian, CEO of Optowell Technology, said that in optical communication products, there is no loss of optical waves (professionally known as “insertion loss”) during transmission through optical fibers. In contrast, in FMCW LiDAR, optical waves have to be transmitted in the air, making it difficult to avoid insertion loss, especially in harsh weather conditions.
What are the adverse effects of insertion loss? According to an executive of a certain ToF LiDAR manufacturer, if the amount of light emitted or received is reduced, the detection range will be shortened. In the field of optical communication, a higher degree of insertion loss is acceptable, because a booster can be added to amplify the light, which can then be transmitted forward. Therefore, the acceptance degree of local insertion loss is relatively high. Compared with this, FMCW LiDAR, without an optical amplifier, has a low tolerance for insertion loss.
Therefore, in order to improve the imaging effect, the requirements for insertion loss indicators of silicon optoelectronic chips in FMCW LiDAR have been raised, posing higher challenges for the technology.
Andy Sun, CTO of RoboSense, said that although FMCW LiDAR is theoretically the same as optical communication products, the requirements for silicon optoelectronic chips are different. For example, optical communication products require higher transmission rates, while FMCW LiDAR requires higher linearity and signal-to-noise ratio for frequency modulation.
In addition, FMCW LiDAR requires a lot of time for data accumulation and processing during measurement, and parallel channels are needed to obtain enough point cloud data. Therefore, the complexity of silicon optoelectronic chips in FMCW LiDAR far exceeds that of communication products.
According to an executive of a certain ToF LiDAR manufacturer, silicon optoelectronic chips are neither lasers nor detectors, but controllers used for frequency modulation. There is no corresponding component in TOF. In fact, “even in the field of optical communication, silicon optoelectronics technology has not yet been widely used”. Therefore, compared with semiconductor technology, it will take a long time for silicon optoelectronics to truly mature.
Lowering Costs in the Short Term is DifficultAlthough industry experts acknowledge that the cost of FMCW lidar can be lower than TOF lidar provided that its integration is high enough and demand is large enough, in the short term the cost of FMCW is actually higher. This is due not only to the use of 1550nm lasers, but also to the receiver, optical elements, electronic components, etc. (excerpted from Aeye’s official website, translated by Maisen Consulting).
Receiver Cost: Although FMCW lidar can use relatively inexpensive PIN detectors, the overall cost of the receiver is higher than that of TOF due to the requirements of front-end optical devices and back-end electronic devices.
Optical Element Cost: The FMCW system uses coherent detection, and the tolerance of all optical devices must be controlled within λ/20. There are few suppliers who can achieve this precision, so the price is naturally expensive. (According to the analysis of Zoson Research’s Zhou Yanwu, even if these “measurement instrument-grade” components are mass-produced in the future, the yield will be very low, so the cost cannot be reduced.)
Electronic Component Cost: The FMCW system requires an ADC conversion rate 2-4 times that of the TOF system, and requires an FPGA that can receive data and perform ultra-high-speed FFT conversion. Even if ASIC is used, the processing system complexity (and cost) required by the FMCW system is several times that of the TOF system.
In addition, FMCW lidar has strict requirements for system integration and signal processing algorithms. Wu Lei, the Photonic IC design director of Sense Photonics, said: “FMCW lidar’s analog front end and physical layer digital signal processing algorithms are very complex.” Especially the laser modulation is extremely difficult, and there are few companies engaged in related research, which further increases the difficulty of cost reduction.
Not Absolutely “Interference Free”
All FMCW lidar manufacturers regard “anti-interference capability” as one of the major highlights in their PR, but according to an article translated by Maisen Consulting from Aeye’s official website, the anti-interference capability of FMCW lidar may not be as unbeatable as claimed by the manufacturers.
The article states that the FMCW system relies on sidelobe suppression based on window functions to solve self-interference (interference from clutter), and for background information, a 10 microsecond FMCW pulse can propagate radially within a range of 1.5 kilometers, and even a shorter 1 microsecond FMCW pulse may be destroyed by strong clutter 150 meters away.
The Aeye article cited an example that if the FMCW lidar is installed behind the windshield, the reflection caused by the windshield or other first surfaces of the lidar system is a potential “stronger interference source.”
In addition, if OPA is used in the scanning part, sidelobe problems cannot be avoided regardless of whether the transceiver system is TOF or FMCW.
The so-called side lobe refers to the weaker power beams beside the main beam (side lobe in the image above). These side beams will also produce echoes when they hit objects, causing interference, right? Typically, side lobes in FMCW lidar systems may cause “false detections”.
In contrast, typical Gaussian ToF systems have no time-based side lobes except for the several centimeters generated by the pulse duration of 2 nanoseconds itself. When capturing small targets, the dynamic range of small and large offset echoes will not affect the light incident on the photodetector.
1550 nm lasers fail in rainy conditions
The 1550nm laser faces problems such as high cost and failure in rainy conditions, as detailed in the previous article of this series “FMCW Lidar Science (Part 2): Lasers, Detectors, and Scanning“. Interested parties may click on the hyperlink to learn more.
Unable to obtain target reflection information
Several founders and technical leaders of FMCW lidar companies stated in interviews with “9 chapter intelligent driving” that FMCW lidar cannot obtain target reflection information.
Reportedly, the reflection rate scanned by FMCW lidar on target objects fluctuates randomly, known as “speckle effect”. This is because the surfaces of target objects are inevitably rough and uneven, and FMCW measures distance using coherent principles. When it hits an object with a very rough surface, the reflection rate may be high in one direction and low in another. So, what is the real reflection rate? It’s difficult to evaluate.
What’s wrong with the lack of reflection information? Before answering this question, we need to clarify the role of target reflection rate data in algorithm development.
The Director of Algorithms at an autonomous driving company indicated that the value of target reflection rate information mainly manifests in the following two aspects:
-
The reflection rates of various target objects are all within a specific range. With reflection rate information, perception (fusion) algorithms can narrow down the range of target types and even roughly classify them (which is a lane marker and which is a normal road).
-
In the currently popular deep learning algorithms, reflection rate information needs to be input into the algorithm model, which helps improve its accuracy in point cloud detection and segmentation.Several FMCW Lidar company executives have stated that the lack of reflectivity information does not affect detection performance.
According to Yao Jian, CEO of LightWave, FMCW Lidar does not require reflectivity data because its high signal-to-noise ratio and velocity data for each pixel can detect only true targets. Then what is the need for reflectivity?
Andy Sun, CTO of RoboSense, stated that reflectivity data is not highly valued in algorithms based on his discussions with many autonomous driving companies. Although reflectivity information is theoretically used to identify and classify targets in conjunction with vision, it is not necessary in practical applications.
However, Sun points out that customers may not be willing to use Lidar that lacks reflectivity data even though it is not essential in algorithms.
Some executives from a FMCW Lidar company do not agree that FMCW Lidar cannot provide reflectivity information. Mr. Xi, CEO of a Lidar company, claims that FMCW can obtain echo strength information through coherent means, thereby providing more comprehensive reflectivity information including two independent polarizations. Lidar companies have not yet focused on collecting reflectivity information.
Although initial level-up companies are enthusiastic about testing FMCW Lidar, the attitude of OEMs is relatively conservative because the algorithm used in TOF Lidar is irreplaceable in FMCW Lidar. The Lidar algorithm and back-end fusion sensing algorithm need to be rewritten, which can be quite challenging for OEMs with weak algorithm capability.
In addition, according to an interview with “Nine Chapters”, software systems of tech companies rely heavily on TOF Lidar data, and thus the sunk cost is high, and the enthusiasm for FMCW Lidar is not high, even causing opposition.A certain person in charge of a TOF LiDAR manufacturer believes that from a commercial perspective, FMCW is at least five to six years behind TOF, and “this year’s FMCW may be similar to TOF in 2015.”
Prospect: Will FMCW LiDAR “eliminate” TOF?
In the previous two articles of this series, we have mentioned many advantages of FMCW LiDAR. In the first part of this article, we have sorted out some of the challenges it faces. So, the question now is: what is the prospect of FMCW LiDAR? Can it really “eliminate” TOF?
Andy Sun, CTO of RoboSense, said that anti-interference is the biggest advantage of FMCW LiDAR. “At present, there are not many vehicles equipped with LiDAR, so the probability of two vehicles equipped with LiDAR meeting is very small. But as more and more vehicles equipped with LiDAR in the future, the probability of mutual interference will be greater and greater. At this stage, TOF LiDAR will face greater challenges, and the advantages of FMCW will be reflected.”
According to Andy Sun, the higher the penetration rate of LiDAR, the more obvious the advantages of FMCW.
Yao Jian believes that FM will replace AM (TOF LiDAR uses pulse amplitude modulation technology “AM”, so it is also called AM LiDAR).
Yao Jian said: electromagnetic waves or light, its development has a certain law. For example, the radio at the beginning was AM, with a lower technical threshold but more noise, and then upgraded to FM. The advantage of FM is that its signal-to-noise ratio is greatly improved. Millimeter-wave radar and optical communication products have also developed from AM to FM, which is the basic law of electromagnetic and optical development.
Talking about the current high cost of FMCW LiDAR, Yao Jian said that many of the optical communication devices used in FMCW LiDAR are only utilizing 20% to 50% of their capabilities, which results in serious waste.
Yao Jian believes that as FMCW LiDAR manufacturers begin to customize devices that meet their own needs, by cutting off redundant functions, similar to switching from GPU to ASIC, the cost will decrease linearly when the production volume increases. If the device is completely chipified, the cost will decrease exponentially.
In Yao Jian’s opinion, once chipification becomes mature, the cost of FMCW LiDAR can be much lower than TOF LiDAR. Moreover, after chipification, “the standardization of passing vehicles is a logical thing.”
Mr. Xi mentioned earlier that both TOF and FMCW are reducing costs. Among them, FMCW is decreasing slower but will achieve a lower cost in the end. “Why will it be lower? When the entire industry chain upstream and downstream is moving towards chipification, the chipification of FMCW can be more thorough than TOF.”Mobileye CEO Shashua also said earlier that their FMCW lidar “will be much cheaper than existing systems.”
However, even CEOs and CTOs who are most bullish about the prospects of FMCW, believe that TOF cannot be completely eliminated, and they believe that in the long run, FMCW will coexist with TOF.
Mr. Xi said that in addition to autonomous driving, FMCW also has a very important application scenario in heavy machinery. “One of the big problems facing heavy machinery is that the distance between equipment can be several hundred meters, and I need to know the distance and relative velocity between them in real time.”
Yao Jian believes that FMCW lidar will be used as the main forward-facing lidar for vehicles because of its long detection range, while TOF, due to its shorter detection range, may be installed in a corner of the vehicle to fill in blind spots. Wu Lei, the chief designer of Zhigang Photonic IC, holds a similar view.
The head of a TOF lidar manufacturer also believes that FMCW cannot eliminate TOF, and ultimately the two will coexist.
Regarding the potential mutual interference problem of TOF lidar in large-scale applications, the head of the aforementioned TOF lidar manufacturer believes that this can be solved through coding technology. He said that there are two types of coding, one is to remove interference at the lidar level, and the other is to write the coding in the perception algorithm. If interference from other lidars is detected, the algorithm will filter it out.
The aforementioned head of the TOF lidar manufacturer said that the ability to measure speed is the core selling point of FMCW lidar, so in high-speed scenarios, if you need to detect vehicles 1000 meters ahead, FMCW has obvious advantages. This is also why unmanned driving companies such as Aurora, which focus on trunk logistics scenarios, choose FMCW. However, due to lower point frequency, FMCW is not as reliable as TOF in detecting static obstacles climbing ahead on highways.
“Nine Chapters of Intelligent Driving” raised such a question to a head of a TOF lidar manufacturer: since FMCW lidar can achieve more thorough chipification, integrating transmission, reception, and scanning into the same chip, does this mean that in the long run, FMCW will have a better chance to enjoy the benefits of Moore’s Law and crush TOF in terms of cost?
In response, the other party said: this cannot be completely explained by Moore’s Law. Of course, it can be more and more integrated, but there will always be some limitations in certain aspects. For example, the width and interface required for each light wave are limited, otherwise the ability of light loss will increase. Here, how can this be supplemented if it feels incomplete?
In his view, even in the medium and long term, TOF will still dominate and “FMCW lidar will be more of a supplement.”
Market situation of FMCW lidarAt present, there are more than a dozen players in the FMCW LiDAR market, but little public information is available regarding the actual progress of each player. According to Yao Jian, the fastest progress should be made by Blackmore, whose LiDAR is used by Aurora. However, it is understood that Blackmore’s current product is still using a discrete component architecture and has not yet been chipified.
In the past year, Aeva, a LiDAR company that has already gone public, has occupied the minds of many technology personnel and investors with the concept of FMCW. According to plan, Aeva will complete the development and testing verification of its C-sample in 2022 and achieve mass production by the end of 2023.
Aeva has deep cooperation with Volkswagen, Denso, and Continental, and Alexander Hitzinger, Senior Vice President of the Volkswagen Group and Head of Autonomous Driving, is also a consultant for Aeva. This means that once the company’s products are verified, there will be no shortage of mass production orders in the future, with a large potential to reduce costs.
However, many interviewees unanimously believe that “the real leader in the FMCW LiDAR market must be Intel and Mobileye.” As the world’s strongest player in the silicon photonics field, Intel has the most technical reserves for FMCW LiDAR.
At CES 2021, Mobileye CEO Shashua said in a speech: “We expect our LiDAR performance to exceed any known solutions in the market, making the LiDAR itself a single, complete, and independent perception system.”
Shashua proudly declared: “This project would require a huge investment without the corresponding assets and capabilities. Fortunately, we have enough intellectual property, professional knowledge, and wafer factories at Intel to know how to integrate active and passive components and waveguides onto chips. With this, we can integrate all of these into SoCs and hand them over to Intel’s silicon photonics division to manufacture in a factory in New Mexico. This is a unique global asset.”
Of course, companies in the optical communication field upstream such as Lumentum, Finisar, photonics, Cisco, Nokia, Huawei Hisilicon, ZTE, and Optcore may all become important players in this industry chain.
In addition to companies with optical communication backgrounds that initially chose the FMCW route like the aforementioned, there are also players such as Hesai that have achieved phased success in TOF LiDAR and are also developing relevant FMCW technology. Will these companies become the dominant players in the FMCW LiDAR market in the future?A spokesperson for a certain TOF LiDAR manufacturer stated that while there are some overlapping aspects between FMCW and TOF, they are fundamentally different things and there are very few areas that can be reused. “For a company, if you have a team of 500 people working on TOF, you may need another team of 500 people to do FMCW. This does not mean that once you have mastered internal combustion engines, you can immediately copy this technology to gas turbines.”
According to the aforementioned TOF LiDAR manufacturer, there are very few LiDAR companies that can do both TOF and FMCW well. “Doing both at the same time means that your company is already divided into two halves, with half of the people working on TOF and the other half working on FMCW. In this highly competitive industry, it is difficult and unreasonable.”
The spokesperson revealed that their company has a total of 1500 people, but 98% of them are working on TOF. This resource allocation is related to the founder’s judgment on future technology and market trends. As mentioned earlier, in his view, even in the medium to long term, TOF will still dominate, while “FMCW LiDAR is more of a supplement.”
The TOF LiDAR manufacturer’s spokesperson said, “The correct approach is to all-in on one of them and do it to the extreme. Otherwise, in the end, you will have both at an 8, but as long as there is a company that has achieved a 9 or 10 on the market, you will have no competitive advantage.”
This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.