The Explosion of 8MP Cameras on Cars
In January 2021, NIO released its first sedan, the ET7. This highly anticipated model had many innovative features, one of which was the announcement that it would come equipped with an 8MP high-definition camera, with 11 of them onboard. The ET7 had the chance to become the world’s first mass-produced car with an 8MP camera, but because of delays in production, it was ultimately beaten to market by its competitors.
The vehicle that beat it was the 2021 BYD Han, which was released in May 2021 and began delivering at the end of that month. The Han became the first global production car equipped with an 8MP camera. With this 8MP forward-facing perception camera and two domestically-produced Horizon J3 autonomous driving chips, the Han achieved outstanding autonomous driving capabilities and excellent market performance, with sales exceeding 10,000 units for several consecutive months.
The 2021 BYD Han set off a trend of 8MP high-definition cameras being installed on cars, and those days of using 100-200MP cameras are gone forever.
Many car manufacturers and suppliers have followed suit, launching similar sensor solutions. In just one year, it seems that 8MP cameras have become standard in Chinese high-end intelligent vehicles.
Regarding specific vehicle models, those that have already or have officially announced the installation of the 8MP camera include the 2021 BYD Han, GAC Aion V, NIO ET7/ET5, Li Xiang One, Xpeng P7, and the Lotus Eletre, all of which are highly anticipated smart car models on the market, and most are mid- to high-end models. Their technological evolution largely represents the trend of the entire industry.
Some of these models use a single 8MP camera to handle forward perception, such as the 2021 BYD Han; some use seven cameras for forward and peripheral vision, such as the GAC Aion V; and others directly equip their vehicles with 11 8MP cameras, such as the NIO ET7.Apart from the vehicle models, many emerging intelligent automobile suppliers have also presented their intelligent driving solutions based on 8MP cameras.
- DJI Automotive launched a scheme based on two 8MP cameras at the 2021 Shanghai Auto Show;
-
Mobileye’s SuperVision intelligent driving solution also relies on 8MP cameras, as demonstrated by the system installed on JiKe001;
-
Horizon Robotics has also introduced front ADAS and L2+ level automatic driving solutions based on 8MP cameras, including Horizon Matrix Mono 3 and Horizon Matrix Pilot.
Both the car manufacturers and suppliers are revealing that the era of using 8MP onboard cameras has arrived. According to predictions from senior professionals in the industry, the 8MP camera would be in the mass production eruption period by 2022 and 2023.
The main reason for this is that the conditions for adopting 8MP cameras in automobiles have already matured, as indicated by two aspects, “market demand” and “technological reserves.”
Firstly, let us take a look at the market demand.
Currently, global vehicle manufacturers are undergoing intelligent transformation. Intelligent driving, as the core of intelligentizations, has become an ability that many companies, new or old, are constructing. Simple ADAS functions can no longer meet the demand, and more advanced automatic driving functions have become the core sales point that attracts consumers to purchase the models.
For instance, the navigation assistance driving is the most popular function in recent years, and it is expanding its use cases from highways to urban areas. The complexity of the situations is getting higher and higher, with various uncertain road conditions occurring frequently. Hence the requirement for the automatic driving system is becoming more demanding, particularly in the perception system. It is expected to detect more accurately and timely.
At this stage, cameras remain the primary sensor for many intelligent driving systems. Thus, higher-level automatic driving functions demand more powerful camera systems. Therefore the entire market requires high-definition cameras with a more extensive view and coverage.
8MP cameras can satisfy such market demand at the technical level.
One of the characteristics of this type of camera is that it has a further detection distance but also has a larger field of view angle.
Currently, on the market, the front view cameras with 100-200MP can effectively detect the distance of 100-150 meters, with a horizontal view of only about 50 degrees. However, the 8MP camera can reach a detection distance of 200-250 meters and has a significantly larger view angle, up to around 120 degrees.And due to its high pixel count, the overall image clarity of the camera is also higher. The resolution of an 8MP camera can reach up to 4K, which means that more details can be seen when identifying target objects.
With a larger field of view and the ability to see farther and clearer, the camera allows for better perception of objects on the road (whether they are vehicles, non-motorized transportation or pedestrians) in advance, enabling avoidance actions to be taken ahead of time. In addition, the larger field of view reduces the blind spots in the vehicle’s perception and enhances its ability to detect nearby target objects.
The second advantage of the camera is its high resolution, which brings a higher dynamic range (HDR) and better LED flicker mitigation (LFM) capability.
For example, in driving scenes such as tunnels, where there is a rapid change in illumination, high dynamic range can ensure that the camera can identify road elements (such as lane markings) clearly, even in tunnels, ensuring the stability of intelligent driving systems. The superior LED flicker mitigation function also ensures that the camera does not misread road information and thus ensures the safety of the entire system.
Of course, this does not mean that the higher the camera resolution, the better its perception performance. Under the same conditions, the higher the camera resolution, the smaller the size of its individual pixels. As a result, its photoelectric conversion efficiency will be lower in low light conditions, leading to a decrease in the camera’s performance in low light. Therefore, the selection of the camera needs to strike a balance, rather than simply pursuing high pixel count.
In addition, these specific problems can be eliminated by optimizing the perception algorithms and coordinating with other sensors under low illumination conditions.
In general, the performance of an 8MP camera is definitely better than that of a 100-200MP camera.
In addition to technical factors, the selection of an 8MP camera also involves cost-effectiveness. Currently, most front-facing cameras on the market adopt a three- or two-lens scheme (such as Tesla), with each lens responsible for its own tasks. However, a single 8MP camera can fully handle this task and even provides a wider field of vision at longer distances, making it a more cost-effective choice.
Furthermore, the second reason why we believe the conditions are mature for the deployment of 8MP cameras is that automakers and suppliers have made “technical reserves”.
In the field of autonomous driving, the hardware arms race has been going on for a long time. The deployment of 8MP cameras is a reflection of this competition. To make good use of the 8MP camera, both hardware and software need to be optimized.Firstly, the richer visual data generated by the 8MP camera brings greater difficulty in data processing, especially for higher demand in computing resources.
Currently, chip computing power is strong enough and processing efficiency is also stronger, allowing companies such as NVIDIA and Horizon Robotics that specialize in autonomous driving chips to handle 8MP camera data without major issues.
Secondly, at the algorithm level, the visual perception system needs to closely cooperate with the 8MP camera to fully exploit its ultimate performance and enable it to be worth and fully utilized.
Currently, many emerging automakers and suppliers are building self-research teams to strengthen their algorithm capabilities at the visual perception level and have done a lot of matching development specifically for the 8MP camera.
Overall, with the market demand and technological reserves, the trend of using the 8MP camera in vehicles is inevitable, and automakers are looking forward to an 8MP camera-based intelligent driving solution that is cost-effective and technologically appropriate.
Automakers call for a cost-effective 8MP camera intelligent driving solution
From the current announced automakers that intend to adopt the 8MP camera, most of them adopt plans using multiple 8MP cameras, which will inevitably increase costs. Furthermore, these automakers generally have self-researched algorithms (such as Weilipeng) that are not open for use by other automakers.
Therefore, most automakers will inevitably choose to bind to suppliers, such as Geely and Mobileye.
In the supplier plan, Mobileye is a typical “black box” plan with tightly bound hardware and algorithms, and is relatively closed in terms of ecology. Automakers have little autonomy. In launching the SuperVison plan on JiKe 001, Mobileye used 7 8MP cameras, which is a configuration that only high-end models can afford, making this not the optimal solution.
DJI’s onboard plan is more cost-effective than Mobileye, using only 2 8MP cameras, and the results achieved so far are still good, but it still lacks achieving the ultimate in cost-effectiveness.
So, what is the better choice?
Currently, Horizon Robotics’ Matrix Mono 3 based on a single 8MP camera seems to be a better option.
A key point is that as an automatic driving chip supplier, Horizon Robotics understands how to maximize the advantages of “8MP camera + Horizon Robotics chip + targeted algorithm” in supplying this solution.
Moreover, in terms of the openness of the plan, as well as support for local development, Horizon Robotics Mono 3 is also a great choice.
Why do we say this? Let’s take a closer look.
What are the advantages of Horizon Robotics’ Mono 3 plan?
——Translated by the AI, Edited by JayIt is necessary to first understand what the Horizon Mono 3 solution is.
According to official introduction:
Mono 3 is a single-object visual perception solution for L2 and above ADAS market, based on the Horizon EGAI chipset J3.
Mono 3 can efficiently and flexibly perform real-time detection and accurate recognition of multiple targets and implement multiple AI tasks under 30FPS lower latency performance. It can perceive vehicles, pedestrians, lane lines, traffic lights, traffic signs, feasible driving areas, and surrounding scenes.
With the adaptation of an 8MP front-facing camera, Mono 3 can realize a larger range and higher accuracy of visual perception. Certainly, Mono 3 can support not only 8MP cameras, but also matching with them achieves the best performance and cost-effectiveness.
In addition to visual perception capability, the Horizon Mono 3 solution also supports high-precision visual positioning functions, including visual matching positioning and multi-source fusion positioning. For example, this solution can coordinate with high-precision maps to enhance lane line recognition results and achieve lane-level positioning accuracy through re-projection. As long as the automobiles need this kind of service, Horizon can do it.
Basically, the Mono 3 solution has two cores: the Horizon self-developed J3 chip and the matching 8MP camera. The combination of the two, together with the sensing algorithm of the Horizon, has many technical advantages.
These advantages are mainly reflected in three aspects:
- Stronger Perception Capability
The Mono 3 solution uses an 8MP camera, which has a longer detection distance and improved image signal processing ability. Therefore, it can support 250 meters vehicle detection (including 3D), 150 meters long-distance lane line and road edge detection, 100 meters long-distance pedestrian & two-wheeled vehicle detection (including 3D), and 100 meters traffic sign and traffic light detection. It can see farther and clearer, so that the system can react longer.
In terms of field of view, the 8MP camera also brings better performance to Mono 3, reaching 120 degrees. The wider lateral perception range is particularly effective for detecting vehicles in adjacent lanes and reducing blind spots. Even with occlusion, Mono 3 can still detect Cut-in vehicles earlier. In addition, the lane line detection capability of Mono 3 has been greatly improved, which can flexibly cope with the changes in the number and structure of lanes, especially in dealing with large curvature bends, and has better perception performance.The perceptual capabilities of Mono 3 have been enhanced, providing richer perception of both dynamic and static targets. Additionally, with the clear imaging effect provided by the 8MP camera, Mono 3 can support detection of various special road obstacles such as cones, construction signs, triangular warning signs, and diversion signs, as well as detection of lane lines in tunnels with changing lighting conditions. These features can all enhance driving safety.
As a Chinese enterprise, developing perception algorithms domestically means that Mono 3 is better equipped to understand China’s road conditions and traffic behaviors, especially for unique vehicle types (such as tricycles, unconventional vehicles, and autonomous vehicles), lane markings (such as grass-roots markings), and traffic signs.
- Possessing visual perceptual positioning and mapping capabilities
In addition to perception, the Mono 3 system also supports high-precision visual positioning, which can be used to supplement lane recognition results through high-precision maps via reprojection, achieving lane-level positioning accuracy. This makes the entire driving experience more refined and provides a better experience for passengers. For example, in some curves with large radii, the system’s driving skills will be more sophisticated.
What’s even more impressive is that the Mono 3 system also supports on-board visual mapping, which enables on-board real-time modeling of the scene and output of a semantically reconstructed local three-dimensional map. This provides customers with a reference design for cloud map aggregation services, and can also collect and report information on special areas such as accident and construction zones on the road to optimize navigation services.
- The Mono 3 system + several millimeter-wave radars can easily achieve navigation-assisted driving functions
Because of the remarkable performance of Mono 3 in terms of perceptual ability and positioning capability, this system can easily achieve navigation-assisted driving functions when combined with information input from several millimeter-wave radars.
For example, the 2021 Ideal ONE supported by the Horizon J3 Radar is equipped with an 8MP camera for forward vision, and the resulting navigation-assisted driving features are commendable.
The advantages of Mono 3 are also due to the accumulation of Horizon’s visual perception algorithm expertise and their continued strategy of optimizing software and hardware coordination (self-developed chips + self-developed algorithms).
Horizon has conducted targeted perceptual algorithm training for many special target objects and challenging scenarios in recent years:
For example, Horizon has developed a monocular depth estimation technology that uses the disparity changes between local regions of pixels in the front and rear frames to determine whether there are obstacles ahead, and then takes appropriate avoidance or braking action. Detection of white/abnormal/overloaded vehicles and overturned vehicles has always been a difficult problem in the industry, and Horizon’s perception performance is better than that of competitors thanks to the application of monocular depth estimation technology.In addition, it is known that the measurement difficulties of single camera distance measurement are higher. The distance measurement solution based on Real 3D and Freespace developed by Horizon Robotics effectively solves the problem of measuring crossing vehicles (such as extra-long trailers), and reduces safety issues caused by target omission.
At CVPR 2021, the final results of the Waymo Open Dataset Challenge were announced, and the Horizon Robotics team won first place in the real-time 3D detection project, reflecting Horizon Robotics’ significant accumulation in the field of perception algorithms to a certain extent.
Apart from the algorithms, Horizon Robotics is capable of designing multitasking network architecture based on its self-developed chips and providing richer and more accurate perception outputs with their excellent image input and processing capabilities. They know how to use the full potential of their self-developed chips.
Having a good technical solution does not necessarily mean that it will be chosen by automakers as a supplier. It is also important to have the ability to help automakers achieve mass production on specific vehicle products. Sometimes, if the supplier teammate is not selected well, the subsequent performance of the entire vehicle model will be affected, while choosing a reliable supplier may result in a popular product.
Horizon Robotics has accumulated significant experience in mass production, with chip shipments reaching one million, and supporting best-selling vehicle models such as the Ideal ONE and Changan Uni series, and their delivery capability in front-loading mass production is trustworthy.
It is reported that an international Tier 1 company has built an 8MP front vision integrated machine based on the Mono 3 solution, which can help automakers achieve intelligent driving functionality on vehicles at low cost and in a short period of time.
Furthermore, the first production model based on the Mono 3 solution is about to be launched, and the solution has won the order for more than 10 vehicle models from multiple automakers, indicating promising mass-production prospects.
Horizon Robotics’ Mono 3 to Promote Popularization of Intelligent Driving Functionality among Automakers
The L2 and above ADAS market targeted by Horizon Robotics’ Mono 3 solution is a market that meets the growth rate of over 37% in the next five years, with a large number of vehicle products and output scale.
Almost all traditional and emerging automakers are actively promoting the adoption of related intelligent driving functionalities to be favored by consumers. Vehicles without corresponding functionalities will be eliminated and disappear into the historical mainstream.
In the future development of intelligent automobiles, assisted driving will be a “standard” function for high-end vehicle models. The core is whether the cost of a system can cover mid-range vehicle models. Since not all automakers have the ability to build luxury self-developed teams, choosing a reliable supplier is the only way.Currently, Horizon Mono 3 scheme, with “domestic chip + 8MP camera + self-developed algorithm”, has sufficient perception abilities and impressive mass production and delivery capabilities, which perfectly embodies the word “reliable”. A practical and easy-to-use monocular vision-based intelligent driving solution based on an 8MP camera has been formed and is ready to be launched. It will promote the integration of practical intelligent driving functions into more models, not just luxury configurations found in high-end cars.
In addition to its technological advantages, the deeper role of Mono 3 is to promote the popularization of driver assistance.
This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.