Author: Mu Mu

Recently, I had discussions with some experts in the industry regarding intelligent driving. Apart from industry trends and technology, there is a new focus: how car companies and tech companies can effectively convey “user value” while creating forward-thinking technology.

This is an interesting topic because the electric intelligent vehicle market has been tirelessly pursuing a grand goal since 2015: achieving “fully autonomous driving.” This is a good thing, at least indicating that we are benefiting from the development dividend brought about by technological progress.

The product manager of an industry giant’s intelligent driving products told me:

While promoting technological progress, it seems that very few companies truly focus on “the real-time value of technology.”

Why is it that Tesla and Weltmeister were able to stand out? In my opinion, they are companies that have always paid attention to “real-time value,” which is simply whether users can enjoy the benefits of technology immediately.

This is a big topic, but at the same time, it is also an extremely valuable problem. Intelligent driving has always been something that requires industry-wide coordination and progress. In the beginning of this year, I test drove the IDEAL L7 in Beijing, and also experienced the internal beta version of the navigation-assisted driving of the L8 Pro.

A few thoughts were sparked after test driving the L8 Pro:

• Companies must transform their technological advancements into “user value”
• Progress in functionality applications must depend on advancements in industrial technology
• High performance and large computing power are based on mass production and openness

Let’s talk about these points in detail below.

“Wakefulness” in the chaos of the industry

On February 24th, the IDEAL L8 Pro officially pushed the NOA high-speed navigation-assisted driving function to users in version 4.3.0 of the system. This update mainly improved three points:

• Acceleration performance during lane changes after NOA is enabled, which improves the success rate of lane changes
• Improvements to the cruise acceleration performance when the vehicle’s speed exceeds 80km/h under NOA
• The ability to dynamically adjust vehicle speed based on surrounding road conditions, improving the success rate of autonomous lane changes

As we can see, these three main improvements are the most common scenarios for NOA.

First, let’s talk about the experience

The test drive route was from a test track in Shunyi to the Great Wall at Badaling. One characteristic of this route is its “nine twists and eighteen turns,” which is very similar to the assisted driving course at the Nürburgring. Do you remember Tesla FSD Beta’s nightmare “Flower Street with Nine Curves”? These two roads have similar twists and turns.

1. Lane Keeping and Lane Changing Strategy of AD Pro

Compared with the ideal ONE, AD Pro has made significant progress in lane keeping, with stable centering ability and long-distance centering ability when driving straight. The benefit is that when you judge that the external vehicle does not invade your lane, your driving is always safe, which will bring users great confidence in safety.

The lane changing strategy of AD Pro is comfortable, not conservative. The conservative concept is a compromise made when the perception hardware and computing platform performance are not enough, while the comfortable feeling is that when the vehicle perceives that it can change lanes, the system will execute the command decisively, but the system will not suddenly give you a sharp turn and acceleration in the previous second, but ask you to turn and accelerate during the lane changing process.

2. Cornering Ability and Optimization Strategy

This is a scenario that tests perception and vehicle control. The Horizon Journey 5 computing platform and 8 million pixel camera ensure that the algorithm has a place to perform. There are two points: one is that after entering a high-speed curve, the vehicle can always hold the apex and slow down appropriately, and the Lane Keeping Assist (LKA) can always maintain stability.

And the speed control in the curve has been greatly improved. It will not suddenly reduce the speed of the car at once. The vehicle will always maintain a dynamic acceleration and deceleration status, which is not a disadvantage. It will not make people feel dizzy, but slightly reduce the speed to ensure safety.

Another optimization strategy is that in continuous curves, the algorithm is optimized to make the acceleration between the first and second curves not too large, making it smooth to enter and exit curves between two or three curves that are close together.

3. Dynamically Adjusting the Vehicle Speed to Improve the Success Rate of Autonomous Lane Changing

Previously, in ideal ONE, due to the limitations of computing power and perception hardware, the vehicle could only judge the distance based on the speed of the vehicles in front and behind in the outer lane when changing lanes, and could only change lanes when it was safe. With the Horizon Journey 5 chip and the perception cameras of the entire vehicle, the algorithm can obtain more adjustment space, allowing the vehicle to perceive the surrounding state in real-time, and achieve the success of autonomous lane changing through accelerating to dodge or slowing down to yield.Of course, there are many details to be optimized, but today we want to talk not only about test drives, but also about the importance of providing users with a “user-friendly” navigation assistance system while pursuing advanced driver assistance systems.

But all of this needs to be built on a reasonable hardware architecture, where what is reasonable is the optimal combination of performance, software, and cost. The smartest thing about Horizon is that it provides the right solution at the right time, as described by a technology leader at a certain chip company.

So, as we can see, Orin and Horizon are ideally used. We mentioned the “real-time value of technology” earlier, but how do we understand this concept?

Actually, it’s very simple. The so-called real-time value lies in whether users can enjoy the expected functional experience at the cost they pay at the present time.

Technology is advancing, but the progress of technology is a dynamic upward curve. We may think that autonomous driving technology is progressing very quickly, but in fact, the technology is still in a state of clear goals, but vague engineering progress.

So far, there are still many automakers that have not realized the function of automatic lane change when the turn signal is manually activated, let alone the function of automatic turn signal and high-speed NOA.

The consensus and goal is to move toward higher computing power and BEV, but the key is to add the dimensions of “time” and “user value”.

The first one is easy to understand. Even if you have enough funds to invest, you cannot skip the time required for technological progress. The general approach of car companies and technology companies is to pre-research the next-generation technology and produce the solutions required by reality.

The second point, “user value,” is an extension of the first one. The solution required for reality is not only a commercial behavior, but also a user value.

Although the release of Tesla FSD has been delayed indefinitely and has not yet entered China, Tesla NOA has been pushed for many years and still maintains a high usage rate.“`

The Horizon Journey 5 is a high-performance chip with high computing power, with the following hardware specifications:

• Maximum single-chip computing power is 128 TOPS
• Maximum single-chip power consumption is 30W
• Minimum latency is as low as 60ms
• Real performance is as high as 1531 FPS
• Supports 16 routes of perception such as camera, lidar, mmWave radar, etc.

This means that vehicles can carry sensors with richer and stronger imaging performance, obtain richer perception data, and the chip can process more data. Ultimately, through algorithmic improvements, the system can achieve coverage of more scenarios.

Therefore, we can see that Ideal has adopted a dual-architecture version of Orin and Horizon Journey 5. Companies such as BYD, SAIC, and FAW Hongqi all have fixed-point projects for Horizon.

Horizon provides “real-time value”, and car companies can provide visible and useful functions to users based on Horizon in the current scenario, while also meeting cost requirements.

There are two factors in providing real-time value, one is technology, and the other is cooperation mode.

Let’s continue to talk about it further.

Horizon’s “Real-time Value”

You will find an interesting phenomenon. Almost all artificial intelligence systems, including autonomous driving, basically use data-driven methods to obtain modules, and then integrate them using rule-based methods. This is almost a feature of all autonomous driving and robot systems.

As data-driven modules continue to merge, the proportion of data-driven methods becomes higher and higher. This also makes the demand for general-purpose chips that support rule implementation on future computing platforms continuously decrease.

This is what is often said, “For future autonomous driving algorithms, specialized chips that support data-driven and neural network model inference calculations will account for a significant proportion increase.”

This will pose a difficult problem. At the beginning of chip design, you must meet the requirements of mass production, high performance, support for data-driven and neural network model inference calculations. This requires that the chip team must have a deep understanding of the algorithm, and make forward-looking judgments on the development trend of the algorithm, so that the hardware can support a wide range of algorithm models in the industry.Therefore, clear planning of the software and hardware architecture is necessary at the beginning of chip design, which is what we often hear as the decoupling of software and hardware in autonomous driving computing platforms.

A mature and reliable computing platform consists of basic hardware, underlying software, middleware, and application layers. In fact, what developers need is the decoupling of the upper layer of middleware to form an efficient development environment.

As Dr. Huang Chang puts it, “the combination of software and hardware is the combination of software and hardware in the computing architecture design phase; the decoupling of software and hardware is the decoupling of software and hardware in the development and usage stages. It is essentially the decoupling of ‘development and computing platforms'”.

In the design phase, a computing platform result is obtained through hardware and software implementation. Here it is important to focus on efficiency while considering flexibility.

Hardware architecture includes on-chip storage arrays, tensor computing unit organization, and middleware instruction sets.

In terms of software design, each specific algorithm needs to be analyzed to determine how to break it down and recombine it to maximize parallelism and efficiency. This includes minimizing delay and saving bandwidth during the inference process on the chip.

At the same time, data parallelism and one-stop analysis can better support this process. On-chip storage management and instruction scheduling are the entire process of algorithm execution on the chip during runtime.

Relying solely on PPA chip design indicators can easily lead to the “misunderstanding” that computing power is the only evaluation criterion for chip performance.

Therefore, Horizon introduced the MAPS concept and evaluation method. MAPS is actually the quantification result of the optimal solution obtained through testing a large number of models based on physical computing power, which is proportional to the physical computing power * actual utilization rate and the speed (frames per second) and accuracy of various models.

It focuses more on allowing users to perceive the real computing power of the chip through visual charts. For example, for cars, horsepower does not reflect the overall driving performance as accurately as the 0-60mph acceleration time does. Similarly, computing power does not reflect the actual performance of automotive intelligent chips, but the MAPS (frames per second) of accurate identification per second is a quantitative expression of the chip’s real ability to process autonomous driving tasks.FPS (Frames Per Second), which is the speed at which the unit’s effective computing power processes algorithms. The essence of efficient computing lies in designing the data pathways between hardware and software architecture, which is not only a hardware issue, but also a software issue.

Why does Horizon Robotics excel in this field?

Dr. Huang Chang, CTO of Horizon Robotics, attributed it to their architecture design.

Journey 5 employs Horizon Robotics’ self-developed third-generation dual-core BPU Bayesian architecture, supporting 2.5D/3D algorithm hardware-native acceleration.

Through pre-processing methods for convolution and Transformer algorithms, Horizon Robotics optimizes key algorithms and extends support for these algorithms. Within the entire BPU’s heterogeneous computing unit, there are bias general-purpose computing units and specialized computing units, enabling the product to respond to various levels of complexity.

In addition to making data and computing unit exchanges more flexible and programmable, Horizon Robotics also designed a very aggressive fusion strategy by dividing the entire neural network into multiple levels and fusing multiple network layers into a single inference step, thereby maximizing the reduction in external storage access bandwidth while leveraging each piece of data.

It is important to note that data retrieval from outside requires a long path, resulting in a significant increase in time and power consumption – in simpler terms, computing inefficiency.

In brief, the greatest advantage of this architecture lies in its “abundant heterogeneous computing resources”:

• In visual processing, Horizon Robotics employs two automotive ISPs, achieving 1.3 GP/s of pixel processing and also supporting “online” real-time processing of HDR high dynamic range images.
• For heterogeneous computing, Journey 5 is equipped with 8 A55 DynamlQ CPU clusters for data post-processing, fusion, and behavior planning, mapping capabilities; 2 P6 DSPs for visual 3D scene reconstruction (motion recovery architecture SFM), visual ranging, etc.; and a specialized CV accelerator capable of image distortion correction, stitching, and light flow processing through the use of a CV acceleration engine.# Translation

The Bayesian architecture uses heterogeneous computing units to provide the best calculation mode ratio for scenarios, achieving a reduction in computational power consumption and delay. The architecture improves the utilization of computing units with high flexibility and large concurrent data bridges. The pulsating tensor calculation kernel reduces computational power consumption, delay, and the necessary data bandwidth. Finally, through compilation optimization algorithms, it achieves highly parallel computing capabilities, resulting in the powerful performance output of Voyage 5.

Efficient computing is not a problem that can be solved solely by hardware. Hardware cannot solve complex, high-speed and rapidly changing algorithms. It must leave ample space in the hardware architecture to adapt to the software, while software must be able to utilize hardware advantages to the fullest extent possible.

So what’s the point of saying all this?

The key is: Hardware must be able to meet the changing trend of algorithms.

From rule-based algorithms to data-driven large neural network models, this may be a widely accepted consensus in the field of autonomous driving. BEV and Transformer will be scaled up in the near future.

Tesla proposed the concept of multiple camera fusion perception at last year’s AI Day, triggering a wave of BEV (Bird’s Eye View) perception research in academic and industrial circles.

BEV perception is a feature layer fusion strategy in multi-sensor fusion. Its core idea is to convert the features generated by multiple sensors into a unified coordinate system, and then merge them together for subsequent perception tasks.

Here, the unified coordinate system refers to BEV, which is the world coordinate system in the top-view.

In the BEV coordinate system, the space around the vehicle is represented as a two-dimensional grid, and each grid corresponds to a region on the input image. In Tesla’s FSD pure visual system, multiple cameras are placed around the vehicle, and there are overlapping areas between the camera’s field of view. Thus, each BEV grid may correspond to regions on multiple images.

The core task of BEV perception is how to merge features from multiple images into corresponding BEV grids.

Here, a method called cross-attention is used. The Transformer network commonly uses self-attention mechanisms, completing feature encoding by utilizing the correlation between input data itself.The Cross Attention focuses on the correlation between two different types of data, which refer to the data under the image coordinate system and the data under the Bird’s Eye View (BEV) coordinate system.

BEV and Transformer share a core standard, which is a neural network model based on unified data. Therefore, machine learning methods play an important role in both the perception and cognition modules. A long time ago, there was a famous saying in the field of machine learning: “Data is the King”, meaning that data is king.

The other implication of this phrase is that the autonomous driving computing platform must support current BEV and Transformer algorithm models while possessing efficient computing power. It’s not only about being able to compute, but compute quickly. Therefore, when the ideal AD Pro smart driving system is created based on the Horizon Journey 5 chip, we understand why Horizon is considered a “big player”.

The next generation computing platform will help advance high-level intelligent driving. It’s now 2023, and users and the market have an intense demand for advanced driver assistance. As we mentioned before, providing value to customers involves providing functional products that can be mass-produced, from ADAS to NOA as examples of vehicle automation levels. Horizon Journey 5 exists to give the market better choices.

The BEV large-scale model is about to enter mass production, delivering 128 TOPS of computing power. Horizon has become an international chip company due to its compatibility advantage with self-driving large models like BEV, among competitors such as:

• Tesla FSD: Dual-chip computing power of 144 TOPS, power consumption of 72 W, single-chip computing power of 72 TOPS;

• NVIDIA Orin: Computing power of 200 TOPS, power consumption of 45 W, with multi-chip series connection capability;

• Mobileye EyeQ 5: Single-chip computing power of 24 TOPS, power consumption of 10 W, with an efficiency ratio of 2.4 TOPS/W;

• Qualcomm: The platform’s single-board computing power is 360 TOPS, with an energy efficiency ratio of 5.5 TOPS/W, meaning that the single-board power consumption is around 70 W;

• Huawei MDC 810: The platform’s computing power is approximately 200 TOPS, with a power consumption of about 60 W.

In other words, in terms of single-chip computing power, Horizon Journey 5’s performance can undoubtedly rank among the top four globally. Dr. Yu Kaibo, the CEO of Horizon, once said:>With the start of mass production of 2022’s 100 TOPS-level chips, the most typical of which are NVIDIA’s Orin and Horizon’s Journey 5, the computing power of vehicle-side has improved. The Journey 5 has been mass-produced in Ideal Auto’s L8 and L7, which includes the upcoming new generation of Tesla autopilot chips, with a single chip capacity of around 200T expected to be announced in March. We can already say that the computing power of the vehicles has been sufficient for urban NOA and can meet the calculation needs of current and even future 2-3 year mass production plans. Computing power anxiety has been temporarily alleviated. However, users also find that for various mass-produced models with computing power ranging from 100T to 1000T, there is no significant difference in the user experience of autopilot. We believe that the focus of competition for the next 2-3 years will be on using software capabilities to fully enhance chip computing power efficiency under reasonable chip configuration, in order to create competitive products and a smooth autopilot experience.

From the currently public information, Horizon Journey 5 may become one of the preferred choices for mass-produced high-speed navigation assistance for automotive companies at this stage.

Current situation:

• The Ideal L8/L7, barring unexpected events, has basically locked in the title of the best-selling product at this level;
• Byd, GAC Aion, SAIC Group, and FAW Hongqi will also mass-produce navigation assistant driving products based on Horizon in 2023;
• Light Boat, Pony, He Duo, EASR will also launch advanced driving assistance solutions based on Journey 5.

This is just the first step for Horizon. Dr. Huang Chang said that the next generation of computing platforms for Horizon is already on the road, and the new platform will adopt a brand new architecture to adapt to more extreme algorithms.
This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.