Author: Engineer X

The scenario of parking, as the starting and ending point of a driving path, has always been a common occurrence for drivers. Currently, parking lots are restricted by the limited space and terrain. It is common to have narrow pathways, complex obstacles, and pedestrian interferences. These factors greatly test the parking skills of drivers, consuming their time and energy, and have always received complaints and grievances.

With the development of intelligent driving technology, many automakers have started to equip their vehicles with an Automated Parking Assist (APA) function, hoping to solve the problem of parking difficulties to some extent and meet the needs of users for convenient parking. So, how do these automatic parking products perform? What effects can current automatic parking achieve? In which areas can improvements be made?

With these questions in mind, we have selected several mainstream vehicle models with already mass-produced automatic parking products in the market and conducted a comprehensive evaluation from multiple perspectives and systems, providing some reference for our colleagues.

Part 1: The Basic Principle of Automated Parking

Automated Parking Assist (APA) is a driving assistance feature. The APA system identifies available parking spaces through sensors and, after identifying multiple spaces, allows the user to select the desired parking space. Based on the user’s choice of parking space and the current position of the vehicle, the system calculates the trajectory for parking and controls the lateral and longitudinal movement of the vehicle to achieve the automatic parking effect, also achieving a certain level of precision and posture accuracy. Similarly, the system can control the vehicle for automatic parking out of the space, which is the reverse process.

During the entire parking process, the driver does not need to operate the vehicle. The entire process is fully automated by the APA system, and the driver only needs to observe the surrounding environment and take control of the vehicle in an emergency situation or when the system cannot complete the parking.

The commonly used solution for mass-produced automatic parking products is the combination of cameras and ultrasonic radars. The camera is responsible for detecting the parking line to identify marked parking spaces, while the ultrasonic radar is responsible for detecting the available space for parking to confirm parking space accessibility.

Moreover, ultrasonic radars can also detect parking areas (spatial parking spaces) formed by surrounding objects without parking lines. In addition to identifying the parking space, cameras and ultrasonic radars can also detect obstacles around the vehicle in real-time to prevent collisions.

For cameras, most of them currently use surround-view cameras to form a panoramic view of the surroundings of the vehicle. At this point, automatic parking assist (APA) can be considered as a higher-level function on top of the surround-view image.

Part 2: Evaluation Indicators for Parking Capability

Based on the basic principles in Part 1, it is clear that automatic parking mainly consists of two parts: identifying the parking space and parking into the parking space. Therefore, our evaluation indicators mainly focus on these two parts. In addition, we also consider the evaluation of parking out of the parking space and human-machine interaction of APA.

2.1 Evaluation Indicators for Identifying Parking Space

For identifying the parking space, the most important aspect is to examine the system’s ability to identify different types of parking spaces, as well as its accuracy and reliability in identification.There are many types of parking spaces, and many elements are involved in parking spaces, such as site type, ground material, parking direction, line features, reference features, etc., which are almost impossible to enumerate. We have organized the main elements involved in parking spaces and analyzed and classified them to form a systematic parking space description method, and then formed a universal parking scene library.

Note:

①Unlined parking space, which refers to a parking space formed by the space between objects in front and back or left and right without parking lines, also known as space parking.

②Motor vehicles and non-motor vehicles, if they are stationary for a long time, are classified as static obstacles, such as other vehicles parked in the parking space; if they are in motion, they are classified as dynamic obstacles. In Table 1, in order to reflect the movable characteristics of pedestrians, animals, non-motor vehicles, and motor vehicles, these types of objects are classified as dynamic obstacles.

The various elements in Table 1 can be combined in various ways to form a systematic and complete parking scene library.

In real parking scenes, most of the parking spaces we encounter are standard and standardized common parking spaces. Here, we use illustrations to show several common types of parking spaces, which can cover more than 90% of parking scenes. Our real car evaluation this time will also focus on common parking spaces.

(1) Vertical – Lined Parking Space

(2) Horizontal – Lined Parking Space

(3) Vertical – Space Parking

(4) Horizontal – Space Parking

2.2 Evaluation Indicators for Park-in Spaces

The process of parking in a space tests the ability of a system to plan the parking trajectory and the ability to control the vehicle to accurately follow the calculated trajectory. We will comprehensively evaluate the performance of park-in spaces based on objective aspects such as parking success rate, time consumption, and subjective aspects such as smoothness and comfort.

Table 2 shows the main evaluation indicators for park-in spaces. Some of these indicators can be further divided, which will be introduced in specific vehicle performance evaluations.

(1) The success rate of parking in a space is the most important indicator to evaluate the park-assist system’s ability. Products with high success rates naturally have stronger parking abilities. We will obtain statistically significant data on parking success rates by repeating the parking process many times.

(2) Parking speed refers to the stable speed during the parking process. Parking speed can reflect the comfort and efficiency of the parking process. Appropriate speed can ensure an efficient and smooth parking process.
(3) The time spent on parking is the most direct embodiment of parking efficiency. The shorter the time, the higher the parking efficiency.

(4) Shift times refer to the number of times the gear shifts between D gear and R gear during the parking process. Shift times reflect the system’s ability to plan the parking trajectory. Generally, the number of shift times is positively correlated with the time spent on parking. The fewer shift times, the shorter the time spent on parking, indicating that the system has planned a more reasonable parking trajectory.

(5) The smoothness of the parking process is the smoothness that users can intuitively feel during the parking process. An excellent APA system should achieve coordinated and unified horizontal and vertical movement during the parking process without obvious pauses and jams, no turning the steering wheel in place, and other phenomena.

(6) The comfort of the parking process is the smoothness and comfort level that users can intuitively feel during the parking process. The stability of the vehicle’s lateral movement, the lightness, and severity of acceleration and deceleration during the parking process, all directly impact the user’s comfort.

(7) The vehicle pose after parking refers to the position and attitude of the vehicle relative to the parking space after parking, mainly including the distance from the parking space border and the relative angle, etc.

The process of pulling out of the parking space is the process of the vehicle automatically driving out of the parking space. It is the reverse process of parking into the parking space.

2.3 Evaluation indicators for pulling out of the parking space

In the current mass-produced APA products, not all products have the function of pulling out of the parking space(because the usage scenario is not many, this function is relatively redundant); and the trajectory planning of pulling out of the parking space is relatively simple, and the pulling-out process is short, so we will only evaluate the vehicle pose after the pull-out, without examining the pull-out process carefully.

2.4 Evaluation indicators for human-computer interaction

In any scenario, the driving of a vehicle is a dynamic closed-loop process of “human-vehicle-environment”. Therefore, for APA, in addition to the evaluation dimensions of vehicle-environment interaction in 2.1~2.3, we also need to consider the interaction between humans and vehicles, namely the content of human-computer interaction.

The specific evaluation indicators for human-computer interaction are closely related to APA’s various processes. We synchronize attention to relevant human-computer interaction performances when evaluating and identifying parking spaces, parking into parking spaces, and pulling out of parking spaces.

Human-computer interaction mainly evaluates the system’s display, user prompts, and other contents, as well as the user’s operation methods of the system, from these two levels.

Part 3: Results of actual vehicle evaluation.# APA Product Evaluation of Three Mainstream Car Models

In this evaluation, we selected APA products for three mainstream car models: Tesla Model 3, XPeng P7, and NIO ET7. Their software has been upgraded to the latest version.

Based on the APA product evaluation criteria in Part 2, we conducted specific real-world evaluations of these three products.

3.1 Vehicle Configuration

For the specific vehicle configurations of these three car models and the hardware parameters related to intelligent driving, please refer to Table 3. It can be seen that Tesla does not have a panoramic camera and therefore relies on 12 ultrasonic radars to implement its automatic parking function; XPeng P7 and NIO ET7 both use 4 panoramic cameras and 12 ultrasonic radars to achieve automatic parking through fusion.

3.2 Evaluation Results of Parking Space Recognition

In section 2.1, we have organized systematic parking scenarios and common parking space types. During the real-world evaluation, we first let the APA system try to identify various parking spaces and record the types of parking space elements it can recognize. Then, we evaluated different APA products’ recognition accuracy of common parking spaces and examined their accuracy in identifying parking spaces.

(1) Recognizable parking space elements.

Due to the actual site conditions, we were unable to conduct actual tests on all parking space elements mentioned in section 2.1. Therefore, we evaluated the effectiveness of these three products in recognizing most parking space elements, as shown in Table 4.

In Table 4, a checkmark (√) indicates that the system can recognize the category of that element, an “X” indicates that the system cannot recognize a parking space with that element, and a dash (-) indicates that no parking space with that element was encountered during the test.

## From the test records in Table 4, we can draw the following conclusions:

1. The type of parking lot is not a limiting factor for parking space recognition ability. Under sufficient lighting conditions, the APA systems of all three tested vehicle models can identify parking spaces, whether it is an open-air parking lot, roadside parking space, underground parking garage, or parking building.

2. The ground material of the parking lot is also not a limiting factor for parking space recognition ability. For common materials such as concrete, floor paint, plastic, tiles, etc., all three products can recognize parking spaces.

3. Basic vertical and horizontal parking spaces can be identified and distinguished by all three products.

4. Regular single-line parking spaces can be recognized by XPeng P7 and NIO ET7, but Tesla Model 3 cannot recognize parking space lines and therefore cannot identify corresponding parking spaces.

5. Color of parking line markings does not affect the recognition ability of parking space lines for XPeng and NIO. However, for U-shaped markings or open parking spaces, the products cannot currently provide recognition. We speculate that this is because the front end of the U-shaped parking space does not have any lines, which happens to be an important feature for visual semantic recognition algorithm, an indispensable component.

6. For unmarked parking spaces, APA can detect the available parking space through ultrasonic waves. All three products can recognize spaces formed by car-to-car or car-to-obstacle, and we speculate that they can also recognize spaces that are obstacle-to-obstacle.

7. None of the three products can recognize any characters on the parking spaces, indicating the absence of character detection algorithms for all three products. As a result, the system will not make any special treatment for specific parking spaces.

8. For common static obstacles in parking lots such as pillars, signs, cylindrical bollards, locks, walls, green belts, and dynamic obstacles such as motor vehicles, non-motor vehicles, pedestrians, all three products can recognize them. However, for some static obstacles, the products will show other elements instead, such as using a vehicle model to represent a lock, indicating the unavailability of a car space on the central control screen for XPeng P7.

The above are the evaluation results of the parking scene elements that the three products can recognize. The results show that XPeng P7 and NIO ET7 basically have consistent recognition of scene elements, and Tesla cannot recognize the parking line markings, but it can detect unmarked parking spaces using ultrasonic radar.

Next, we will select common types of parking spaces to evaluate the accuracy of parking space identification by these three products.(2) Accuracy of Parking Space Detection.

The accuracy of parking space detection reflects the precision, reliability, and environmental sensitivity of APA sensors. It is of great significance for mass-produced APA products.

In day-to-day parking scenarios, vertical-lined and horizontal-lined parking spaces are the most common types. Therefore, we tested the accuracy of the automatic parking system’s detection of these two types of parking spaces for the Model 3, P7, and ET7 vehicles. (Space parking is also a common type of parking space, but because space parking is an area formed by surrounding objects, its size, orientation, etc. are not controllable and random, so it is difficult to evaluate the accuracy of space parking detection. Therefore, we will not evaluate space parking detection in this test.)

The evaluation of vertical-lined parking was carried out in an underground parking lot, and we tested fully enclosed, semi-enclosed, and corner-lined spaces. The evaluation of horizontal-lined parking was carried out in roadside parking spaces in a park, and only fully enclosed parking spaces were tested.

The specific testing method and steps were as follows:

(a) After entering the parking area, control the vehicle to travel at a speed lower than 10kph while opening the automatic parking APA on the center console screen.

(b) After APA is turned on, control the vehicle to drive at different speeds with a step length of 1kph, starting from 10kph, until the vehicle exceeds the speed threshold and exits the parking space detection. Record the speed threshold at this point.

(c) Keep the speed at 10kph, re-enter the parking space detection, and control the vehicle to pass several parking spaces continuously (200 vertical-lined and 50 horizontal-lined spaces). Record the number of parking spaces recognized by the system, as well as the number of missed and erroneous identification of parking spaces, and the recognition effect of parking space occupancy.

(d) Organize and analyze the results of step (c) to obtain the statistical results of the accuracy of parking space detection of the three APA products.

The evaluation results of the accuracy of parking space detection are shown in Table 5. It can be seen that:

Regarding the speed requirement: The upper speed limit for both Tesla Model 3 and XPeng P7 to use automatic parking to search for parking spaces is 24kph, which means that when the vehicle speed reaches 25kph, it will exit the parking space detection. The upper car speed limit for using automatic parking to search parking spaces for NIO ET7 is 21kph, which means that when the vehicle speed reaches 22kph, it will exit the parking space detection. According to common engineering development habits, we speculate that Tesla, and XPeng developed APA systems based on a speed threshold of 25kph; NIO developed an APA system based on a speed threshold of 20kph but displayed to users the car speed limit of 24kph and 21kph to avoid state jumps caused by overshoot of the speed threshold.## Evaluation Results of Parking into Parking Spaces

As for the accuracy of identifying vertical-lined parking spaces, Tesla Model 3 fails to detect parking lines and therefore cannot recognize parking spaces marked by lines. XiaoPeng P7 can recognize all 196 parking spaces, with only 4 missed recognitions and no false recognitions (mistaking non-parking spaces for parking spaces). NIO ET7 is able to identify all 200 parking spaces with no missed or false recognitions.

In terms of the recognition of parking spaces marked by horizontal lines, Tesla Model 3 still cannot recognize them. Both XiaoPeng P7 and NIO ET7 can recognize all 50 parking spaces without missed or false recognitions. However, due to the small sample size, these results have limited statistical significance and are for reference only.

Moreover, both XiaoPeng P7 and NIO ET7 can accurately recognize the occupancy of parking spaces where other vehicles, as well as common obstacles like ground locks and cones, are present, with an accuracy rate of 100%.

The above table shows that XiaoPeng P7 and NIO ET7 have high accuracy in identifying parking spaces, even up to 100%. This demonstrates that their performance in recognizing parking spaces with APA is stable and reliable, with good results. However, Tesla can only recognize spatial parking spaces and is unable to evaluate it based on parking lines, thus rendering it unable to be rated according to this criterion.

In section 2.2, we have organized the evaluation criteria for parking abilities. In the real car evaluation, we can further refine these evaluation criteria and conduct a parking ability test for the three APA products based on these criteria.

Considering the standardization of parking space sizes, we will perform the test using standard-lined parking spaces. Since Tesla can only recognize spatial parking spaces and not parking lines, we will select vertical-lined parking spaces with cars parked on both sides and horizontal-lined parking spaces with cars parked in front and behind, respectively. Vertical parking space tests will be conducted in underground parking lots, while horizontal parking space tests will be carried out in ground-level parking lots inside the park.The specific test methods and steps are as follows:

(a) Select a specific parking space in advance as the test parking space, which must be a standard marked parking space with cars on both sides (perpendicular to the parking space) or in front and back (horizontal to the parking space), and with enough parking space;

(b) Turn on the APA. After the system recognizes the parking space, select the parking space and follow the system prompts. Then the system will start controlling the vehicle for automatic parking;

(c) Except for sudden situations such as the intrusion of obstacles, the vehicle is not intervened throughout the process until the system prompts the parking is completed or prompts the user to take over;

(d) Record objective data such as vehicle speed, parking time, and the number of collisions during the parking process;

(e) Record the lateral and longitudinal control performance of the system on the vehicle during the parking process, such as the angle and speed of the steering wheel, and whether the brake is applied sharply, and record the coordination and smoothness of the comprehensive lateral and longitudinal control;

(f) Repeat steps b-e and statistically analyze the success rate of the three APA products for parking and the performance of the system during each parking, repeating 50 tests for both perpendicular and horizontal parking spaces.

Figure 8 is a schematic diagram of the selected test parking space.

Table 6 shows the results of the parking ability evaluation. As can be seen:

There is little difference in the success rate of the three products in parking, and the success rate of Tesla Model 3 is slightly lower than that of XPeng P7 and NIO ET7, but it is still over 90%, which means that at least 45 of the 50 parking tests were successful, and only a few times were unable to be completed, requiring user intervention.

In addition, for both perpendicular and horizontal parking spaces, the success rates of the three products are basically the same. However, considering the vehicle size (length, width, and wheelbase) of NIO ET7, which is larger than that of Tesla Model 3 and XPeng P7, we believe that NIO ET7 achieving a success rate of over 90% is very remarkable.When it comes to the speed of parking, all three products maintain a speed of 2kph, which is perceived as comfortable by passengers.

In 50 tests for vertical parking, the average parking time for all three products ranged between 40-50 seconds, which is similar to the time it takes for manual parking. This indicates that the APA system has reached a level equivalent to manual parking. The average time for parking the XPeng P7 is the shortest, while the Nio ET7 takes the longest, with an 8.8-second difference between the two. Considering the size of the ET7, however, we believe that the difference is acceptable. The Tesla Model 3 is smaller than the XPeng P7, but takes 6.8 seconds longer to park, so its performance is inferior to that of the XPeng P7.

In 50 tests for horizontal parking, the average parking time for all three products was around 40 seconds, which is shorter than the time it takes for manual lateral parking. This indicates that APA performs better than manual parking for horizontal parking. Although the order of the average parking times is the same as in the vertical parking tests, the difference between the products is not significant.

As mentioned in section 2.2, the number of curb rubs is directly related to the parking time. The actual test results also confirm this: the average number of curb rubs for all three products is consistent with their average parking time ranking. The minimum number of curb rubs during the actual test was one, and the maximum was four.

In terms of smoothness during parking, the overall experience with these three products is mediocre. The biggest problem is the need to turn the steering wheel in place, as well as the need to make multiple stops to adjust the direction during the parking process. This gives us the feeling that they are “too mechanical and not smart enough, with insufficient coordination in controlling the vehicle in both horizontal and vertical directions, more like a novice rather than a skilled driver”. In addition, turning the steering wheel in place also damages steering system components.

In terms of user comfort, the experience is also mediocre for all three products. Although longitudinal acceleration and deceleration are smooth, lateral stability is poor – the specific performance is that the steering wheel rotates too quickly, with a large amplitude, which is very harsh, and not smooth enough.

After parking, all three APA products can achieve a good position and posture for the vehicle, with high centering, a suitable distance from the left, right, front, and back vehicles, and the same angle as the orientation of the vehicle and the parking space, without any obvious angles or deviations.“`markdown

According to the evaluation of various indicators in Table 6, we can draw the following conclusions: NIO ET7 has the strongest ability to park while XPeng P7 is the second, and Tesla Model 3 is the weakest. However, the differences between the three are not significant, and they can all achieve the level of manual parking.

3.4 Evaluation Results of Parking Out Function

Currently, not all APA products have the function of parking out. Among the three tested models, only XPeng P7 has this function. Therefore, we will not make a comparison but examine the performance of XPeng P7 in parking out:

(1) The automatic parking out function can only be used after the automatic parking in function is used and then the car is turned off and on again. If the car is parked manually, the auto parking out function cannot be used.

(2) For vertical parking spaces, the system only parks the car halfway out, and the vehicle is handed over to the user to take over automatically.

(3) For horizontal parking spaces, the system parks the car out to the position where it is level with the front vehicle, thus completing the automatic parking out.

It can be seen that the conditions for using the automatic parking out function are harsh, and the parking effect for vertical parking spaces is not ideal, resulting in low usage. For most parking spaces, users can manually park out quickly.

3.5 Evaluation Results of Human-Machine Interaction

We mainly evaluate the human-machine interaction of APA from three aspects: the way of function activation, the displayed content and effects of the interface, and the effect of voice prompts.

(1) The way of APA activation.

Tesla Model 3: There is no separate APA function switch. When the vehicle travels at a speed below 24kph, it automatically detects parking spaces in real-time.

Xpeng P7: There is an APA icon on the central control screen. Click the icon to activate the APA function and start recognizing parking spaces; or activate the APA function by voice command: "Hello X P, automatic parking." 

NIO ET7: Similar to Xpeng P7, the function can be activated through the APA icon on the central control screen or by voice command: "Hi NOMI, I want to park." 

![Xpeng P7](https://42how-com.oss-cn-beijing.aliyuncs.com/article/image_20221201220623.png)

![NIO ET7 APA icon for automatic parking function Figure 10](https://42how-com.oss-cn-beijing.aliyuncs.com/article/image_20221201220631.png)

(2) Display on HMI when identifying parking spaces. 

When identifying parking spaces, all three products will display real-time parking space situation around the vehicle. However, there are differences in the specific display content and display effects. 

Tesla Model 3: Cannot display parking space lines, only shows the area where parking is possible, as well as other vehicles or obstacles around the parking area; the parking area is marked with P. 

Xpeng P7: It displays the detected parking space lines as well as other vehicles or obstacles, and can display up to 6 available parking spaces at the same time, and these parking spaces are numbered in numerical form. However, the parking space number cannot be remembered. When the vehicle moves and the available parking spaces change, the system will renumber, which confuses the original numbering, resulting in a poor user experience. 

NIO ET7: It displays the detected parking space lines as well as other vehicles or obstacles, and can display up to 8 available parking spaces at the same time, and these parking spaces are also numbered numerically. However, NIO ET7 also has the problem of not remembering the parking space number. 

![Tesla Model 3](https://42how-com.oss-cn-beijing.aliyuncs.com/article/image_20221201220653.png)The English Markdown text is as follows, while retaining the HTML tags in Markdown:

(3) Display during parking.

During the parking process, all three products will display the surrounding environment detected by the camera in real time, showing the information of the target parking space and vehicle driving in the rendered model. However, the specific display content and effect are different.

• Tesla Model 3: Displays the parking space the car is about to park in, but does not show the objects around the space in the rendered model.

• Xpeng P7: Displays the parking space the car is about to park in, as well as the vehicles and obstacles around the space. The model of the car shows the current driving direction.

• NIO ET7: Displays the parking space the car is about to park in, but does not show the objects around the space in the rendered model. The rendered model also displays the distance that the car is about to continue driving, providing good user experience and increased controllability for the vehicle’s next action. At the same time, the current gear (D or R) is displayed above the car in the overhead view formed by the panoramic camera, informing the user of the current forward or backward state.

“`

(4) Voice prompt.

Voice prompt can inform users of the current status of the system and the operation they need to perform through sound, which is an important part of improving the user experience.

Tesla has no voice prompt, while XPeng and NIO have voice prompts. During the recognition of parking spaces and the parking process, they inform the user of the current state of the automatic parking system, such as the start of parking, completion of parking, etc.

We subjectively evaluate the effect of voice prompts from three aspects: fluency, accuracy, and timeliness, and the results are shown in Table 7. The voice prompt of XPeng and NIO is fluent and accurate, but the real-time performance of NIO’s voice prompt is poor, which frequently emits voice prompts 1-2 seconds after the vehicle starts moving. That is, the voice prompt of the NIO ET7 is often the status 1-2 seconds ago, while the XPeng P7 does not have this problem.

Part 4: Conclusion

Starting from the basic principles of automatic parking, this article compiles the basic elements and standardized scenario library of parking, and proposes the key evaluation indicators for automatic parking products. Based on these evaluation indicators, we conducted a comprehensive evaluation of three mainstream APA products of new forces from several aspects such as recognizing parking spaces, parking in and out, human-computer interaction, etc.

From our evaluation results, it can be seen that the automatic parking capabilities of NIO ET7 and XPeng P7 are comparable, but considering the factor of vehicle size, NIO ET7 can be said to be slightly better; Tesla’s automatic parking capability is inferior to that of NIO and XPeng.

Such evaluation results are positively correlated with the hardware configuration of the three models: NIO ET7’s super-strong computing power is much larger than the 30TOPS computing power of XPeng P7, so it is not surprising that NIO ET7 is slightly better; while Tesla lacks a surround camera and only depends on 12 ultrasonic radars, its effect is certainly inferior to the surround and ultrasonic fusion solutions of NIO and XPeng. However, we can still see that XPeng’s APA software and algorithm level is very high, and the use of computing power is also reasonable, which enables XPeng to be comparable to the 1016TOPS computing power of NIO in terms of parking with only 30TOPS of computing power.This evaluation focuses on common parking scenarios due to limitations in venue and time, and therefore has certain limitations. The results are for reference only. In the future, with the conditions permitting, we can expand more scenario elements to test the ability of APA in recognizing parking spaces. We can also conduct more parking attempts, such as 1000 times or even more, to test the ability of APA in parking into a space.

All the raw data related to this evaluation are archived. If any colleagues are interested, you are welcome to communicate with us at any time.

This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.