Your Intelligent Cabin: is it Practical?

“Your Intelligent Cabin: is it Practical?” is the HMI usability measurement column created by GeekCar and Thoughtworks for intelligent cabin HMI (Human-Machine Interface) assessment.

In each issue, we choose a unique cabin product and use Thoughtworks HMI dynamic usability evaluation system to quantitatively evaluate the most useful functions while driving, thus providing concrete scores.

Here, you can see the most objective and quantitative “usability” evaluation of intelligent cabins.

The rapid development of automotive intelligence has led to the emergence of an increasing number of intelligent cabin functions, such as Xpeng P7-the entertainment cabin: music, gaming, video-watching in-car… When the cockpit becomes more abundant, the in-car experience becomes more extreme. But what about their performance in the “driving” scenario, which is the most basic?

The protagonist of this issue’s intelligent cabin HMI experience metric is the Xpeng P7. Let’s see if the Xpeng P7 with its rich ecosystem is easy to use under the HMI dynamic usability evaluation system.

First of all, let’s talk about the results. By testing the usability of 20 common tasks in the core driving scenario, the comprehensive average score of the Xpeng P7 (Xmart OS) system is 77.63 points-“Operation is smooth”.

Compared with the three previous tested models (Ideal ONE, Roewe RX5, and Mercedes-Benz C200L + Carplay), this is currently the highest score achieved.

Evaluation Criteria

Three types of interaction were chosen: voice interaction (Little P), central control interaction (central control touch screen), and physical button interaction on the steering wheel.

The objects evaluated are the three major indicators that affect driving: in-car single scan time, total in-car scan time, and operation completion time of in-car equipment. Based on the data obtained from each indication and its mathematical fitting, a specific score is obtained to evaluate the overall usability of the intelligent cabin HMI.

In order to ensure the fairness of the scoring, basic operation tasks, complex tasks, and 20 tasks related to driving assistance were carefully tested.

Evaluation Details

Let’s take a look at the specific scores of the three interaction modes (voice, central control, and steering wheel) and the performance of 20 specific tasks.

Voice Interaction: 75.12 – Efficient Operation

The voice system has always been a selling point of the XPeng P7, and there are many tasks that voice interaction can support for this car. However, in order to ensure the fairness of horizontal comparison between different models, we have selected the same four voice-dominant interactive tasks under this system, namely, adjusting temperature, finding favorite singers and selecting songs to play, navigating home, and searching for and navigating to a specific location.

Among these four tasks, for the “adjusting temperature” command, which is a simple and direct instruction, the P7 voice responds quickly and accurately, and currently has the highest score in single-item voice interaction. For the “Hi XPeng, navigate to xxx” type-of one-shot voice interaction, when the wake-up word and command are linked together to convey instructions to XPeng, the cabin system can also respond quickly.

However, when “searching and navigating to a specific location” requires multiple rounds of conversations to complete, the score is lower than expected, with only 65.61 points, at the level of “cautious operation”.

What causes such a big difference in scores?

Task One:

Searching and Navigating to a Specific Location – 65.61 points – Cautious Operation

During the driving process, using voice to call up map navigation is a very frequent use case. Unlike tasks such as adjusting air conditioning and volume, which can be perceived through other senses, every step of adding a location, selecting a route, and planning a path in navigation requires the driver to scan the screen to confirm, occupying visual resources. Therefore, when searching and navigating via voice, the response speed, accuracy, speed, and number of rounds of conversation required by the voice system all directly affect the final performance.

From the scores of various evaluation indexes of XPeng P7’s voice-dominant tasks, the scores of “total scanning time” and “maximum scanning time per session” are both above 80 points, while the “completion time” index lags behind with only 44.36 points.

Problem: Multiple rounds of conversation confirmation lead to increased completion time.“`markdown

In the XPeng P7, there is an interesting setting when using voice-activated map navigation: when searching for and navigating to a specified location for the first time after opening the map, multiple rounds of dialogue are required between the person and the system to select the location and path, which results in a longer navigation time. However, changing the destination or re-planning the route during navigation is not as complicated and can be done with a single command.

Perhaps XPeng defaults to initiating first-time navigation after the vehicle has come to a stop?

Let’s compare the dialogue of different car models under the same scenario:

For the same task, the XPeng P7 and the Roewe RX5 and Mercedes-Benz C200L have similar single-scan times, but what leads to differences in task time and total scan time is the additional rounds of dialogue and voice broadcast time for the P7. The Mercedes-Benz C200L+Carplay, which uses a knob to select navigation paths, has a shorter task time due to the time saved from not having voice broadcasts, and also has the same advantages with its single-maximum scan time. This is because in the task process, discrete hand-eye collaboration can allow the eyes to leave the screen at any time, while continuous hand-eye collaboration through touch will require the eyes to constantly stare at the screen to guide the fingers.

In our previous review of the XPeng P7 in the “Intelligent Cockpit Intelligence Bureau,” we found that XPeng had placed certain restrictions on its voice-activated interaction mechanism. One of the specific manifestations is that voice actions need to take place in a state of dialogue recognized by both the person and the car, which on the one hand avoids frequent waking up due to errors, and on the other hand improves the accuracy of voice interaction.

This mechanism reduces the probability of error in voice interaction, but for driving scenarios, this “cautiousness” may sacrifice efficiency to a certain extent.

Central control interaction: 73.45 — smooth operation

“`The cockpit features fifteen tasks that are mainly completed through the central control interface. With a satisfactory comprehensive central control operation experience, most tasks are performed smoothly, and only a few may require careful operation. No tasks fall into the dangerous or extremely dangerous category.

As compared to other competitors in the same series, the central control interaction performance of the XPeng P7 is the best, with a comprehensive score surpassing that of the Ideal One (71.44 points), Roewe RX5 PLUS (55.09 points), and Mercedes-Benz C200L (70.22 points).

Below are the performance indicators for each evaluation of the central control interaction tasks:

Of course, even with a high score in comprehensive central control performance, there are still six tasks in the “careful operation” level, such as “switching audio sources” and “switching song playback mode,” which may not be very good for driving experience. Let’s take a closer look:

Task 2:

Switching song playback mode – 59.29 points – careful operation

Performance indicators (the score represents the extent to which the task exceeds X% of the top 4 tasks in the industry):

Issue 1: The icon is not easy to operate

For drivers at the driving position, the interaction area of the central control screen can be divided into the best interaction area, touchable area, and worst interaction area. The closer the area is to the driver, the smaller the range of arm movement and deviation from the front when scanning the view.

As shown in the figure below, the icon for “switching song playback mode” is located in the lower right of the central control screen, in the “worst interaction area.” In this area, the driver needs to move their head to a large extent, scan the interface multiple times, and move their arms more to touch the icon. As a result, the driver’s view is away from the road for a long time, coupled with body movement, and poses potential dangerous driving situations.

It is worth noting that there is a protrusion under the screen of XPeng P7.

In previous experiences, we questioned whether this protrusion would create a risk of getting “stuck”. However, when switching the music playback mode, we discovered the function of this protrusion: it provides support for the driver’s arm and reduces the difficulty of operating with a hanging arm. However, even with this small “help,” drivers still need to operate with caution as it is difficult to reach.

Issue 2: Icons are too small to read

The icon for switching playback mode in XPeng P7 is not only located in a hard-to-reach area but is also relatively small for driving scenes. Drivers need to stare at the car dashboard interface for a long time to carefully check and confirm the mode status of the icon, which could lead to visual distraction from the road and dangerous driving.

Issue 3: The touch area is too small

As shown in the testing video, the music playback control component on the default XPeng P7 page is relatively small, and the touch area is too small. This caused drivers to click buttons multiple times to no avail or accidentally touch areas between the “pause” button and “next/previous song” buttons, which led to entering the media player page. These issues of ineffective clicks and accidental touches negatively impact user experience and data conversion.

Task Three:

Selecting songs by favorite artists for playback: 61.67 – Operate with caution

Performance in various indicators (the score indicates that the task exceeds the top 4 automakers in the industry by X% on the corresponding indicator).

Issue 1: Long Path and Multiple Steps

To play a favorite artist’s track, the user needs to complete a series of steps: “main page” – “music default page” – “artist page” – “select artist” – “select favorite song.” The driver needs to click multiple times to complete the task, resulting in a long path and a longer scanning time. Therefore, there is a potential danger when performing this task while driving.

Issue 2: Poor Playlist Layout and Visual Obstruction

The artist’s playlist is displayed on the right side of the car screen. When the driver selects a song, the naturally extended arm will obstruct the line of sight, causing the driver to need to move the arm more to see the playlist clearly. Additionally, the font size of the song names is small, forcing the driver to constantly scan the car screen, resulting in a long time of visual distraction from the road and a potential driving safety hazard.

Steering Interaction: 94.67 – Excellent Experience

Among the commonly used tasks, four tasks are completed mainly through steering interaction. The steering buttons of XPeng P7 are not the traditional, clearly bordered physical buttons, and there is the risk of misoperation during operation. However, in testing, the steering interaction performance was extremely excellent, and all tasks were at or above the “excellent experience” level, with a score of 94.67.

The performance of each evaluation index of the steering interaction task is as follows:

Conclusion

Through this usability evaluation, it can be seen that the HMI experience of XPeng P7 is generally excellent, and none of the three types of interactions has a clear shortcoming, ensuring both driving safety and smooth interaction.

Among them, the steering interaction performed the best, and all tasks were executed quickly and accurately. The overall performance of the central control interaction was good, but if problems such as long task paths and small icons could be optimized, the interaction experience could be further improved.The most impressive aspect to me was the voice interaction. In just a few short months, I participated in two evaluations of the XiaoPeng P7 voice system under different frameworks, and the results gave me slightly different feedback.

Overall, the XiaoPeng P7 voice system is worthy of recognition, with prompt response times, an AI voice system that can provide more accurate responses, and multi-round dialogue with semantic interruption that make conversations more natural. At the same time, voice interaction can provide emotional interaction to a certain extent while not occupying the hands.

However, in terms of the evaluation dimension prioritizing efficiency, such as “driving,” it seems that voice interaction is not efficient enough when complex tasks require multiple rounds of dialogue to clearly express commands, as well as the time required for the voice prompt itself. Perhaps the key to improving voice interaction in the next step is how to balance the meticulous and thoughtful experience and feedback efficiency.

For those interested in intelligent cockpits, check this out~

“Is your intelligent cockpit user-friendly?” is a new column launched by GeekCar and Thoughtworks to measure cockpit experience. Here, we welcome everyone who is interested in intelligent cockpits, regardless of your identity, and we hope to hear your opinions.

Feel free to leave your thoughts, or scan the QR code below to add the author of this article.

There’s more fun cockpit content waiting for you!

This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.