Huawei’s New HarmonyOS Smart Cockpit Will Be Available by the End of the Year
Author: Michelin
In the past year, we have seen Huawei’s efforts in various areas of the automotive industry: autonomous driving, in-car chips, Huawei’s selection of cars… Since the new HarmonyOS was pushed on mobile phones and computers in June, when will we see Huawei’s achievements in the smart cockpit field — the implementation of HarmonyOS smart cockpit has become a suspense for everyone.
At the Huawei Developer Conference 2021 (Together) on October 22, Yu Chengdong, Huawei’s Executive Director, CEO of Consumer Business Group, and CEO of Huawei Smart Car Solutions BU, announced: “By the end of this year, the new HarmonyOS smart cockpit will be installed on the next-generation Sailing SUV, which is a collaboration between Huawei and Xiaokang” (most likely the previously rumored Sailing M5).
Along with this news came a previously unexposed on-board system operation interface, which is a tablet-like layout that will be applied to the year-end released Sailing Huawei smart car, and is designed by Huawei’s global design team based on Sailing’s needs.
What different experience can the new system and comprehensive solution bring to smart cockpits and smart cars?
Ecology, the Digital Foundation of Smart Cars
“The greatest companies of this era are ecological companies.”
This was the opening sentence of Yu Chengdong’s keynote speech at HDC2021. From the familiar Google and Apple, to today’s protagonist Huawei, the ecology is indeed one of their winning strategies for success, as it is also true for cars. From the information island of traditional on-board systems to the gradually enriched smart cockpit, the ecology has gradually provided smart cars with a digital foundation, allowing them to have digital functions.
What cannot be separated from ecology is the installation and number of developers. So far, more than 150 million devices, including mobile phones, tablets, smart screens, and watches, have used HarmonyOS, over 1,800 ecological partners, 4,000 intelligent ecological products, and jointly based on HarmonyOS and HarmonyOS Connect to create the “1+8+N” intelligent terminal equipment. These phones, watches, smart screens, Huawei Sound sound systems, and future smart cars achieve intelligent interconnection through the HarmonyOS system.
On the software application ecology, HarmonyOS makes it easy for developers to transition from mobile ecosystems like smartphones to smart cockpits by modularizing and decoupling the operating system. In terms of software development, it is not a matter of choosing between developing mobile or car-end apps on HarmonyOS, but rather being able to deploy them on multiple terminals after a single development effort. Additionally, it can also simultaneously meet the requirements of different automotive companies for in-vehicle systems, reducing the cumbersome work of customizing in-vehicle applications.
The ability of HarmonyOS to develop on multiple terminals has led to a fast deployment of intelligent cabin applications. Currently, 25 high-quality applications have been integrated and it is planned to launch 100+ applications before the end of the year.
One of the results of the joint operation of the intelligent hardware ecology and application ecology is the bidirectional flow of HarmonyOS intelligent cockpit and other smart devices.
In the demo cabin of HarmonyOS intelligent cockpit, we experienced the bidirectional flow function: after connecting the smartphone to the car’s WiFi in the cockpit, selecting “transfer to HarmonyOS car machine” on the phone, the central control screen will display the transfer request, and the central control screen can continue the phone call or continue playing music and radio which has just been listened on the phone. No need to specially connect Bluetooth after getting in the car or search for the radio or music you want to listen to in the in-vehicle application.
For those applications that are not currently matched with HarmonyOS, HarmonyOS provides a similar screen casting function like CarPlay and Carlife, allowing smartphone applications to be cast to the central control screen.
Apart from ecology, what else can Hongmeng cockpit bring?
In addition to its rich ecology, Huawei and HarmonyOS bring a complete set of software and hardware solutions to the intelligent cockpit.## From the HarmonyOS subsystems HarmonyOS-A and HMS-A for car machines, to the car intelligent displays, Kirin car modules, 7.1-channel Huawei Sound audio system, up to the 70-inch AR-HUD, and the application and hardware ecosystem, including the application market. These software and hardware modules make the solution for intelligent cabins more complete, achieving six major extension kits for intelligent cabins which are voice control, visual perception, audio effects, driving safety, AR-HUD, and the seamless flow function mentioned above.
During the experience, in addition to the richness of the ecological applications, two most obvious feelings are: one is the high fluency of system operation; the other is that this set of cabin system tries to call the cabin sensors as much as possible to improve the effectiveness of functions through the integration of hardware and software.
Regarding the fluency, in addition to daily interface sliding, application switching, and searching, we experienced the “basic operation” of HarmonyOS on computers – the split-screen function. Other mainstream car machine systems on the market also have similar functions, but with this system, when opening both maps and high-demand applications such as Kuaishou, searching for navigation, switching between the left and right sides of the screen, we felt that the operation was smoother.
The integration of HarmonyOS cabin system’s software and hardware is reflected in specific functions.
In terms of voice control, HarmonyOS uses the AI-based intelligent voice assistant Xiaoyi as an interaction entry to provide common functions on car machines, such as “visible and say”, multi-tone detection, and multi-user concurrent, while also using the fusion of voice and DMS cameras in the cabin to provide multimodal interaction functions, such as lip reading and voiceprint recognition.
We tried it out in the demo cabin, and found that in the case of only moving our mouths without making any sound, the system could recognize simple commands corresponding to the mouth movements to adjust the temperature, volume, and even control the home connected to the car to open the curtains. Voiceprint recognition is used for payments on car terminals. Of course, what the demand scenarios are for lip reading and voiceprint recognition and whether voiceprint recognition can be used to realize the commercialization of car terminals, still depends on the specific applications it will be used together with in the future.
Of course, the DMS camera in front of the cabin is not only used for multi-modal interaction of voice but also for preventing distraction and fatigue monitoring and warning. When the DMS camera detects that the main driver has been looking away from the front for a long time while driving, or has closed their eyes, or even started to doze off with their mouth slightly open, the system will pop up a warning for driver monitoring.
In addition to the camera, Huawei has also equipped this HarmonyOS smart cockpit with millimeter-wave radar to improve gesture recognition accuracy. With the cooperation of millimeter-wave radar and camera, the cockpit can be connected to the phone, allowing users to control the cockpit camera with gestures and take selfies, which can be automatically sent to the phone’s social network accounts.
These solutions also need to be modularized and combined according to different needs. For example, the millimeter-wave radar in the cockpit will not appear on the next-generation Seres Huawei Select SUV as mentioned earlier. However, it can be experienced in the delivery of the BAIC Blue Valley Arcfox α S HI version, and also the industry’s largest 70-inch AR-HUD function.
Conclusion
This year, HarmonyOS Smart Cockpit made its debut at the Huawei Developer Conference for the first time, and we also saw Huawei’s all-out attack in the automotive field.
“As the future unfolds,” just like the main theme repeatedly mentioned at the Huawei Developer Conference 2021 (Together), the HarmonyOS ecosystem, smart cockpit solutions, and even the preview of HarmonyOS 3.0 for developers show us the possibilities that smart technology can bring to intelligent vehicles and the Internet of Everything, making those imaginations of intelligent vehicles more easily achievable.
As for when these technologies and solutions appear on products in a mature manner, it depends on market demand.
This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.