On April 19, 2023, the Shanghai International Automobile Industry Exhibition saw the third appearance of Senscape, the triple-product system of Sensetime, featuring its Drive, Cabin, and Cloud components. The event also marked the industry debut of six new intelligent cockpit products. Additionally, this year’s auto show showcased nearly 30 cooperative models with Senscape.
In 2022, Senscape’s intelligent driving and cabin products were adapted for and delivered to 27 car models, estimating a total of over 8 million cars with its solution. The solution covered more than 80 car models from over 30 car companies. This year’s auto show presented nearly 30 cooperative models that carried Senscape.
In the intelligent driving field, Senscape’s leadership in driving and navigation products was adopted by several car models, including GAC Aion LX Plus and Nezha S. Sensetime was the first Tier-1 technology supplier in China to provide L2+ level production to car makers. During the auto show, Senscape presented its product mounted on GAC Aion LX Plus, which utilised Senscape’s comprehensive 360 degree vehicle and object detection that enabled high-speed NDA auxiliary driving. Nezha S, a cooperative model, was also showcased during the auto show.
In the intelligent cabin sector, Senscape’s display included NIO ES7, which supported real-time driver fatigue reminders in travel scenarios, as well as passenger sensing technologies such as Face ID and Child Presence Detection (CPD). Senscape upgraded its attention-detector technology using a 3D modeling approach in accordance with different driver heights, sitting postures and environment luminosity, increasing its reporting accuracy rate by 53% in extended scenarios. Meanwhile, the reporting accuracy rate was increased by 47% in complex scenarios such as for those with small eyes or under backlighting as Senscape continued to optimize its detectors for specific models. Senscape also enabled real-time perception capabilities based on steering wheel columns, which serves to identify the driver’s movements regardless of the turning position of the steering wheel. These features were also applied to models based on the NIO NT2.0 platform, such as the ET7 and the ET5.
Shangtang Jueying provided software solutions for the first intelligent B-pillar interactive system of Extremely Krypton X that was showcased at this auto show. The Dongfeng Mengshi 917, GAC Trumpchi E9, Yingku, and M8 cars showcased at this auto show will also incorporate multiple functions of Shangtang Jueying’s driver and passenger perception. Moreover, Gaohe HiPhi cooperated with Jueying’s intelligent cockpit’s multiple advanced functions. Chang’an Yida also installed Jueying’s health detection products that can detect various health indicators of passengers in the car cabin in less than half a minute.
During the auto show, Shangtang Jueying also signed a strategic cooperation agreement with Hangsheng Electronics, a leading enterprise in China’s automotive electronics industry. The two parties will leverage their respective strengths and technologies in the fields of intelligent driving and intelligent cabin, jointly develop more intelligent and efficient solutions, and promote the application of technology and the enhancement of commercial value.
Shangtang launched the “RiRiXin SenseNova” large-scale model system that possesses various generative AI abilities, such as language modeling and digital humans. It can provide a multimodal human-car interaction experience for the cabin.
When the experience guests arrive at the Jueying Future Showroom, the Shangtang AI digital human will actively greet and guide the guests to their seats. Inside the showroom, the Shangtang language model “Shangliang SenseChat” interacts and communicates with the guests. The “Miaohua SenseMirage” text-to-image creative platform is also used for the first time in the showroom, automatically generating photos in various styles for guests with the help of the Shangtang AIGC model.
In the past six years, Shangtang Jueying has released more than 30 functions, among which more than 10 are industry-firsts. At this auto show, Shangtang Jueying has launched six innovative products, including Lip Reading Recognition, Sentinel Mode, Intelligent Rescue, Air Touch, AR K Song, and Intelligent Screensaver.
Among them, Shangtang Jueying’s visually-based “Lip Reading Recognition” function is a world first. It uses multi-modal fusion algorithms to enable users to issue more than 40 command words and wake-up words for various in-car interactions through lip-reading, avoiding accidental triggering of voice interaction in scenarios such as human interference, music playback, high-speed wind noise, and tire noise. “Air Touch” allows users to complete various screen operations, such as clicking and drag-and-drop, through various gestures, with mouse-level precision and real-time response.Sentry mode is becoming a standard safety feature in more and more car models. By combining AI with surround-view cameras, the SenseAuto Pilot Entry from SenseTime-Jueying can intelligently identify threats around the vehicle. Sentry mode can also provide different levels of warning based on the set sensitivity, and it can perform well in low-light conditions, such as at night.
SenseTime-Jueying has built a full-stack intelligent driving capability and created a solution for both driving and parking. SenseAuto Pilot Entry from SenseTime-Jueying uses a fish-eye camera that repurposes driving function for parking. The Jueying’s memory parking product supports path memory, path tracing, intelligent avoidance, and target parking in cross-layer parking lots.
Based on the support of the “SenseNova” large model system, the product solution enabled SenseTime-Jueying to achieve mass production of BEV perception in China for the first time and create the automatic driving GOP perception system. The SenseTime surround-view perception algorithm BEVFormer++ won the 2022 Waymo Challenge, with performance improvements of over 60% compared to baseline methods. The automatic driving GOP sensing system developed by SenseTime-Jueying can reduce human costs for target data acquisition by 94%, and has been put into mass production. With a general visual model, Jueying achieves high-precision recognition of more than 3,000 kinds of object targets, reducing the dependence of intelligent driving systems on lidar and high-precision maps.
SenseTime-Jueying independently developed UniAD, which integrates end-to-end autonomous driving solutions from perception to decision-making. In addition, SenseTime-Jueying also uses large models for automated data annotation and product detection, building a complete closed-loop data system.
As of now, SenseTime-Jueying has collaborated with more than 30 car manufacturers around the world, has more than 50 ecological partners, and has accumulated over 31 million units of installed devices.
This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.