On the morning of July 5th, SenseTime Technology announced on the “Boundless Love – Towards New Strength” AI Forum that SenseTime JueYing has taken the lead in the industry to achieve the in-vehicle deployment of a native multimodal large model. This breakthrough indicates a significant step for SenseTime in the field of in-vehicle AI, demonstrating its leading position in multimodal perception and interaction.
SenseTime JueYing’s in-vehicle 8B model can achieve a first packet delay as low as 300 milliseconds, with an inference speed reaching 40 Tokens per second, covering mainstream computing platforms. This performance metric presents a significant competitive advantage within the existing in-vehicle AI field, offering rapid and efficient responsiveness for vehicle systems.
At the WAIC 2024 booth, SenseTime JueYing showcased its ability to adapt 2.1B and 8B multimodal large models across multiple computing platforms. Particularly, the deployment solution of an 8B model on a 200 TOPS+ platform demonstrates its robust multimodal perception and interaction capabilities. This display not only proves the technical prowess of SenseTime JueYing but also opens up new possibilities for the development of future intelligent vehicle products.
Specifically designed for multimodal large models, SenseTime JueYing created the high-performance computing engine “HyperPPL” that supports mainstream in-vehicle computing hardware and operating systems, and adapts various in-vehicle chip deployment platforms. HyperPPL supports over 400 hardware operators such as flash decode, segment prefill, and optimizes their performance. It supports int8, int4 modes and post-training quantization, thereby achieving ultimate inference efficiency. The inference speed exceeds 40 Tokens/S on the Orin platform, and it shows evident advantages in memory.
In conjunction, SenseTime JueYing will collaborate with partners to develop new intelligent vehicle products, encouraging the advancement and usage of in-vehicle AI technology further. Through cooperation with leading enterprises in the industry, SenseTime hopes to expedite the popularization of intelligent in-vehicle products, providing consumers with a smarter and more convenient driving experience.
In summary, SenseTime’s breakthrough in the in-vehicle deployment of native multimodal large models not only showcases its superior position in AI technology but also offers new directions and possibilities for the evolution of intelligent vehicle products in the future.
This article is a translation by AI of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.