The Past and Present of Unreal Engine
If you are a gamer, you should be familiar with the name of Unreal Engine. As a game engine, countless masterpieces based on Unreal have been developed, such as Fortnite and the popular game Monkey King: Hero is Back.
Its extremely detailed and realistic graphics are well-known among gamers, and its sophisticated development process and powerful real-time rendering technology make it a favored engine among many game companies.
The distinguishing feature of Unreal Engine lies in its comprehensive application system. In the past decade or so, Unreal Engine has been extending its reach beyond the gaming industry and now covers areas such as architecture, film, healthcare and automotive, which has a wide range of applications even within the same industry. For example, it can be used as an early-stage design tool or as an external display tool in building design. It can be said that 3D visualization has penetrated into various industries.
Today’s focus is on the performance of Unreal Engine in the field of intelligent automotive cockpit.
We had the opportunity to conduct a brief interview with Mr. Zhang Ting, the CTO of Mozi Technology, the partner of Lotus Hyper OS, the intelligent cockpit system of Lotus, at UNREAL OPEN DAY 2022. The following is the interpretation of the content based on the interview.
Digital Twinning
Digital twinning is creating a world and vehicle on the car machine that has a state identical to that of the real world, simulating and displaying visually stunning effects.
Lotus’ in-car system is named Lotus Hyper OS. It is based on two 8155 SOCs, Android architecture and features a high-quality 3D interactive interface built with Unreal 4. The whole system is highly customizable and intelligent.
To create the realism of the virtual world, UNREAL ENGINE game engine and high-polygon vehicle models were used, with PBR materials and HDRI environment lighting technologies, creating an unusual car interface that looks and feels like the real thing.
From colors, configurations and appearances, to door opening and wing lift-up status, the in-car system can maintain synchronization with the real-time state of the vehicle.
In addition, in Lotus Hyper OS, users can view real-time weather information read from cloud services, combined with car sensor data to display weather conditions, daytime and night-time drive effects on the screen. These visual effects that are both realistic and gorgeous, rely on the rich special effects tools and interfaces offered by the UNREAL ENGINE.
Will 3D HMI be the trend?
Moovita’s CEO Zhang Ting: “For vehicles, people interact with them from the outside in. This approach can decrease the learning curve for users.”
Generally, someone familiar with using a smart system is used to working with hierarchical menus. These menus are based on the understanding of a specific thing, and people can take the correct action with their eyes and hands.
For example, in a traditional menu, in order to adjust the tail wing, you need to first click on the car control button, then select the corresponding external settings column in the first-level menu, and finally change the tail wing’s on/off status in the second-level menu.
In the 3D vehicle control interface, first switch to an angle where the rear wing control can be clicked, and then click the interaction control to start setting.
So, for the interaction logic of the 3D smart cockpit, the control logic has not fundamentally changed. Because the hierarchy relationship in the 3D user interface is integrated, the only difference between the two interaction controls is the difference between visualization and textual description.
But for users, a user who is unfamiliar with the intelligent interaction method can directly operate the 3D vehicle control interface, because in this scenario, the interaction completely matches the real-world cognition, while the hierarchy menu needs to be transformed.
Although the understanding of these two has no difference in essence. And, quick control functions, such as maps, music, and air conditioning, can be integrated into the entire interface by placing them on the edge of the Lotus Hyper OS main interface, as well as integrated for quick control of high-frequency functions.
However, corresponding to this, the 3D interaction effect still needs to be used together with the hierarchical menu. Completely abandoning the hierarchical menu will make the entire vehicle control settings lack logical consistency, and the interaction design of all 3D games in the past has also demonstrated this truth.
For example, if you need to adjust the detailed settings related to vehicle driving, functions like kinetic energy recovery intensity and steering wheel force are abstract and cannot be interacted with as actual content in the 3D interface.
The demand for visual effects from the public is gradually increasing, and all products in this era are becoming more and more design-oriented. If a better visual effect can be achieved on the basis of practicality, this may be the next-generation cockpit system mentioned in the Lotus system promotion, a system that is oriented towards the future.
Highly Customizable
As a technology supplier, LuSuKu Studio of Shanghai Mosi Technology needs to customize the development process for various needs when facing different carmaker clients.
To this end, Mosi Company has created a hardware performance evaluation tool, MoPerf, internally. This tool runs tests on various lighting, materials, and model scales on existing hardware platforms to obtain support information for various effects in the Unreal Engine, and then customize the development of the entire system based on this information and needs.
With the help of MoPerf, subsequent development work has been made more efficient and convenient. The team can flexibly use this tool to understand the performance of the chip and choose among technologies and effects based on performance to obtain the optimal solution under the same hardware.
For example, on Eletre, two 8155 chips are used to handle the computing tasks for 5 cockpit screens. Through MoPerf’s testing in advance, it was found that this hardware platform can support real-time lighting and some special effects, which greatly reduces the debugging time in the entire development process.
In Lotus Hyper OS, the corresponding function of real-time lighting is day-to-night conversion and weather effects, which is also one of Lotus’s initial requirements for the cockpit system.
New Applications of Game Engines
In the past, Mosi Technology has more than ten years of experience in game development, and has experience in developing many 3A games both domestically and internationally. For a game company, past development experience in the game industry will be very useful for applications in intelligent cockpits.
On the one hand, the development tools of the Unreal Engine are very unified and have high versatility, especially the 3D visualization part. Benefiting from the mature and advanced lighting and shading technology of the Unreal Engine, the PBR (Physically Based Rendering) material process, which is widely used in the game and animation industry, can also be directly applied to it, which will greatly help the efficiency of subsequent development.On the other hand, due to the design and programming of interactive logic involved, and the fact that the game itself is a highly programmatic system, the complexity of a conventional game architecture may be many times more complicated than that required for Lotus Hyper OS’s scene interaction.
Apart from these, Unreal Engine’s physics and effects systems are hardly used in this simplified Lotus scene. Streamlining everything allows the entire system to provide optimal visuals and run smoothly on the dual 8155 hardware platform.
Interview Summary of Zhang Ting, CTO of Mozi Technology
Q: LuSuKu’s genes come from the gaming industry. How can we apply our previous experience to the intelligent cockpit? What are the common points?
A: Actually, there is a very fundamental thing. Our brand name, LuSuKu, is quite interesting. From our perspective, it doesn’t have any special meaning. It is our own creation based on the name of a racing team in our original project. We crafted this team ourselves, and it was really interesting in the story. So we took the name of this team and used it as our business brand name. So I think our genes will have some differences from others. Besides gaming, first of all, we like to do something we love – cars. For our team, many of us are especially fond of it. When we work on other content like gaming and cultural entertainment content, we often involve cars. We also have a lot of experience in car games. So for us, entering this industry was somewhat accidental, but it also has a certain inevitability. Because we had previously been exposed to things like digital twinning quite a bit.
To a certain extent, cars are because we like them ourselves. From the perspective of developers and users, we all have a certain understanding of them, so this is an opportunity for us to enter this industry. Also, from a technical standpoint, we actually define ourselves more as a product company or service provider with R&D capability. Fundamentally speaking, using something and developing something are two different states.For our company, it is more about exploring the advantages of game engines such as UNREAL and integrating our research and development capabilities to turn it into a product. This is something we have done frequently in the past when developing games, and has resulted in a wealth of technology that can be applied to topics such as 3D modeling for automotive interfaces. However, since the processing power of the 8155 chip used in car interfaces is limited compared to modern smartphones or computers, there is a need to find a way to effectively utilize our technology on this platform and integrate it with the car’s extended features.
Fortunately, our experience in the past has shown that a company with graphic and imaging capabilities in research and development can effectively address this challenge, and provide clients with products that have unique features. In terms of hardware requirements, the computational environment of the project is predetermined when brands design and plan their vehicles, so we must work with the existing environment.
As someone who has been in the game development industry for over a decade, I am quite familiar with this issue. Techniques for developing better graphics and effects require more processing power. Similarly, from the time game consoles first emerged to their eventual obsolescence five to seven years later, the visuals and features became increasingly sophisticated as a result of the continuing exploration of computational limits. This experience helps us to find better ways to integrate our technology into cross-industry products without breaking their original definitions and concepts.
Our experience has taught us the importance of collaboration in developing good products that incorporate our technological capabilities into existing designs, without disrupting their original intentions; this is a valuable lesson that we will continue to apply.Q: Because many of UE5’s technologies, including Nanite and Lumen, are revolutionary for the industry and can greatly improve development efficiency in traditional game workflows, will UE5 provide some assistance in the field of intelligent cockpit car machines in the future?
A: Technological upgrades will gradually enter the HMI field, but blindly implanting technology into a particular field will make it a burden. The choice of what technology to gradually implant in this field will come from our discussions with manufacturers about the product, as well as the direction of research and demand for the car itself. Nanite and Lumen, as you mentioned earlier, are good topics that I believe will have great potential in the future. In addition, the growth trend of computing power will definitely free up more space.
Q: In the early stages, was the choice of the Snapdragon platform an active or forced decision?
A: For us, we don’t really pay much attention to the current platform situation. For over a decade, we have often faced a topic where everyone wants better results, more functions, and more user-friendly products, and everyone hopes that the original power consumption of the product can be controlled. For us, the first thing we face in the industry of intelligent cockpits is that the current chip and architecture choices are probably at a certain level, but we also believe that in the future, everyone will gradually develop towards high computing power. Therefore, for us, it is more important to implant more technologies that are achievable and helpful to users into the current platform, and the environment will be more important than saying we will make a hardware platform choice.
Q: This time, our cooperation is actually based on the Android platform to build the entire system. Will there be more systems developed in Unreal Engine in the future, such as all kinds of transaction systems and task systems in games, with all these third-party apps built into Unreal Engine?
A: We can’t say for sure whether it will be third-party or not, because first and foremost, this is a car selection issue. But from a purely technical standpoint, what you said is completely possible. For example, the development of in-car games may become an independent field. This topic must exist, and since this is the case, it may have more room for development, including the deep integration of other third-party applications. I think it is entirely possible. If we simply talk about the topic of entertainment, its space and possibilities will be very large.
Therefore, I think that because of your relevant professional background, you may be more interested and sensitive in this area technically. I think everyone can see this, brand owners can also see it, and they may imagine whether there will be some extensions or even involve some ecological topics in the future. For us, there is room for this.
Q: If we leave aside the factor of our computing power, how big do you think our imagination in the intelligent cockpit can be?
A: This is a very interesting topic. I think we can talk about it for a long time. In fact, its imagination space is definitely very large, because why we say that we have investment and watching in innovative products and entertainment in the cockpit. It is hoped that what you just said will become a new future, and there will be suitable application content, space, and functions for different application scenarios in the cockpit, single-person, friends, or family. So I think this topic is a good opportunity to start discussing from now on. We can slowly look at the development direction of the overall vehicle brand and the technical architecture. For us, including entertainment, in-car games, artificial intelligence, and the extension of life application scenarios, these topics will be involved. I think we should have a completely open-minded attitude towards these things, and they may gradually come.
Q: NVIDIA previously released a chip called Thor, which was mainly aimed at autonomous driving. However, they mentioned during the press conference that it can also be applied to sensing and decision-making in autonomous driving and computing in the intelligent cockpit. What do you think of such a chip?
A: I think NVIDIA has always been at the forefront in this area, but what I am looking forward to more is whether we can have some self-owned chip brands, and whether our own chips can achieve certain applications in certain fields. This is the most anticipated area for me. So, NVIDIA has many things that can achieve good status, which everyone can see. We are now beginning to pay more attention to some new companies in the country in the automotive field. This is a very promising and interesting topic, and this actually what we have been doing for a long time. It is not just us, but the whole industry has been doing it for a long time. Actually, this point is also a long-term cooperation for chip companies, and everyone is making progress together.For a single car, there are at least tens of thousands of different components. I think there will be various topics of cooperation and shared development in this regard. Therefore, the direction of the industry continues to develop in a positive way, and our involvement in this matter is definitely a positive choice for ourselves.
In conclusion
Overall, the application of Unreal Engine in intelligent car cabins is a relatively new field. It is obvious to any user that the graphic interface is becoming more and more advanced in terms of visual effects. It is simply because of the inter-generational differences highlighted by the 3D visual interactive interface.
Actually, in the first few years of the development of intelligent cabins, which is now, the direction and space for future exploration is still very narrow. For example, in Zhang Ting’s opinion, the hardware and software requirements are continuously explored. Software will never be satisfied with the existing hardware, and vice versa, as clearly seen in the development history of games in the past decade.
In the automotive industry, the intelligent solutions of brands and cars at the same level are not similar, and there are differences in brand positioning, hardware solutions, and system solutions. In such an intelligent environment where the entire industry does not have a clear direction, companies that are exploring new routes, are basically like pioneers.
Based on these factors, we will ponder on whether overall intelligence will be the trend of intelligent cars in the next five years? And in this trend, will 3D interaction design be a better solution? As the main battlefield for the global automotive industry’s electrification and intelligence, will China seize the opportunity to make progress in the chip industry?
At least a new round of software arms race has begun quietly.
This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.