Taking a Different Perspective on High-Precision Maps
At the “Inaugural Yan Knows Intelligent Vehicle Annual Conference”, Chen Xuejuan, the leader of the high-precision map development team for FAW (Nanjing) Technology Co., Ltd. and FAW’s Artificial Intelligence Research Institute, gave a speech on “The Development Path of Mainstream L4 High-Precision Maps” and shared FAW’s approach to high-precision maps as a mainframe factory, the need for a self-developed map team, and the direction of the team’s business, which serves as a model for mainframe factories.
She introduced two aspects in her speech: first, mapping techniques suitable for small-scale forward-looking research; second, innovative business map inspection, which better supports the development of the entire team’s autonomous driving algorithm.
Chen Xuejuan explained that FAW Nanjing Co., Ltd. is a very important branch of the FAW Group and an important part of the company’s “Three Countries and Five Regions” global strategy. In order to seize the opportunity of autonomous driving and attract talents to invest in domestic research and development, the Group separated the Changchun Artificial Intelligence Institute last April and established a subsidiary in Nanjing.
After market analysis and matching with group resources, the company established development goals for three phases: early, middle, and late. In the early stage, it focuses mainly on two aspects: AI perception and AI big data.
In terms of AI perception, the company has a clear goal of building a pure visual automatic driving mass production route, forming technical accumulation of other auxiliary technologies during the R&D iteration process, and landing core software production, planning to conduct research on vision, radar, positioning, fusion, etc., and outputting 13 core software products in the entire R&D process, enabling FAW’s intelligent driving business. At the same time, the R&D in Nanjing is expected to be helpful not only for the Hongqi brand but also for the mass production of Besturn.
In conventional situations, the high-precision map department of the mainframe factory is only a demand proposer, proposing their own requirements to various map providers and not capable of building maps or even modifying them.
Because there is no industry unified labeling standard for high-precision maps at present, each company has different utilization rates for high-precision map element information, resulting in different needs and different business priorities. In addition, map providers have rich high-precision map raw data but tend to open specified element information based on demand.
FAW Nanjing has a very clear positioning for its high-precision map business, which is different from that of other map providers and mainframe factories.In the company’s strategic planning, the high-precision map module aims to assist the entire company’s autonomous driving, not just the purchase of high-precision maps understood by most OEMs, or the purchase of high-precision map software. Instead, it aims to customize high-precision map software, enrich road attributes in a short period of time according to the different R&D stages, and quickly modify existing maps. The three major business segments of the entire map department include map production, map engine, and map quality inspection.
Map Production: They have their own small-scale map production capabilities. However, this type of map production is different from the development methods of large vendors such as Navinfo or AutoNavi, who need automated mapping, scalable automated mapping capabilities, smaller map accuracy errors, and more importantly, legitimate and compliant map production and delivery cycles.
Regulatory restrictions determine that the map delivery cycle to the vendors is relatively long. During the rapid iteration and continuous exploration of the first half of R&D, this purchasing method cannot respond to their needs in a timely manner. Therefore, they need to have a very small internal mapping team to support the requirements proposed by perception, positioning, and control planning and respond quickly.
For example, Chen Xuejuan said that according to the difficulty of the requirements, they can currently complete a testing map with dedicated functions in as little as 2 hours. Basically, they can understand the intentions of both parties, develop functions and maps after a meeting in the morning, and complete the map production for online testing in the afternoon. Even for slightly more complex functions, it can also be completed in about two weeks. This iteration speed cannot be achieved by purchasing maps.
Map Engine: Provide an interface to meet the requirements of other modules for reading map elements. This is also a question of how to use the map. The map team not only needs to produce maps and ensure map quality but also needs to open up an interface for other modules to read map element information, guiding other modules to use the map smoothly.
Map Quality Inspection: This is a very important link and an innovative field proposed by First Automotive Works (FAW) Nanjing. The entire cycle from R&D to mass production may be involved.
In the R&D stage, because they build the map themselves, they need to check if there are any defects in the map they made and do quality assurance by themselves. In the mass production stage, how to judge whether the maps delivered by vendors meet the requirements of other modules, whether the map quality is qualified, cannot rely solely on the words of the vendors. They need to check the maps provided by the vendors themselves to have confidence. This link is an idea proposed to the company’s management last year and is also a feature that distinguishes them from other OEMs.From January this year, we started researching the system and are still in the initial stage. However, there is a clear development direction with significant achievements so far. Not only have we produced practical outputs but also found some mapping problems and avoided them in the next mapping processes. The related patents are also being applied for, and we are very hopeful that this forward-looking research will be applied to mass production.
What is the difference between the mapping process and the mapping company?
According to Chen Xuejuan, the principles and procedures of making our maps are different from those of mapping companies. Mapping companies prioritize the efficiency and speed of mapping, pursuing the ultimate automation rate. However, as a main automobile manufacturer, mapping is not our main business. As a supporting department, the map department only needs to build some simple scenarios, for example, ranges of one or two kilometers, for testing purposes.
For instance, if the control planning department requests to create a roundabout area, we will only build that particular area.
Normally, special scenarios require special processing. At present, we do not have the capability to consider common city scenes such as overpasses, ramps, and tunnels, and such scenarios require correspondingly attentive and patient research and development regarding perceptual modules, localization modules, and control planning modules. Conversely, when we are building our own maps, we do not need to consider the efficiency or scale that mapping companies often do because the supporting departments only need to fulfill the requirements for mapping demands monthly or quarterly.
Each time we map, we plan the range, difficulty level, and types of road elements in advance and leave sufficient time. Since the frequency of mapping is low and the range is small, manual annotation is feasible.
Currently, using fully manual annotation can also meet the needs of business departments. Considering the high repetitiveness and labor-intensive lane line annotation task, we are contemplating automaticizing this part of the work, to relieve the burden of the annotation team. Whether it is necessary to implement almost fully automated annotation in the later stage needs further consideration. The focus is not only on the level of technological difficulty but also on whether it is necessary to continue dedicating human resources to identify elements with low frequency on maps. For example, speed bumps appear less frequently and can be manually annotated with a low workload and high accuracy, so algorithms and codes do not necessarily have to be developed for automatic recognition and annotation.
Currently, FAW-NJ uses a two-dimensional annotation system, not a three-dimensional point cloud system. This very basic annotation system can also meet basic needs because most of the elements attach to the ground, with traffic lights being processed separately. As this approach has some limitations, we plan to develop a three-dimensional point cloud-based annotation system in the future.
Even if we replace the two-dimensional annotation system with a three-dimensional one, encountering situations in which the ground reflectivity is similar to that of the laser radar is inevitable. At this juncture, introducing images is more reliable, relying on data fusion consisting of images and point clouds to support annotation. This is also a direction of future technical expansion.## Building Mapping Technology Route
The process of building mapping is the same as the mainstream mapping scheme in the industry, which consists of five main steps: data collection, data splicing and processing, labeling, generating the final map file, and map inspection. Currently, autonomous vehicles are directly used for mapping, with two types of vehicles: one is a car provided by the headquarters of Changchun FAW Group, equipped with a mechanical LiDAR, and the other is a car equipped with a solid-state LiDAR developed by Nanjing, designed for mass production.
The reason for having different mapping vehicles is that the regions where the vehicles are located have different initial design plans. FAW Group currently only maps where it has its own R&D institutions, which are currently only in Changchun, Nanjing, and Hainan in China. These three places all have different types of autonomous vehicles for testing. The map team serves other modules and tries to utilize whatever resources are available on-site to avoid vehicle transportation and reduce costs.
Data processing after obtaining data packets adopts a top-to-bottom projection method, using LiDAR information to generate high-precision maps. The data packet can support three formats because different versions of autonomous vehicles in the group use different systems, so the formats have to be passively adapted.
The process is simple, just read the point cloud information and concatenation processing can be done to meet most of the scene requirements. Of course, there may be defects, such as heavy shadows at turning corners, which may be improved with the use of NDT algorithms. For example, when the reflectivity of road markings is similar to that of the ground, it is impossible to distinguish between the two, and rough mapping methods are used. Thus, some closed-loop scenes may not have good effects. These issues currently exist and need to be solved one by one.
Currently, the annotation is purely manually done, and all element types and lane relationships need to be manually labeled. Lines and points on the base map need to be drawn manually, and left turn road relationships and traffic lights also need to be manually labeled.
Because the scope of mapping is very small, such as 5 kilometers, it is impossible to have 20 or more traffic lights. If there are a few, each one can be processed manually, which will not occupy too much of the developer’s time, and completing the annotation can meet the needs of other departments.
The annotation system was initially used by the supplier’s system, but later found that it could not meet the growing R&D needs, and could only develop its own system to expand and enrich road information at any time.
“`
To generate the final map file, two formats are needed for the L4 project of FAW-NJ based on the Apollo system: Apollo OpenDrive and standard OpenDrive. The latter is used by simulation colleagues. The generated map can be adjusted according to requirements. Minor problems are quickly iterated through patching to avoid wasting time on system changes.
FAW-NJ has built two large-scale maps so far. One is the headquarters campus in Changchun, which was built by themselves. The other is the publicly accessible roads in the Nanjing pilot zone, a 5 square kilometer area designated by the Nanjing government for self-driving vehicles similar to the self-driving demonstration area in Anting, Shanghai. There are also several small maps near the company for specific testing projects.
It can be seen that the mileage of the maps and map vendors cannot be compared. This also confirms the previous statement that having basic mapping capabilities is only to support the needs of other modules in the company, not to compete with map vendors.
After completion of the mapping, it needs to be checked. Last year, the importance of this issue was not fully realized. The map file was directly used for road tests, and various departments tried it out and reported issues for modification. This was very costly. First, it is difficult to track and find problems when other business departments raise concerns about the map when they are not aware of the problems in the map on initial release. Secondly, the manpower cost of road testing is very high. Everyone prepares for a while to go to the test site, finds that there is a problem with the map, and then starts to feedback and modify it. This situation is not good for the overall team and constantly puts the map team to the test. Other teams may have complaints because of waiting for problem finding and modification.
Chen Xuejuan stated that after exploration, a three-level map inspection system has now been formed. Firstly, the code-level test will inspect the map file for any potential problems, such as omitting actual traffic lights due to negligence. Secondly, simulation will be used to identify any potential issues. Lastly, road tests will be conducted. Key issues have already been checked and modified during the inspection and modification process before final delivery. Thus, the probability of common errors is reduced, and the quality of the maps delivered is higher than before. At least, other teams will not frequently encounter problems, and complaints will reduce accordingly. In case of urgency, these three types of inspections can be performed in parallel, greatly reducing waiting time on the test site and corresponding modifications, while saving human and material resources.
So far, the map team believes that quality inspection is a crucial part that requires the establishment of a systematic approach, which can select different inspection criteria based on different needs.
“`Currently, quality inspection is composed of a set of rules. From January to March, over 20 rules have been written, and some have not been included yet. Each rule has strict levels which can be classified into three categories: ERROR, WARNING, and INFO. These three attributes indicate the severity of the violation of the rule.
For example, if a map is missing a key element, the checking item for the element will not pass, and if the attribute for checking key elements is error, it means that this inspection has violated one rule with the error attribute. In this case, it is highly likely that the conclusion of this inspection will be a fail, indicating that the map is not qualified.
Another example is the inspection of narrow lanes. Some scenarios may allow vehicles to pass smoothly, while others may make it impossible to pass due to the lane being too narrow. This lane width checking item is marked with the attribute warning because it only causes a failure in some scenarios.
At this point, it is impossible to determine whether there is a problem with the quality of the map. A specific analysis is needed. If the annotator has annotated it incorrectly, it is a quality issue of the map. If the actual scenario is that the lane is very narrow, then the annotator should still annotate it as very narrow according to the real-world situation.
There are also INFO messages, such as displaying the types of elements, including the entire map range. This requires manual inspection because it is impossible to explain in the rules how many types of elements there are in this map, only humans can judge such areas and know which map corresponds to which location.
Therefore, this type of information can only be read from the map and displayed to humans in a textual and semantic way for confirmation.
Chen Xuejuan admits that this system has been in development for three months and is still in a semi-automatic state, where manual execution of the items is required. There are still 20-30 inspection rules in the entire rule library that need to be manually selected and executed. The current checklist is still manual and not user-friendly. If the library increases to 100 rules, the person in charge may collapse. This is when an automated inspection system needs to be implemented.
She said that they plan to use the idea of continuous integration to do automatic detection in the later stages. The process will involve selecting the required items and manually picking the good ones. Then, the map file will be handed over to the system, which will then perform the inspections one by one. Finally, a report will be generated, and emails will be sent and archived to avoid the current situation of manual execution of items and then filling out the checklist. It is expected that a prototype of the automated system will be available by the end of this year.Except for an automated framework, the core of the inspection system is the inspection rule library, which is an expert system that has no references and needs to be kept confidential. It will gradually grow and become more complex over time. Currently, the Nanjing company has applied for a patent for this system, and there may be software copyright output in the future. It may even be applied to mass production projects after maturity to ensure the quality of the maps delivered by the company.
For internal needs, Chen Xuejuan stated that she has the ability to create small-scale maps to support other modules and fulfill the internal team’s demands. Using the company’s maps can be costly due to the long delivery cycle and short mapping range. Creating small internal maps can save costs and respond quickly.
There is no doubt that in the later stages of production, the company will still purchase maps from map vendors because it is not cost-effective to obtain A-level qualifications themselves.
She emphasized that the map inspection system is the focus of the future and the characteristic of the mapping business of FAW Nanjing. The system will be continually improved to become more automated, and more rules will be added. By accumulating experience during the research and development phase, the company can also control the maps purchased from the vendors during the mass production phase. It is not that they do not trust the vendors completely, but they cannot rely entirely on them.
What is more important is that the map vendors’ maps will undergo two inspections: one during map creation and one when the host factory accepts the maps. Although there may be overlap in the inspections, they provide double assurance and make the maps more reliable. This system is not only useful for L4 autonomous driving but also adaptable for L3 after making the corresponding modifications.
This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.