Tesla Sends Invites for AI Day

On August 4, 2021, Tesla began sending out invitations for its AI Day. During this event, the company will unveil its supercomputer, Dojo, designed for deep neural network training.

Just a month earlier, Tesla released its FSD Beta 9 to early access users after 128 days of development. CEO Elon Musk confirmed that, after 2-3 more iterations, FSD Beta will be rolled out on a larger scale across the United States.

These events indicate that Tesla’s FSD (Full Self-Driving) technology, which has faced years of delays and resulted in lawsuits, is making significant breakthroughs.

However, as attention focuses on Autopilot’s products and technological advancements, Elon emphasized the importance of talent within the company, stating that “talent is critical, like a sports team where having the best players typically wins, but strategy and teamwork can make a huge difference in outcomes.”

Following this statement, let’s delve into Tesla’s Autopilot department, including its talent, operating mechanisms, and technology vision.

Hands-On Approach and Elite Efficiency

The organizational structure within the Autopilot department at Tesla has undergone several major adjustments. Since Stuart Bowers, the former Vice President of Autopilot engineering, left the company in August 2019, the department maintained a relatively stable organizational structure, as shown in the diagram below.

Tesla’s Autopilot department is characterized by two key features: Elon’s hands-on approach and elite efficiency.

After chip guru Jim Keller left Tesla in April 2018, no executives were specifically responsible for Autopilot. During Tesla’s Q1 financials call in May, Elon stated that the company did not seek a replacement for Jim’s position. Instead, the role was split between several Vice Presidents and Senior Directors, all reporting directly to Elon himself. As CEO, Elon is ultimately responsible for Autopilot’s operations.From an architectural standpoint, Autopilot currently has five executives reporting directly to Elon. They are Pete Bannon, Vice President of Hardware Engineering and Low Voltage Electronic Systems; Andrej Karpathy, Senior Director leading Tesla AI (not limited to Autopilot); Ashok Elluswamy, Director responsible for full-stack algorithms; Milan Kovac, Director of Custom Linux Kernels and Low-Level Code; and CJ Moore, Director of Software Integration and Validation.

However, in fact, many former Autopilot executives, including Jim Keller, have stated that Elon is deeply involved in supervising every aspect of Autopilot system engineering and will intervene in any component at any time, such as the machine learning compiler led by Bill McGee.

So far, Autopilot’s development still follows the principle of “California Leads the World”. The Autopilot development team is entirely located within the USA, including the software and AI teams in Fremont, California, the hardware engineering teams in California and Texas Austin, and the video annotation teams in California and New York.

In addition, Tesla has also set up Autopilot product support positions in Beijing, Shanghai, Chongqing, Sydney, Paris, and many other places in the USA to collect regional market bugs and development needs to improve Autopilot.

Now to talk about the second characteristic, “fight with elite soldiers.” Almost every autonomous driving company has an algorithmic genius who oversees the full-stack algorithm of the autonomous driving system, such as Ren Shaoqing from NIO, Yilun Wang from Li Auto, and Tiancheng Lou from Pony.ai. But Andrej Karpathy of Tesla is different.

When Andrej joined Tesla in June 2017, he assembled an AI team of about 30 people. After several years of fleet growth and algorithm evolution, in June 2021, this team decreased to about 20 people.

A more convincing proof is that when Jim Keller was at Tesla, he had only slightly more than 100 engineers under his leadership. When he left Tesla for Intel, the engineering team he led had expanded to over 10,000 people, a 100-fold increase in scale.In the past 5 years, the responsibilities of Tesla’s Autopilot hardware engineering team, consisting of just over 100 people, have continued to expand. For example, Ganesh Venkataramanan, who was responsible for the Tesla FSD chip silicon and system, is now focused on the Dojo supercomputer business. However, the overall size of the hardware engineering team has not increased significantly.

As of today, Tesla’s Autopilot manages a global fleet of over 1.6 million vehicles (calculated based on the HW 2.0+ vehicle model equipped with eight cameras), but excluding the labeling and product support team, the core team still maintains a strength of about 300 people.

As a comparison, as of April this year, the team size of Huawei’s car BU intelligent driving solution department has exceeded 2,000 people. In the foreseeable future, the autonomous driving teams of NIO, Li Auto, and XPeng will all exceed Tesla’s team size.

This is related to Elon’s management philosophy formed in SpaceX. In 2020, SpaceX launched 26 times throughout the year, with a space payload of 247 tons, accounting for 54% of the total global payload for the year. At the same time, it launched 14 batches of Starlink satellites, making it the world’s largest satellite operator. However, the entire SpaceX team in 2020 amounted to only 8,000 people.

Elon’s response to this is, “It is a mistake to hire huge numbers of people to get a complicated job done. Numbers will never compensate for talent in getting the right answer (two people who don’t know something are no better than one), will tend to slow down progress, and will make the task incredibly expensive.”

It’s hard to deny the efficiency brought by a streamlined team. In terms of progress, FSD Beta’s update frequency of up to 7 days per release is undoubtedly a model of industry execution efficiency. But at the same time, perhaps due to prioritization issues caused by R&D resources, FSD Beta is experiencing long-term stagnation in overseas markets such as Greater China and Europe.

Knights of the Round Table and No Boss Mechanism

How much does ElonMusk value Autopilot? According to Andrej, Elon regularly attends meetings to learn about the progress of Autopilot, with meeting frequency ranging from once a day to once a week depending on whether a new version of Autopilot is nearing release and needs review.

What about the Autopilot team during non-meeting times?

In an interview last year, Elon declared that the less than 200 engineers in the Autopilot software department could find work wherever they wanted at Tesla and that there was no one who could really be their boss.

This statement is largely interpreted as an appreciation of the Autopilot team or as an unusual public relations move to showcase Tesla’s talent density.

But in a subsequent praise by a netizen for Andrej (who previously recorded the popular deep learning course CS321n at Stanford University and, in addition to his previous role as a research scientist at OpenAI, also had internship experience at Google Brain and Deepmind, making him a super internet celebrity in the field of artificial intelligence), Elon commented, “Andrej is great, but it should be said that Tesla has a team of genius Autopilot/AI. Andrej and I are getting too much credit.”

On July 10, Elon further explained the operating mechanism of the Autopilot department. “The Autopilot software is technically led by Ashok, Andrej, and Milan, but it is largely a ‘knights of the round table’ structure. There are many talented engineers in the Autopilot/AI team who decide for themselves what they want to do. It’s a bit like Valve.”The explanation contains significant information. Firstly, the knights of the round table, who first appeared in the legend of King Arthur written by Robert Wace in 1155, symbolize “equality”. There was no distinction in status between knights, and everyone was allowed to express their opinions.

Therefore, there is no distinction in status between the executives of Autopilot, and everyone is allowed to express their opinions. This is Elon’s description of the operating mechanism of the Autopilot team.

Another keyword, Valve, is well-known to many. This company, which created DOTA2, Half-Life, Team Fortress 2, and most importantly, Steam, is undoubtedly one of the most successful gaming companies in the world.

Valve’s organizational structure has attracted attention and can be traced back to nearly a decade ago. For a long time, Valve was wholly owned by founder Gabe Newell, and there were no external investors demanding more from Valve’s operations, allowing Gabe to fully express his organizational ideas.

In short, as the CEO, Gabe did his utmost to avoid interfering with daily operations or even larger decisions. Valve has no managers or formal positions, and all employees are free to pursue any project they find the most interesting.

In contrast, Gabe believes that recruiting is the most important function of the company, and Valve highly values recruiting employees who can blend into the Valve culture and actively contribute to projects. To facilitate employees freely joining and quitting a project, Valve’s workstations are designed to be mobile. There is even a separate workstation relocation guide in Valve’s employee handbook.

Therefore, from Valve, we can obtain the second mechanism of Tesla’s Autopilot department, which is a high degree of self-drive and team collaboration abilities.To this point, looking back, we can notice that there is a certain underlying logic in the high talent density and exacting battle of the Autopilot department at Tesla and the “Round Table Knights” Noboss operating mechanism. Only a talented team with high-quality and self-driven abilities can collaborate effectively in an equal but mostly leaderless operating mechanism.

Despite the repeated postponements in the commercialization of FSD, judging from Elon’s attitude, he is satisfied with the operation of the Autopilot department.

## The Technical Vision of the Global Optimal Solution

After discussing those “no meeting” times, let’s revert back to “Elon’s meeting time”. Without Elon, it is hard to imagine that there would be any executive at Tesla who has the courage to order the removal of all the millimeter-wave radars in the Model 3/Y produced in North America after May 25, and fully turn to the technology route of pure visual perception for autonomous driving.

This is the fundamental difference between the Autopilot department at Tesla and Valve. The Valve CEO has always tried his best to avoid intervening in any specific project, while Elon needs to make some crucial decisions on Autopilot.

The biggest controversial issue around Autopilot is undoubtedly the technology route of pure visual perception that does not use LIDAR, millimeter-wave radar, or rely on high-precision maps based on eight cameras.

Before this, there were many seemingly nearly true and even convincing reasons why LIDAR was not adopted in Tesla’s perception architecture in Autopilot 2.0 hardware that was installed in October 2016. For example, LIDAR had the following major problems:

There were no supply chain resources that met vehicle regulations.

The cost was too high.

The integration degree was low, and the volume was too large.

Each of the above was unacceptable for Tesla. But Tesla’s strategy of removing radar on May 25 made none of these logics internally consistent anymore. For the radar in 2021, there were supply chain resources that met vehicle regulations, the cost was acceptable, and the integration process was mature. Similar to other automakers, Tesla had achieved the integration of radar inside the front cabin as early as 2014. But in the end, Tesla removed radar.Another positive logic suggests that Elon’s core idea on Autopilot is biomimetics. If a person can achieve the necessary perception for driving through vision dominated by the eyes, then an autonomous vehicle can do it too. However, in fact, Elon has never said this sentence. People are familiar with this sentence maybe because it has been posted on the official website of another visual perception giant, Mobileye, for a long time?

Elon’s viewpoint on LIDAR and high-precision maps is that they would trap autonomous driving in local optimal solutions. This expression first appeared in Tesla’s Q4 and full-year earnings conference in 2017.

Many companies rely on LiDAR to help cars perceive, which is very puzzling. In my opinion, this (LiDAR) is a crutch that will propel other companies into local optimal solutions that they are difficult to get rid of.

In October 2020, when commenting on a netizen who said “Waymo’s autonomous driving car must be able to drive in areas covered by high-precision maps, without outdated information and conflicting perception”, Elon commented:

Indeed. We have been wrong about the goal for too long. It gives people a misleading sense of near victory- an attractive local optimal solution- but the real world is too chaotic and strange. Our new system can drive automatically in places we have never seen before.

He also used similar expressions when evaluating SpaceX’s Merlin engine design and American spacecraft. “I think we need to be very careful to avoid getting stuck in local optimal solutions.”

We can see that these local optimal solutions in these contexts all belong to routes with low system scalability but easier engineering implementation. The global optimal solution pursued by Tesla or SpaceX is the strongest system with theoretical performance that existing technology can reach, but there is a huge engineering gap between theory and commercialization, making it extremely difficult to land. This is true for both Autopilot and Starship.

Undoubtedly, the choice of technological path will have a decisive impact on the future of Tesla’s autonomous driving. Elon’s judgment of the technological vision will influence Tesla’s autonomous driving, and thus the company’s position in the automotive industry. Therefore, we want to hear the opinions of executives who participate in building the pure visual Autopilot, and how they view Elon’s technological vision.First up is chip guru Jim Keller. According to Jim, in addition to developing the FSD chip, he was also involved in the development of the Dojo supercomputer project before leaving Tesla:

“I really like the way he (Elon) thinks about problems… It’s like you think you know what first principles are and then you talk to Elon about everything you know and you realize you haven’t even scratched the surface. He’s a firm believer that whatever you’re doing is only a local maxima.”

Elon is very good at breaking everything down into its parts, like what are the true first principles, what’s the true way to look at things without assumptions, and how to constrain it? It’s super crazy.

Another executive, Andrej Karpathy, has been with Tesla for 5 years, and had many opportunities to communicate directly with Elon in the two years before joining Tesla when he was at OpenAI. His comment is longer and more detailed:

In many ways, he (Elon) is obviously an incredibly incredible person. I’m still trying to tap into his superpower.

His intuition is incredibly sharp. I think he makes the right judgments in many ways, and sometimes I think he lacks information because he doesn’t know everything about everything. But his judgment is very good. I haven’t fully understood how this happens.

He has a way of simplifying a very complex system, like what are the fundamental and truly important first principles components of a system? And then articulating them. I find this to be a very different way of thinking.

Sometimes I get overwhelmed by systems. I feel like I need to know everything about a system in order to make the right decision, but that’s not his mode of operation. He somehow has a way of distilling a system into a simpler system.

Whether in employment or not, Jim and Andrej are undoubtedly followers of Elon’s technical vision. In today’s internet world where everyone talks about “first principles” and minimalist design, their evaluations are worth pondering.

Technology, products, experience, and design are all comparables and even replicable, but today’s many top-tier autonomous driving companies can no longer rely on these strategies to continue moving forward. Apart from innovation, there is no other strategy that can achieve long-term competition in this new category of autonomous driving.

In fact, it is difficult to say whether Tesla’s pursuit of a global optimum solution is right or wrong until autonomous driving cars are truly put into practice. Today, everyone in the autonomous driving unknown territories, including Apple, Waymo, Huawei, and Tesla, is taking every step forward in creating history, or trial and error. And the only thing that is certain is that all autonomous driving companies except Tesla choose to embrace LIDAR, or high-precision maps, or both – in short, non-pure visual routes.In July 2018, Elon told Bloomberg reporters that Model 3 was Tesla’s last project that was bet on the whole company. Now, FSD fully self-driving, which is even more of a gamble than Model 3, has come to the center of the table.

This article is a translation by ChatGPT of a Chinese report from 42HOW. If you have any questions about it, please email bd@42how.com.