For years, Elon Musk has talked about Dojo \u2014 the AI supercomputer that will be the cornerstone of Tesla\u2019s AI ambitions. It\u2019s important enough to Musk that in July 2024, he said the company\u2019s AI team would \u201cdouble down<\/a>\u201d on Dojo in the lead up to Tesla\u2019s robotaxi reveal,<\/a> which happened in October.\u00a0\u00a0<\/p>\n But what exactly is Dojo? And why is it so critical to Tesla\u2019s long-term strategy?<\/p>\n In short: Dojo is Tesla\u2019s custom-built supercomputer that\u2019s designed to train its \u201cFull Self-Driving\u201d neural networks. Beefing up Dojo goes hand-in-hand with Tesla\u2019s goal to reach full self-driving and bring a robotaxi to market. FSD, which is on hundreds of thousands of Tesla vehicles today, can perform some automated driving tasks but still requires a human to be attentive behind the wheel.\u00a0<\/p>\n Tesla\u2019s Cybercab reveal has come and gone, and now the company is gearing up to launch an autonomous ride-hail service using its own fleet of vehicles in Austin this June.<\/a> Tesla also said during its 2024 fourth-quarter and full-year earnings call<\/a> at the end of January that it plans to launch unsupervised FSD for U.S. customers in 2025.\u00a0<\/p>\n Musk\u2019s previous rhetoric has been that Dojo would be the key to achieving Tesla\u2019s goal of full self-driving. Now that Tesla appears close to nearing that goal, Musk has been mum on Dojo.\u00a0<\/p>\n Instead, ever since August 2024, talk has been around Cortex<\/a>, Tesla\u2019s \u201cgiant new AI training supercluster being built at Tesla HQ in Austin to solve real-world AI.\u201d Musk has also said<\/a> it will have \u201cmassive storage for video training of FSD & Optimus.\u201d\u00a0<\/p>\n In Tesla\u2019s Q4 shareholder deck<\/a>, the company shared updates on Cortex, but nothing on Dojo.\u00a0<\/p>\n Tesla has positioned itself to spend big on AI and Dojo \u2014 and now Cortex \u2014 to reach its goal of autonomy for both cars and humanoid robots. And Tesla\u2019s future success really hinges on its ability to nail this down, given the increased competition in the EV market. So it\u2019s worth taking a closer look at Dojo, Cortex, and where it all stands today.\u00a0<\/p>\n Musk doesn\u2019t want Tesla to be just an automaker, or even a purveyor of solar panels and\u00a0energy storage systems. Instead, he wants Tesla to be an AI company, one that has cracked the code to self-driving cars by mimicking human perception.\u00a0<\/p>\n Most other companies building autonomous vehicle technology rely on a combination of sensors to perceive the world \u2014 like lidar, radar and cameras \u2014 as well as high-definition maps to localize the vehicle. Tesla believes it can achieve fully autonomous driving by relying on cameras alone to capture visual data and then use advanced neural networks to process that data and make quick decisions about how the car should behave.\u00a0<\/p>\n As Tesla\u2019s former head of AI, Andrej Karpathy, said at the automaker\u2019s first AI Day in 2021<\/a>, the company is basically trying to build \u201ca synthetic animal from the ground up.\u201d (Musk had been teasing Dojo since 2019, but Tesla officially announced it at AI Day.)<\/p>\n Companies like Alphabet\u2019s Waymo have commercialized Level 4 autonomous vehicles<\/a> \u2014 which the SAE defines as a system that can drive itself without the need for human intervention under certain conditions\u00a0\u2014 through a more traditional sensor and machine learning approach. Tesla has still yet to produce an autonomous system that doesn\u2019t require a human behind the wheel.\u00a0<\/p>\n About 1.8 million people have paid the hefty subscription price<\/a> for Tesla\u2019s FSD, which currently costs $8,000 and has been priced as high as $15,000<\/a>. The pitch is that Dojo-trained AI software will eventually be pushed out to Tesla customers via over-the-air updates. The scale of FSD also means Tesla has been able to rake in millions of miles worth of video footage that it uses to train FSD. The idea there is that the more data Tesla can collect, the closer the automaker can get to actually achieving full self-driving.\u00a0<\/p>\n However, some industry experts say there might be a limit to the brute force approach of throwing more data at a model and expecting it to get smarter.\u00a0<\/p>\n \u201cFirst of all, there\u2019s an economic constraint, and soon it will just get too expensive to do that,\u201d Anand Raghunathan, Purdue University\u2019s Silicon Valley professor of electrical and computer engineering, told TechCrunch. Further, he said, \u201cSome people claim that we might actually run out of meaningful data to train the models on. More data doesn\u2019t necessarily mean more information, so it depends on whether that data has information that is useful to create a better model, and if the training process is able to actually distill that information into a better model.\u201d\u00a0<\/p>\n Raghunathan said despite these doubts, the trend of more data appears to be here for the short-term at least. And more data means more compute power needed to store and process it all to train Tesla\u2019s AI models. That is where Dojo, the supercomputer, comes in.\u00a0<\/p>\n Dojo is Tesla\u2019s supercomputer system that\u2019s designed to function as a training ground for AI, specifically FSD. The name is a nod to the space where martial arts are practiced.\u00a0<\/p>\n A supercomputer is made up of thousands of smaller computers called nodes. Each of those nodes has its own CPU (central processing unit) and GPU (graphics processing unit). The former handles overall management of the node, and the latter does the complex stuff, like splitting tasks into multiple parts and working on them simultaneously. GPUs are essential for machine learning operations like those that power FSD training in simulation. They also power large language models, which is why the rise of generative AI has made Nvidia the most valuable company on the planet.\u00a0<\/p>\n Even Tesla buys Nvidia GPUs to train its AI (more on that later).\u00a0<\/p>\n Tesla\u2019s vision-only approach is the main reason Tesla needs a supercomputer. The neural networks behind FSD are trained on vast amounts of driving data to recognize and classify objects around the vehicle and then make driving decisions. That means that when FSD is engaged, the neural nets have to collect and process visual data continuously at speeds that match the depth and velocity recognition capabilities of a human.\u00a0<\/p>\n In other words, Tesla means to create a digital duplicate of the human visual cortex and brain function.\u00a0<\/p>\n To get there, Tesla needs to store and process all the video data collected from its cars around the world and run millions of simulations to train its model on the data.\u00a0<\/p>\n Tesla appears to rely on Nvidia to power its current Dojo training computer, but it doesn\u2019t want to have all its eggs in one basket \u2014 not least because Nvidia chips are expensive. Tesla also hopes to make something better that increases bandwidth and decreases latencies. That\u2019s why the automaker\u2019s AI division decided to come up with its own custom hardware program that aims to train AI models more efficiently than traditional systems.\u00a0<\/p>\n At that program\u2019s core is Tesla\u2019s proprietary D1 chips, which the company says are optimized for AI workloads.\u00a0<\/p>\n Tesla is of a similar opinion to Apple in that it believes hardware and software should be designed to work together. That\u2019s why Tesla is working to move away from the standard GPU hardware and design its own chips<\/a> to power Dojo.\u00a0<\/p>\n Tesla unveiled its D1 chip, a silicon square the size of a palm, on AI Day in 2021. The D1 chip entered into production as of at least May this year. The Taiwan Semiconductor Manufacturing Company (TSMC) is manufacturing the chips using 7 nanometer semiconductor nodes. The D1 has 50 billion transistors and a large die size of 645 millimeters squared, according to Tesla. This is all to say that the D1 promises to be extremely powerful and efficient and to handle complex tasks quickly.\u00a0<\/p>\n \u201cWe can do compute and data transfers simultaneously, and our custom ISA, which is the instruction set architecture, is fully optimized for machine learning workloads,\u201d said Ganesh Venkataramanan, former senior director of Autopilot hardware, at Tesla\u2019s 2021 AI Day. \u201cThis is a pure machine learning.\u201d<\/p>\n The D1 is still not as powerful as Nvidia\u2019s A100 chip, though, which is also manufactured by TSMC using a 7 nanometer process. The A100 contains 54 billion transistors and has a die size of 826 square millimeters, so it performs slightly better than Tesla\u2019s D1.\u00a0<\/p>\n To get a higher bandwidth and higher compute power, Tesla\u2019s AI team fused 25 D1 chips together into one tile to function as a unified computer system. Each tile has a compute power of 9 petaflops and 36 terabytes per second of bandwidth, and contains all the hardware necessary for power, cooling and data transfer. You can think of the tile as a self-sufficient computer made up of 25 smaller computers. Six of those tiles make up one rack, and two racks make up a cabinet. Ten cabinets make up an ExaPOD. At AI Day 2022, Tesla said Dojo would scale by deploying multiple ExaPODs. All of this together makes up the supercomputer.\u00a0<\/p>\n Tesla is also working on a next-gen D2 chip that aims to solve information flow bottlenecks. Instead of connecting the individual chips, the D2 would put the entire Dojo tile onto a single wafer of silicon.\u00a0<\/p>\n Tesla hasn\u2019t confirmed how many D1 chips it has ordered or expects to receive. The company also hasn\u2019t provided a timeline for how long it will take to get Dojo supercomputers running on D1 chips.\u00a0<\/p>\n In response to a June post on X<\/a> that said: \u201cElon is building a giant GPU cooler in Texas,\u201d Musk replied that Tesla was aiming for \u201chalf Tesla AI hardware, half Nvidia\/other\u201d over the next 18 months or so. The \u201cother\u201d could be AMD chips, per Musk\u2019s comment in January<\/a>.\u00a0<\/p>\n Taking control of its own chip production means that Tesla might one day be able to quickly add large amounts of compute power to AI training programs at a low cost, particularly as Tesla and TSMC scale up chip production.\u00a0<\/p>\n It also means that Tesla may not have to rely on Nvidia\u2019s chips in the future, which are increasingly expensive and hard to secure.\u00a0<\/p>\n During Tesla\u2019s second-quarter earnings call, Musk said that demand for Nvidia hardware is \u201cso high that it\u2019s often difficult to get the GPUs.\u201d He said he was \u201cquite concerned about actually being able to get steady GPUs when we want them, and I think this therefore requires that we put a lot more effort on Dojo in order to ensure that we\u2019ve got the training capability that we need.\u201d\u00a0<\/p>\n That said, Tesla is still buying Nvidia chips today to train its AI. In June, Musk posted on X<\/a>:\u00a0<\/p>\n Of the roughly $10B in AI-related expenditures I said Tesla would make this year, about half is internal, primarily the Tesla-designed AI inference computer and sensors present in all of our cars, plus Dojo. For building the AI training superclusters, Nvidia hardware is about 2\/3 of the cost. My current best guess for Nvidia purchases by Tesla are $3B to $4B this year.<\/p>\n<\/blockquote>\n \u201cInference compute\u201d refers to the AI computations performed by Tesla cars in real time and is separate from the training compute that Dojo is responsible for.<\/p>\n Dojo is a risky bet, one that Musk has hedged several times by saying that Tesla might not succeed.\u00a0<\/p>\n In the long run, Tesla could theoretically create a new business model based on its AI division. Musk has said that the first version of Dojo will be tailored for Tesla computer vision labeling and training, which is great for FSD and for training Optimus<\/a>, Tesla\u2019s humanoid robot. But it wouldn\u2019t be useful for much else.\u00a0<\/p>\n Musk has said<\/a> that future versions of Dojo will be more tailored to general-purpose AI training. One potential problem with that is almost all AI software out there has been written to work with GPUs. Using Dojo to train general-purpose AI models would require rewriting the software.\u00a0<\/p>\n That is, unless Tesla rents out its compute, similar to how AWS and Azure rent out cloud computing capabilities. Musk also noted during Q2 earnings that he sees \u201ca path to being competitive with Nvidia with Dojo.\u201d<\/p>\n A September 2023 report from Morgan Stanley predicted that Dojo could add $500 billion<\/a> to Tesla\u2019s market value by unlocking new revenue streams in the form of robotaxis and software services.\u00a0<\/p>\n In short, Dojo\u2019s chips are an insurance policy for the automaker, but one that could pay dividends.\u00a0<\/p>\n Reuters reported<\/a> last year that Tesla began production on Dojo in July 2023, but a June 2023 post<\/a> from Musk suggested that Dojo had been \u201conline and running useful tasks for a few months.\u201d<\/p>\n Around the same time, Tesla said it expected Dojo to be one of the top five most powerful supercomputers by February 2024 \u2014 a feat that has yet to be publicly disclosed, leaving us doubtful that it has occurred. <\/p>\n The company also said it expects Dojo\u2019s total compute to reach 100 exaflops in October 2024.\u00a0(One exaflops is equal to 1 quintillion computer operations per second. To reach 100 exaflops, and assuming that one D1 can achieve 362 teraflops, Tesla would need more than 276,000 D1s, or around 320,500 Nvidia A100 GPUs.)<\/p>\n Tesla also pledged in January 2024 to spend $500 million<\/a> to build a Dojo supercomputer at its gigafactory in Buffalo, New York.<\/p>\nTesla\u2019s Dojo backstory<\/h2>\n
What is a supercomputer?<\/h2>\n
Why does Tesla need a supercomputer?<\/h2>\n
Tell me more about these chips<\/h2>\n
![\"Ganesh](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/08\/Tesla-AI-Day-Dojo-Tile.png?w=680\")
What does Dojo mean for Tesla?<\/h2>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/08\/GettyImages-2162480419.jpg?w=680\")
\n
How far along is Dojo?<\/h2>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/08\/GettyImages-524212924.jpg?w=680\")