Colossus AI Hits 200,000 GPUs as Musk Ramps Up AI Ambitions

Shutterstock

Elon Musk’s Colossus AI infrastructure, said to be one of the most powerful AI computing clusters in the world, has just reached full operational capacity. Designed to push the boundaries of AI, this massive computing system now consists of 200,000 GPUs, all running on Tesla Megapack batteries. This is a significant milestone in Musk’s growing push into AI.  

With the on-site substation going online and connecting to the main power grid, phase 1 of Colossus AI infrastructure, located in Memphis, TN, is now complete. The supercomputer is now running at 150 MW from the grid, according to the Greater Memphis Chamber. The additional 150-megawatt Megapack battery system will act as a backup power source, ensuring continued operation during outages or periods of heightened electricity demand.

Colossus AI is the flagship product of Musk’s official AI company, xAI. The supercomputer was first activated in July last year with 100,000 Nvidia GPUs, after being built at an astonishing pace. The entire project was completed in 122 days, while the hardware installation to training phase took only 19 days. The pace of the project impressed Nvidia CEO Jensen Huang, who pointed out that projects of this scale typically take around four years, making its deployment remarkably fast. 

Shutterstock

“As far as I know, there’s only one person in the world who could do that,” said Huang. “Elon is singular in his understanding of engineering and construction and large systems and marshaling resources; it’s just unbelievable.”

However, the speed came at a cost, as the facility initially lacked a direct connection to the power grid. To keep operations running, the site depended on natural gas turbine generators for electricity, raising concerns about emissions and sustainability.

Early reports suggested 14 turbines were supplying power, each generating 2.5 MW, but observations from residents indicated the number may have exceeded 35 in the surrounding area. That is more than twice the permitted limit. This reliance on temporary power sources had sparked discussions about the long-term energy plan for the facility, especially as xAI looks to scale up operations further.

Adding more GPUs to the infrastructure means that the AI cluster can now rely more on grid power rather than gas-powered generators. This will help improve efficiency and address environmental concerns. Reportedly, xAI plans to remove half the temporary generators by the end of the summer. The other half of the temporary generators will have to remain to deliver the electrical needs of the second phase of the Memphis Supercluster.

Musk plans to double the capacity of Colossus AI before the end of this year. Another 150 MW is going to be added, taking the total capacity to 300 MW. This translates to powering 300,000 homes. It’s not surprising that this massive power demand has sparked concerns about whether the Tennessee Valley Authority (TVA) has sufficient capacity to support it. 

xAI has publicly stated plans to expand its Colossus supercomputer to over 1 million GPUs.For the local economy, Colossus AI promises economic development and infrastructure investment. However, concerns persist regarding disruptions to power quality for residents and the project’s environmental impact.

Shutterstock

“You don’t become the moniker for technological innovation because someone comes in and exploits your natural resources, your water, exploits the loopholes that allow them to pollute the air,” said KeShaun Pearson, the director of the grassroots organization Memphis Community Against Pollution (MCAP). “That’s not what makes you a technological city. That spin is dangerous because it opens our city up for exploitation even further.”

The road to powering a million GPUs started when Musk founded xAI in July 2023. The stated goal of “understanding the true nature of the universe.” In more practical terms, Musk wanted an AI lab under his own direction, free from the influences of Microsoft, Google, or other major tech firms.

The company is an answer to the growing dominance of OpenAI (which now has Microsoft as a close partner) and Google’s DeepMind. xAI is also integrated with Musk’s other ventures, including SpaceX and Tesla. With Colossus now operating at full capacity, xAI is positioned to accelerate the development and deployment of AI across Musk’s broader ecosystem.

Related Items 

Big Data Smackdown: Data Management Vs. Data Governance

AI Today and Tomorrow Series #4: Frontier Apps and Bizops

How to Build a Lean AI Strategy with Data