Microsoft is making available in Azure new infrastructure to train extremely large artificial intelligence models, specifically a supercomputer designed to train OpenAI’s AI models.
This latest development is considered as a first step toward making the next generation of very large AI models and the infrastructure needed to train them available as a platform for other organisations and developers to build upon.
“The exciting thing about these models is the breadth of things they’re going to enable,” said Microsoft CTO Kevin Scott, adding that the potential benefits extend far beyond narrow advances in one type of AI model.
“This is about being able to do a hundred exciting things in natural language processing at once and a hundred exciting things in computer vision, and when you start to see combinations of these perceptual domains, you’re going to have new applications that are hard to even imagine right now,” Scott said.
The supercomputer developed for OpenAI is a single system with more than 285,000 CPU cores, 10,000 GPUs and 400 gigabits per second of network connectivity for each GPU server.
According to Microsoft, their machine ranks fifth compared with other machines listed on the TOP500 supercomputers in the world.
Hosted in Azure, the supercomputer also benefits from all the capabilities of a robust modern cloud infrastructure, including rapid deployment, sustainable data centres and access to Azure services.
“As we’ve learned more and more about what we need and the different limits of all the components that make up a supercomputer, we were really able to say, ‘If we could design our dream system, what would it look like?’” said OpenAI CEO Sam Altman. “And then Microsoft was able to build it.”
OpenAI’s goal is not just to pursue research breakthroughs but also to engineer and develop powerful AI technologies that other people can use, said Altman. The supercomputer developed in partnership with Microsoft was designed to accelerate that cycle.