What is Meta's New AI Supercomputer?

Social media conglomerate Meta is getting serious about AI.

In June last year Tesla unveiled it AI supercomputer. At the time it was the (5th most powerful in the world) to train self-driving AI. It is being used to train the neural nets powering Tesla’s Autopilot and upcoming self-driving AI called Dojo supercomputer.

Six months later not to be undone Meta has a similar plan. So why are BigTech companies working on supercomputers to train AI?

Meta has built an AI supercomputer it says will be world’s fastest by end of 2022

Social media conglomerate Meta is investing around $10 billion a year on the Metaverse, but it needs better AI to power that Metaverse dream as well.

The company says its new AI Research SuperCluster, or RSC, is already among the fastest machines of its type and, when complete in mid-2022, will be the world’s fastest.

What is RSC?

• Meta said in a blog post that its new AI Research SuperCluster (RSC) would help the company build better AI models that can learn from trillions of examples, work across hundreds of languages, and analyze text, images and video together to determine if content was harmful.

• You can read Meta’s blog post here.

• Meta announced on January 24th, 2022 that developing the next generation of advanced AI will require powerful new computers capable of quintillions of operations per second.

Mark Z. says Meta is calling it RSC for AI Research SuperCluster and it’ll be complete later this year in 2022.

What is a Supercomputer Anyway?

Today’s supercomputers are made up of thousands of connected processors, and their speed has grown exponentially over the past few decades.

The first supercomputer, released in 1964, was called the CDC 6600.

“This research will not only help keep people safe on our services today, but also in the future, as we build for the metaverse,” the company said in a blog post.

BigTech is Racing to Develop Powerful Supercomputers to Train A.I.

The race to AI Supremacy among BigTech companies is increasingly a processing race as well. If you think about it Meta is just keeping up with its competitors too. The news demonstrates the absolute centrality of AI research to companies like Meta. Rivals like Microsoft and Nvidia have already announced their own “AI supercomputers,” which are slightly different from what we think of as regular supercomputers.

Meta researchers have already started using RSC to train large models in natural language processing (NLP) and computer vision for research, with the aim of one day training models with trillions of parameters.

• Each year companies are finding new reasons to develop supercomputing to train AI better.

• RSC will be used to train a range of systems across Meta’s businesses: from content moderation algorithms used to detect hate speech on Facebook and Instagram to augmented reality features that will one day be available in the company’s future AR hardware.

Next-Gen AI will Help Power the Metaverse

Ultimately, the work done with RSC will pave the way toward building technologies for the next major computing platform — the metaverse, where AI-driven applications and products will play an important role.

With A.I. developing more rapidly, at the advent of Web 3.0 and the Metaverse, it’s actually a pretty exciting time to be a futurist enthusiast and to watch what BigTech is doing.

Meta says RSC will be used to design experiences for the metaverse — the company’s insistent branding for an interconnected series of virtual spaces, from offices to online arenas. Well we finally know where some of that huge $10 Billion per year investment in the Metaverse by Meta is going it appears.

The company said it believed the supercomputer was currently among the fastest AI supercomputers running, or will be later in 2022.

A.I. Unifying People at Scale in the Metaverse?

For instance, Meta hopes RSC will help them build entirely new AI systems that can, for example, power real-time voice translations to large groups of people, each speaking a different language, so they can seamlessly collaborate on a research project or play an AR game together. That sounds like something Google would be working on too, among others.

While Meta is seen among the more “evil” of the BigTech giants, all of these companies are doing research and spending on technology that ultimately benefits everyone to some degree over the span of decades.

Meta’s AI supercomputer is due to be complete by mid-2022, likely by July, 2022.

Meta has been committed to long-term investment in AI since 2013 and innovation at FAIR has been improving. Data2vec is such an example.

If you value my work on covering AI, please share it with your network and consider being a paid supporter, I cannot continue to write without significant community support.

The Scope of RSC is Pretty Insane

Phase one of RSC is already up and running and consists of 760 Nvidia GGX A100 systems containing 6,080 connected GPUs (a type of processor that’s particularly good at tackling machine learning problems). Meta says it’s already providing up to 20 times improved performance on its standard machine vision research tasks.

• The first generation of this infrastructure, designed in 2017, has 22,000 NVIDIA V100 Tensor Core GPUs in a single cluster that performs 35,000 training jobs a day. Up until now, this infrastructure has set the bar for Meta’s researchers in terms of its performance, reliability, and productivity.

• Before the end of 2022, though, phase two of RSC will be complete. At that point, it’ll contain some 16,000 total GPUs and will be able to train AI systems “with more than a trillion parameters on data sets as large as an exabyte.”

Supercomputers like RSC and Tesla’s Dojo will continue to set the part for computing power to train AI. The future of A.I. is thus a layer of how the future of technology can scale, in mega-projects like Robo-taxis or the Metaverse.

According to the Verge, the true utility of supercomputers is to be found in the work they do, not their theoretical peak performance. For Meta, that work means building moderation systems at a time when trust in the company is at an all-time low and means creating a new computing platform.

RSC: Under the hood

I hope you enjoyed this sneak peak on Facebook’s supercomputer, thanks for reading!

• Early benchmarks on RSC, compared with Meta’s legacy production and research infrastructure, have shown that it runs computer vision workflows up to 20 times faster, runs the NVIDIA Collective Communication Library (NCCL) more than nine times faster, and trains large-scale NLP models three times faster.

• That means a model with tens of billions of parameters can finish training in three weeks, compared with nine weeks before.