Absolute sample on 18432 CUDA cores and 64 TFLOPs graphical horsepower

Absolute sample on 18432 CUDA cores and 64 TFLOPs graphical horsepower

by

We have a very delicious rumor circulating today and it comes from a very credible source. @ kopite7kimi, the Twitter leak responsible for almost all of the Ampere leaks, has revealed a bit about the upcoming NVIDIA Ada Lovelace architecture (probably called NVIDIA ADA). Our colleagues at 3DCenter extrapolate a lot of information from the matrix size, which Kopite apparently more or less confirmed. Although the source is very reliable, we still mark this post as a rumor due to the extent of the leak.

NVIDIA ADA GPU leaked: Sample 64 TFLOPs GPU with 18432 CUDA cores and 5nm process architecture

The Ada Lovelace architecture – which is probably just called NVIDIA ADA – was recently leaked by Kopite (and confirmed by Videocardz) and it looks like we already have the preliminary specifications of NVIDIA’s upcoming GPU. As we mentioned in the original Ada article, it seems that Hopper was temporarily delayed (and with it the NVIDIA’s MCM ambitions). Fortunately, it seems that NVIDIA has kept its pedal to the metal and its Ada architecture, which is favored by the AD102 GPU, will be an absolute beast. Below is the original leak of the matrix size:

The people at 3DCenter quickly extrapolated a bunch of details (we revised their TFLOP numbers to be a little more conservative with a 1.75 GHz clock), which Kopite confirmed:

For those who want all the information in one place, here is a table that summarizes everything:

NVIDIA Lovelace AD102 Rumored GPU Specifications

GPU AD102 GA102 TU102
Architecture Ada Lovelace Ampere Turing
Process 5nm Samsung 8nm TSMC 12nm NFF
Graphic Processing Groups (GPC) 12 7 6
Texture Processing Clusters (TPC) 72 42 36
Multiprocessor Power (SM) 144 84 72
CUDA core 18432 10752 4608
Theoretical TFLOPs 64.5 37.6 16.1
Release March 2017 September 19 2022 (TBC)

The NVIDIA AD102 “ADA GPU” apparently has 18432 CUDA cores, based on the information provided by Kopite. It is almost twice the core in Ampere, which was already a big step away from Turing. The only way this is even possible is because NVIDIA seems to be building on the 5nm process, which has a significant matric area and power reduction. Interestingly, if you assume a clock speed of 1.75 GHz, you can also achieve the ADA 102 GPU’s highest single precision performance: 64 TFLOPs.

According to Kopite, the ADA architecture will contain a much larger L2 cache (both Turing and Ampere have 6 MB of cache), which means that it could be an important architectural review (as Turing was for Pascal and Pascal for Fermi / Kepler) instead of just shrinking the usual process. It is also unclear at the moment whether NVIDIA will use Samsung’s 5nm process or TSMC. Although the company has had poor returns at Samsung in the past, the fact that TSMC has been completely suffocated and for some time no longer has access to the capacity means that NVIDIA may be more willing to tap Samsung and ‘unlimited’ production.

NVIDIA ADA GPU Architecture Summary

In many ways, Ada Lovelace can be considered the world’s first computer lover. She is the first person to realize that the analytical engine proposed by Charles Babbage had out-of-the-box applications and also published what is thought to be the first algorithm (becoming the first computer programmer) carried by such a machine . It was almost half a century before Alan Turing would complete their work and invent the general computer during World War II.

NVIDIA is known to base its architectures on leading physicists, mathematicians and scientists, and Ada Lovelace is no different. Videocardz has actually managed to find an important tip in NVIDIA’s own retail store, which apparently confirms this rumor that Lovelace architecture is the next generation of GPUs of the company. If you look at the heroes on display during GTC’s keynote address for 2018, you will find not only Ada Lovelace, but what are possibly all future architectural codenames of NVIDIA. Jensen may have left the entire future road map (as far as code names are concerned) sly in the GTC’18 chapter.

There are now several rumors suggesting that Lovelace architecture will be based on a 5nm process. Since NVIDIA switched to Samsung’s foundry, it’s unclear whether 5nm refers to a TSMC process or Samsung. However, remember that a recent report from Korea also confirmed an instruction from NVIDIA at 6nm – meaning that there is another generation of NVIDIA ahead of Lovelace, or that the 6nm process was for the refresh setup.

Source