Personal blog of Yzmir Ramirez

Nvidia today announced that its new Ampere-based data center GPUs, the A100 Tensor Core GPUs, are now available in alpha on Google Cloud. As the name implies, these GPUs were designed for AI workloads, as well as data analytics and high-performance computing solutions.

The A100 promises a significant performance improvement over previous generations. Nvidia says the A100 can boost training and inference performance by over 20x compared to its predecessors (though you’ll mostly see 6x or 7x improvements in most benchmarks) and tops out at about 19.5 TFLOPs in single-precision performance and 156 TFLOPs for Tensor Float 32 workloads.

Image Credits: Nvidia

“Google Cloud customers often look to us to provide the latest hardware and software services to help them drive innovation on AI and scientific computing workloads,” said Manish Sainani, Director of Product Management at Google Cloud, in today’s announcement. “With our new A2 VM family, we are proud to be the first major cloud provider to market Nvidia A100 GPUs, just as we were with Nvidia’s T4 GPUs. We are excited to see what our customers will do with these new capabilities.”

Google Cloud users can get access to instances with up to 16 of these A100 GPUs, for a total of 640GB of GPU memory and 1.3TB of system memory.

In the chip game, more is usually better, and to that end, Ampere announced the next chip on its product roadmap today, the Altra Max, a 128-core processor the company says is designed specifically to handle cloud-native, containerized workloads.

What’s more, the company has designed the chip so that it will fit in the same slot as their 80-core product announced last year (and in production now). That means that engineers can use the same slot when designing for the new chip, which saves engineering time and eases production, says Jeff Wittich, VP of products at the company.

Wittich says that his company is working with manufacturers today to make sure they can build for all of the requirements for the more powerful chip. “The reason we’re talking about it now, versus waiting until Q4 when we’ve got samples going out the door is because it’s socket compatible, so the same platforms that the Altra 80 core go into, this 128-core product can go into,” he said.

He says that containerized workloads, video encoding, large scale out databases and machine learning inference will all benefit from having these additional cores.

While he wouldn’t comment on any additional funding, the company has raised $40 million, according to Crunchbase data, and Wittich says they have enough funding to go into high-volume production on their existing products later this year.

Like everyone, the company has faced challenges keeping a consistent supply chain throughout the pandemic, but when it started to hit in Asia at the beginning of this year, the company set a plan in motion to find backup suppliers for the parts they would need should they run into pandemic-related shortages. He says that it took a lot of work, planning and coordination, but they feel confident at this point in being able to deliver their products in spite of the uncertainty that exists.

“Back in January we actually already went through [our list of suppliers], and we diversified our supply chain and made sure that we had options for everything. So we were able to get in front of that before it ever became a problem,” he said.

“We’ve had normal kinds of hiccups here and there that everyone’s had in the supply chain, where things get stuck in shipping and they end up a little bit late, but we’re right on schedule with where we were.”

The company is already planning ahead for its 2022 release, which is in development. “We’ve got a test chip running through five nanometer right now that has the key IP and some of the key features of that product, so that we can start testing those out in silicon pretty soon,” he said.

Finally, the company announced that it’s working with some new partners, including Cloudflare, Packet (which was acquired by Equinix in January), Scaleway and Phoenics Electronics, a division of Avnet. These partnerships provide another way for Ampere to expand its market as it continues to develop.

The company was founded in 2017 by former Intel president Renee James.

Ampere, the chip startup run by former Intel President Renee James, announced a new chip today that she says is designed specifically to optimize for cloud workloads.

Ampere VP of product Jeff Wittich says the new chip is called the Ampere Altra, and it’s been designed with some features that should make it attractive to cloud providers. This involves three main focuses including high performance, scalability and power efficiency — all elements that would be important to cloud vendors operating at scale.

The Altra is an ARM chip with some big features.”It’s 64-bit ARM cores or 160 cores in a two-socket platforms –we support both one socket and two socket [configurations]. We are running at 3 GHz turbo, and that’s 3 GHz across all of the cores because of the way that cloud delivers compute, you’re utilizing all the cores as much of the time as possible. So our turbo performance was optimized for all of the cores being able to sustain it all the time,” Wittich explained.

The company sees this chip as a kind of workhorse for the cloud. “We’ve really looked at this as we’re designing a general purpose CPU that is built for the cloud environment, so you can utilize that compute the way the cloud utilizes that type of compute. So it supports the vast array of all of the workloads that run in the cloud,” he said.

Founder and CEO James says the company has been working with their cloud customers to give them the kind of information they need to optimize the chip for their individual workloads at a granular configuration level, something the hyper scalers in particular really require.

“Let’s go do what we can to build the platform that delivers the raw power and performance, the kind of environment that you’re looking for, and then have a design approach that enables them to work with us on what’s important and the kind of control, that kind of feature set that’s unique because each one of them have their own software environment,” James explained.

Among the companies working with Ampere early on have been Oracle (an investor, according to Crunchbase) and Microsoft, among others.

James says one of the unforeseen challenges of delivering this chip is possible disruptions in the supply chain due to the Corona-19 virus and its impact in Asia where many of the parts come from, and the chips are assembled.

She says the company has taken that into consideration and has been able to build up a worldwide supply chain she hopes will help with hiccups that might occur because of supply chain slow downs.