Think about a system that may discover a number of approaches to complicated issues, drawing on its understanding of huge quantities of information, from scientific datasets to supply code to enterprise paperwork, and reasoning by the probabilities in actual time. This lightning-fast reasoning isn’t ready on the horizon. It’s occurring at the moment in our clients’ AI manufacturing environments. The dimensions of the AI techniques that our clients are constructing at the moment—throughout drug discovery, enterprise search, software program growth, and extra—is actually outstanding. And there’s rather more forward.
To speed up innovation throughout rising generative AI developments akin to reasoning fashions and agentic AI techniques, we’re excited to announce common availability of P6e-GB200 UltraServers, accelerated by NVIDIA Grace Blackwell Superchips. P6e-GB200 UltraServers are designed for coaching and deploying the biggest, most subtle AI fashions. Earlier this 12 months, we launched P6-B200 situations, accelerated by NVIDIA Blackwell GPUs, for various AI and high-performance computing workloads.
On this submit, we share how these highly effective compute options construct on all the things we’ve discovered about delivering safe, dependable GPU infrastructure at a large scale, in order that clients can confidently push the boundaries of AI.
Assembly the increasing compute calls for of AI workloads
P6e-GB200 UltraServers characterize our strongest GPU providing so far, that includes as much as 72 NVIDIA Blackwell GPUs interconnected utilizing fifth-generation NVIDIA NVLink—all functioning as a single compute unit. Every UltraServer delivers a large 360 petaflops of dense FP8 compute and 13.4 TB of complete excessive bandwidth GPU reminiscence (HBM3e)—which is over 20 occasions the compute and over 11 occasions the reminiscence in a single NVLink area in comparison with P5en situations. P6e-GB200 UltraServers assist as much as 28.8 Tbps mixture bandwidth of fourth-generation Elastic Cloth Adapter (EFAv4) networking.P6-B200 situations are a flexible choice for a broad vary of AI use circumstances. Every occasion gives 8 NVIDIA Blackwell GPUs interconnected utilizing NVLink with 1.4 TB of excessive bandwidth GPU reminiscence, as much as 3.2 Tbps of EFAv4 networking, and fifth-generation Intel Xeon Scalable processors. P6-B200 situations supply as much as 2.25 occasions the GPU TFLOPs, 1.27 occasions the GPU reminiscence dimension, and 1.6 occasions the GPU reminiscence bandwidth in comparison with P5en situations.
How do you select between P6e-GB200 and P6-B200? This selection comes all the way down to your particular workload necessities and architectural wants:
- P6e-GB200 UltraServers are perfect for essentially the most compute and reminiscence intensive AI workloads, akin to coaching and deploying frontier fashions on the trillion-parameter scale. Their NVIDIA GB200 NVL72 structure actually shines at this scale. Think about all 72 GPUs working as one, with a unified reminiscence area and coordinated workload distribution. This structure permits extra environment friendly distributed coaching by lowering communication overhead between GPU nodes. For inference workloads, the flexibility to totally comprise trillion-parameter fashions inside a single NVLink area means quicker, extra constant response occasions at scale. When mixed with optimization methods akin to disaggregated serving with NVIDIA Dynamo, the massive area dimension of GB200 NVL72 structure unlocks vital inference efficiencies for varied mannequin architectures akin to combination of consultants fashions. GB200 NVL72 is especially highly effective when it is advisable deal with extra-large context home windows or run high-concurrency functions in actual time.
- P6-B200 situations assist a broad vary of AI workloads and are a perfect choice for medium to large-scale coaching and inference workloads. If you wish to port your current GPU workloads, P6-B200 situations supply a well-recognized 8-GPU configuration that minimizes code modifications and simplifies migration from present era situations. Moreover, though NVIDIA’s AI software program stack is optimized for each Arm and x86, in case your workloads are particularly constructed for x86 environments, P6-B200 situations, with their Intel Xeon processors, will likely be your best selection.
Innovation constructed on AWS core strengths
Bringing NVIDIA Blackwell to AWS isn’t a couple of single breakthrough—it’s about steady innovation throughout a number of layers of infrastructure. By constructing on years of studying and innovation throughout compute, networking, operations, and managed providers, we’ve introduced NVIDIA Blackwell’s full capabilities with the reliability and efficiency clients count on from AWS.
Strong occasion safety and stability
When clients inform me why they select to run their GPU workloads on AWS, one essential level comes up constantly: they extremely worth our concentrate on occasion safety and stability within the cloud. The specialised {hardware}, software program, and firmware of the AWS Nitro System are designed to implement restrictions in order that no person, together with anybody in AWS, can entry your delicate AI workloads and knowledge. Past safety, the Nitro System essentially modifications how we keep and optimize infrastructure. The Nitro System, which handles networking, storage, and different I/O capabilities, makes it potential to deploy firmware updates, bug fixes, and optimizations whereas it stays operational. This capability to replace with out system downtime, which we name dwell replace, is essential in at the moment’s AI panorama, the place any interruption considerably impacts manufacturing timelines. P6e-GB200 and P6-B200 each function the sixth era of the Nitro System, however these safety and stability advantages aren’t new—our progressive Nitro structure has been defending and optimizing Amazon Elastic Compute Cloud (Amazon EC2) workloads since 2017.
Dependable efficiency at huge scale
In AI infrastructure, the problem isn’t simply reaching huge scale—it’s delivering constant efficiency and reliability at that scale. We’ve deployed P6e-GB200 UltraServers in third-generation EC2 UltraClusters, which creates a single cloth that may embody our largest knowledge facilities. Third-generation UltraClusters lower energy consumption by as much as 40% and scale back cabling necessities by greater than 80%—not solely bettering effectivity, but additionally considerably lowering potential factors of failure.
To ship constant efficiency at this huge scale, we use Elastic Cloth Adapter (EFA) with its Scalable Dependable Datagram protocol, which intelligently routes visitors throughout a number of community paths to keep up easy operation even throughout congestion or failures. We’ve repeatedly improved EFA’s efficiency throughout 4 generations. P6e-GB200 and P6-B200 situations with EFAv4 present as much as 18% quicker collective communications in distributed coaching in comparison with P5en situations that use EFAv3.
Infrastructure effectivity
Whereas P6-B200 situations use our confirmed air-cooling infrastructure, P6e-GB200 UltraServers use liquid cooling, which permits greater compute density in giant NVLink area architectures, delivering greater system efficiency. P6e-GB200 are liquid cooled with novel mechanical cooling options offering configurable liquid-to-chip cooling in each new and current knowledge facilities, so we will assist each liquid-cooled accelerators and air-cooled community and storage infrastructure in the identical facility. With this versatile cooling design, we will ship most efficiency and effectivity on the lowest value.
Getting began with NVIDIA Blackwell on AWS
We’ve made it easy to get began with P6e-GB200 UltraServers and P6-B200 situations by a number of deployment paths, so you possibly can shortly start utilizing Blackwell GPUs whereas sustaining the operational mannequin that works finest to your group.
Amazon SageMaker HyperPod
When you’re accelerating your AI growth and wish to spend much less time managing infrastructure and cluster operations, that’s precisely the place Amazon SageMaker HyperPod excels. It gives managed, resilient infrastructure that routinely handles provisioning and administration of enormous GPU clusters. We preserve enhancing SageMaker HyperPod, including improvements like versatile coaching plans that will help you achieve predictable coaching timelines and run coaching workloads inside your finances necessities.
SageMaker HyperPod will assist each P6e-GB200 UltraServers and P6-B200 situations, with optimizations to maximise efficiency by holding workloads throughout the similar NVLink area. We’re additionally constructing in a complete, multi-layered restoration system: SageMaker HyperPod will routinely exchange defective situations with preconfigured spares in the identical NVLink area. Constructed-in dashboards will provide you with visibility into all the things from GPU utilization and reminiscence utilization to workload metrics and UltraServer well being standing.
Amazon EKS
For big-scale AI workloads, in case you choose to handle your infrastructure utilizing Kubernetes, Amazon Elastic Kubernetes Service (Amazon EKS) is commonly the management airplane of selection. We proceed to drive improvements in Amazon EKS with capabilities like Amazon EKS Hybrid Nodes, which allow you to handle each on-premises and EC2 GPUs in a single cluster—delivering flexibility for AI workloads.
Amazon EKS will assist each P6e-GB200 UltraServers and P6-B200 situations with automated provisioning and lifecycle administration by managed node teams. For P6e-GB200 UltraServers, we’re constructing in topology consciousness that understands the GB200 NVL72 structure, routinely labeling nodes with their UltraServer ID and community topology data to allow optimum workload placement. It is possible for you to to span node teams throughout a number of UltraServers or dedicate them to particular person UltraServers, providing you with flexibility in organizing your coaching infrastructure. Amazon EKS screens GPU and accelerator errors and relays them to the Kubernetes management airplane for non-compulsory remediation.
NVIDIA DGX Cloud on AWS
P6e-GB200 UltraServers will even be obtainable by NVIDIA DGX Cloud. DGX Cloud is a unified AI platform optimized at each layer with multi-node AI coaching and inference capabilities and NVIDIA’s full AI software program stack. You profit from NVIDIA’s newest optimizations, benchmarking recipes, and technical experience to enhance effectivity and efficiency. It affords versatile time period lengths together with complete NVIDIA knowledgeable assist and providers that will help you speed up your AI initiatives.
This launch announcement is a crucial milestone, and it’s only the start. As AI capabilities evolve quickly, you want infrastructure constructed not only for at the moment’s calls for however for all the probabilities that lie forward. With improvements throughout compute, networking, operations, and managed providers, P6e-GB200 UltraServers and P6-B200 situations are able to allow these prospects. We will’t wait to see what you’ll construct with them.
Sources
In regards to the writer
David Brown is the Vice President of AWS Compute and Machine Studying (ML) Providers. On this position he’s liable for constructing all AWS Compute and ML providers, together with Amazon EC2, Amazon Container Providers, AWS Lambda, Amazon Bedrock and Amazon SageMaker. These providers are utilized by all AWS clients but additionally underpin most of AWS’s inner Amazon functions. He additionally leads newer options, akin to AWS Outposts, that deliver AWS providers into clients’ non-public knowledge facilities.
David joined AWS in 2007 as a Software program Improvement Engineer based mostly in Cape City, South Africa, the place he labored on the early growth of Amazon EC2. In 2012, he relocated to Seattle and continued to work within the broader Amazon EC2 group. Over the past 11 years, he has taken on bigger management roles as extra of the AWS compute and ML merchandise have develop into a part of his group.
Previous to becoming a member of Amazon, David labored as a Software program Developer at a monetary trade startup. He holds a Laptop Science & Economics diploma from the Nelson Mandela College in Port Elizabeth, South Africa.
