Oracle unveils massive AI supercluster powered by thousands of AMD Instinct GPUs – another huge win for AMD!

In the fast-evolving world of Artificial Intelligence, AMD’s Instinct MI300X emerges as a powerhouse AI accelerator, catching the attention of major cloud service providers. With its ability to handle intensive AI workloads, this cutting-edge technology is now finding its way into the core infrastructure of leading cloud companies, paving the way for exciting advancements in AI capabilities.

1. Vultr’s Bold Move: Recently, Vultr made headlines by announcing their order of "thousands" of MI300X units, showcasing their confidence in this revolutionary technology.
2. Oracle Cloud Infrastructure’s Leap: Following in Vultr’s footsteps, Oracle Cloud Infrastructure (OCI) has embraced AMD’s hardware by incorporating it into their new OCI Compute Supercluster instance, BM.GPU.MI300X.8. This supercluster is tailor-made to handle massive AI models with billions of parameters, supporting a staggering 16,384 GPUs in a single cluster.
3. Unleashing AI Potential: The incorporation of MI300X in OCI’s infrastructure unlocks the potential for large-scale AI training and inference, providing the necessary memory capacity and throughput for the most demanding AI tasks. This setup is particularly well-suited for Large Language Models (LLMs) and intricate deep learning operations, pushing the boundaries of AI capabilities.

Oracle’s vigorous preproduction testing efforts have affirmed the MI300X’s performance in real-world scenarios. The GPU showcased impressive results with a 65-millisecond "time to first token" latency for the Llama 2 70B model, efficiently handling 3,643 tokens across 256 concurrent user requests. Another test involving 2,048 input and 128 output tokens demonstrated an end-to-end latency of 1.6 seconds, aligning closely with AMD’s benchmarks.

The OCI BM.GPU.MI300X.8 instance boasts 8 AMD Instinct MI300X accelerators, offering 1.5TB of HBM3 GPU memory with a bandwidth of 5.3TB/s, complemented by 2TB of system memory and 8 x 3.84TB NVMe storage. Oracle plans to make this bare-metal solution available for $6 per GPU/hour, providing customers with a cost-effective option to accelerate their AI workloads.

As Oracle Cloud Infrastructure expands its range of high-performance bare metal instances, powered by the formidable AMD Instinct MI300X accelerators, customers can benefit from enhanced inference capabilities, eliminating the overhead associated with virtualized compute commonly used for AI infrastructure. This move reflects Oracle’s commitment to offering diverse options for accelerating AI workloads at a competitive price point.

The integration of AMD’s Instinct MI300X technology in cloud infrastructure marks a significant step towards revolutionizing AI capabilities, empowering businesses to explore new horizons of innovation and efficiency.As the era of AI continues to unfold, embracing cutting-edge technologies like the Instinct MI300X becomes imperative for companies seeking to stay ahead in the rapidly evolving landscape of Artificial Intelligence.