AMD has officially released the Instinct MI350P, its most powerful PCIe-based AI accelerator to date. This card is specifically designed for the air-cooled enterprise server market, where the complexity of liquid cooling remains a major adoption barrier for hyperscale chips like the H100 OAM.
Featuring a record 144GB of HBM3E memory, the MI350P delivers a massive bandwidth jump over the previous MI300X. Initial benchmarks show a 40% lead over Nvidia’s H200 NVL in LLM inference workloads, particularly for Mixture-of-Experts (MoE) models like Mistral Large and Claude 3.5. The card operates within a 450W TDP, allowing for standard 2U and 4U chassis deployments without specialized power modifications.
AMD also unveiled ROCm 7.0, which includes native support for MXFP4—the first 4-bit floating point format for training. This enables a 2x throughput increase for pre-training tasks without significant loss in accuracy.