Introduction to MLPerf Inference v4.1 Benchmark
The MLCommons consortium on Wednesday unveiled the results of the MLPerf Inference v4.1 benchmark. This benchmark provides impartial performance assessments of popular AI inferencing accelerators available in the market. Among the brands evaluated were Nvidia, AMD, and Intel.
AMD’s Instinct MI300X: A Competitive Edge
The results highlighted AMD’s Instinct MI300X accelerators as competitive solutions to Nvidia’s “Hopper” H100 series AI GPUs. Utilizing the opportunity, AMD showcased the AI inferencing performance uplifts possible with its next-generation EPYC “Turin” server processors, which power these MI300X machines. The “Turin” server processors feature “Zen 5” CPU cores, a 512-bit FPU datapath, and enhanced performance in AI-relevant 512-bit SIMD instruction-sets such as AVX-512 and VNNI.
Key Features of AMD’s AI Solution
The Instinct MI300X stands out due to its strong memory subsystem, FP8 data format support, and efficient KV cache management. These features enable it to deliver impressive AI inferencing capabilities.
Benchmarking with Llama2-70B Model
The MLPerf Inference v4.1 benchmark focused on the 70 billion-parameter Llama2-70B model. AMD’s submissions included machines featuring the Instinct MI300X, powered by current EPYC “Genoa” (Zen 4), and next-gen EPYC “Turin” (Zen 5) processors. These GPUs are supported by AMD’s ROCm open-source software stack.
The benchmark evaluated inference performance using 24,576 Q.
