AMD Instinct MI300

Alan Smith, Eric Chapman, Chintan Patel, Raja Swaminathan, J. Wuu, Tyrone Huang, W. Jung, Alexander Kaganov, Hugh McIntyre, Ramon Mangaser

The AMD Instinct™ MI 300 Series accelerators were conceptualized to extract maximum HPC and AI capability from the latest silicon and advanced packaging technology, designed to operate as CPU hosted PCle® device, MI300X, as well as a self-hosted accelerated processing unit (APU), MI300A. AMD chiplet capabilities and advanced packaging allow AMD’s first-ever integration of data center class CPU, GPU accelerated compute, AMD Infinity Cache, and 8-stack HBM3 memory system into a single package. Observing that many Al and HPC operators are memory bound, AMD targeted MI300 to deliver over 5TBps of HBM3 peak bandwidth.

S. Tandon, Leopold Grinberg, Gheorghe-Teodor Bercea, Carlo Bertolli, M.H. Olesen, Simone Bna, Nicholas Malaya

AMD Instinct$^\text{TM}$ MI300A is the world's first data center accelerated processing unit (APU) with memory shared between the AMD"Zen 4"EPYC$^\text{TM}$ cores and third generation CDNA$^\text{TM}$ compute units. A single memory space offers several advantages: i) it eliminates the need for data replication and costly data transfers, ii) it substantially simplifies application development and allows an incremental acceleration of applications, iii) is easy to maintain, and iv) its potential can be well realized via the abstractions in the OpenMP 5.2 standard, where the host and the device data environments can be unified in a more performant way. In this article, we provide a blueprint of the APU programming model leveraging unified memory and highlight key distinctions compared to the conventional approach with discrete GPUs. OpenFOAM, an open-source C++ library for computational fluid dynamics, is presented as a case study to emphasize the flexibility and ease of offloading a full-scale production-ready application on MI300 APUs using directive-based OpenMP programming.

Alan Smith, Gabriel H. Loh, S. Naffziger, J. Wuu, Nathan Kalyanasundharam, Eric Chapman, Raja Swaminathan, Tyrone Huang, W. Jung, Alexander Kaganov, Hugh McIntyre, Ramon Mangaser

The semiconductor industry has deployed chiplet-based system-on-chip architectures for several years. Central to a successful chiplet-based product is the die-to-die interconnect technology between the chiplets. Based on product requirements, some chiplet designs can utilize a single interconnect technology such as 2-D signals over an organic substrate or higher-density 2.5-D integration technologies. With increasing demands on compute and memory capabilities, high-performance products are now moving to heterogeneous integration, which combines multiple advanced packaging technologies all within a single system on chip. To address the market demands for high-performance artificial intelligence solutions, AMD has introduced the AMD Instinct MI300X accelerator. This article details the chiplet interconnect design required to support a sophisticated package that takes high-volume heterogeneous integration to a new level.

Chintan Buch

Exponential growth in the number of parameters used to train machine learning (ML) models for artificial intelligence (AI) training & inference applications consequently requires extensive compute resources like CPUs, GPUs, and memory, all working in tandem at high bandwidth. Heterogeneous integration via chiplet architectures is also key to enabling economically feasible growth of power efficient computing, given the slowdown in Moore’s law. In this paper, we summarized the key advanced packaging technologies that directly enabled the heterogenous integration of multiple chiplets like CPUs, GPUs, active interposers, high bandwidth memory (HBMs) die and passive components in the largest, most complex, and high power (550 W+) AMD Instinct™ MI300 accelerator package built by AMD. Three key technologies are described: direct Cu-Cu hybrid bonding, 2.5D integration on large silicon interposer, and metal TIM-based cooling solution used in MI300. Package level reliability results are also presented.

Jyun-Ping Kao

Large-language models (LLMs) are rapidly being applied to radiology, enabling automated image interpretation and report generation tasks. Their deployment in clinical practice requires both high diagnostic accuracy and low inference latency, which in turn demands powerful hardware. High-performance graphical processing units (GPUs) provide the necessary compute and memory throughput to run large LLMs on imaging data. We review modern GPU architectures (e.g. NVIDIA A100/H100, AMD Instinct MI250X/MI300) and key performance metrics of floating-point throughput, memory bandwidth, VRAM capacity. We show how these hardware capabilities affect radiology tasks: for example, generating reports or detecting findings on CheXpert and MIMIC-CXR images is computationally intensive and benefits from GPU parallelism and tensor-core acceleration. Empirical studies indicate that using appropriate GPU resources can reduce inference time and improve throughput. We discuss practical challenges including privacy, deployment, cost, power and optimization strategies: mixed-precision, quantization, compression, and multi-GPU scaling. Finally, we anticipate that next-generation features (8-bit tensor cores, enhanced interconnect) will further enable on-premise and federated radiology AI. Advancing GPU infrastructure is essential for safe, efficient LLM-based radiology diagnostics.