---
myst:
    html_meta:
        "description": "AMD Instinct MI300A acceptance criteria — prerequisites, health checks, system validation, and performance benchmarks for CDNA 3 APU platforms."
        "keywords": "MI300A, AMD Instinct, APU, CDNA 3, ROCm, system acceptance, validation, benchmarks, HBM3"
---

# AMD Instinct MI300A

The AMD Instinct™ MI300A is a data-center Accelerated Processing Unit (APU) that integrates AMD "Zen 4" CPU cores and CDNA 3 GPU compute dies on a single package with unified HBM3 memory. This document provides MI300A-specific prerequisites, health checks, validation steps, and performance acceptance criteria.

## Overview

The MI300A is built on the CDNA 3 architecture (gfx942) and combines CPU and GPU compute dies sharing a single coherent pool of 128 GB HBM3 per APU. Unlike discrete OAM accelerators, MI300A platforms are vendor-defined; a typical qualified configuration hosts 4 MI300A APUs per node.

- **[MI300A Product Page](https://www.amd.com/en/products/accelerators/instinct/mi300/mi300a.html)**
- **[MI300A Data Sheet](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/data-sheets/amd-instinct-mi300a-data-sheet.pdf)**
- **[AMD Instinct MI300 Series microarchitecture](https://instinct.docs.amd.com/latest/gpu-arch/mi300.html)**

## System requirements

### Operating system support

For the most up-to-date information on supported operating systems and distributions, refer to the official ROCm documentation:

[ROCm System Requirements - Supported Distributions](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html#supported-distributions)

```{note}
[ROCm docs](https://rocm.docs.amd.com) is the single source of truth for supported versions, distribution compatibility, and required dependencies for the ROCm toolkit.
```

For BIOS, IOMMU, transparent hugepages, NUMA, and OS-level tuning that applies to all AMD Instinct hosts, see [BIOS settings](../common/bios-settings.md), [OS tuning](../common/os-tuning.md), and [Kernel parameters](../common/kernel-parameters.md). MI300A requires a Linux kernel that supports "Zen 4" (≥ 5.18 recommended).

### GPU identification

All MI300A APUs (PCI vendor:device `1002:74a0`) should appear in `lspci` output:

```bash
sudo lspci -d 1002:74a0
```

Expected output example:

```bash
0000:01:00.0 Processing accelerators: Advanced Micro Devices, Inc. [AMD/ATI] Aqua Vanjaram [Instinct MI300A]
0001:01:00.0 Processing accelerators: Advanced Micro Devices, Inc. [AMD/ATI] Aqua Vanjaram [Instinct MI300A]
0002:01:00.0 Processing accelerators: Advanced Micro Devices, Inc. [AMD/ATI] Aqua Vanjaram [Instinct MI300A]
0003:01:00.0 Processing accelerators: Advanced Micro Devices, Inc. [AMD/ATI] Aqua Vanjaram [Instinct MI300A]
```

## APU-specific OS tuning

Because the MI300A integrates CPU and GPU on a single package with shared HBM3, several OS-level settings that are optional on discrete OAM systems are required or strongly recommended here.

### Transparent huge pages

The `transparent_hugepage=always` kernel parameter is required on MI300A. The unified CPU+GPU shared-memory model depends on transparent huge pages for correct and performant operation. Add it to `GRUB_CMDLINE_LINUX` and update GRUB as described in [Kernel Parameters](../common/kernel-parameters.md).

To verify the setting is active after boot:

```bash
cat /sys/kernel/mm/transparent_hugepage/enabled
```

The output should show `[always]` as the active selection.

### Increase allocatable GPU memory (`amdttm`)

By default, only 96 GiB of the 128 GiB HBM per APU is allocatable via HIP device allocations. To make the full 128 GiB available, load the `amdttm` module with updated page limits. The values correspond to the number of 4 KiB pages.

Check the current limit:

```bash
cat /sys/module/amdttm/parameters/pages_limit
```

To set the full 128 GiB across all four APUs on a single-node system (total 512 GiB), run:

```bash
sudo modprobe amdttm pages_limit=134217728 page_pool_size=134217728
sudo modprobe amdgpu
```

```{note}
For a two-socket node with two APUs, halve the values: `pages_limit=67108864 page_pool_size=67108864`.
The per-APU limit is `33554432` pages × number of APUs on the node.
```

To make the setting persistent, add it to `/etc/modprobe.d/amdttm.conf`:

```text
options amdttm pages_limit=134217728
options amdttm page_pool_size=134217728
```

If the filesystem is not yet mounted when the kernel driver loads (for example, on early-boot initrd), pass the values as boot parameters instead:

```text
amdttm.pages_limit=134217728 amdttm.page_pool_size=134217728
```

Verify the new setting:

```bash
cat /sys/module/amdttm/parameters/pages_limit
```

### Limit GPU memory allocation percentage

Many AI applications were originally developed for discrete GPUs and probe available memory until exhaustion. On the MI300A APU, where CPU and GPU share the same HBM pool, this can starve the OS of memory.

Use the following environment variables to bound how much of the shared HBM the GPU side can claim:

```bash
export GPU_MAX_ALLOC_PERCENT=<value>
export GPU_SINGLE_ALLOC_PERCENT=<value>
```

Setting either variable to `100` allows the GPU to allocate all free memory. Before doing so, account for expected CPU workload and OS usage. For example, on a 128 GiB system where the OS requires 8 GiB, setting `GPU_MAX_ALLOC_PERCENT=100` authorizes a single workload to allocate up to 120 GiB. Any over-allocation is handled by the kernel's out-of-memory (OOM) policies unless swap space is configured.

Add these variables to `/etc/environment` or the user shell environment.

### Enable memory compaction

The MI300A APU dynamically shares memory between the CPU and GPU. Without periodic memory compaction, fragmentation accumulates and application performance degrades over time. Compaction is enabled by default on Ubuntu but disabled by default on RHEL-based distributions.

Enable proactive compaction:

```bash
echo 20 > /proc/sys/vm/compaction_proactiveness
echo 1 > /proc/sys/vm/compact_unevictable_allowed
```

## Acceptance criteria

The MI300A system acceptance process validates that the platform is correctly configured, stable, and performing to expectations. Follow the sequence: Prerequisites → Basic Health Checks → System Validation → Performance Benchmarks.

### System acceptance process

1. **[Prerequisites validation](#prerequisites-validation)** - Ensure all system requirements and dependencies are met
2. **[Basic health checks](#basic-health-checks)** - Verify hardware detection and basic system health
3. **[System validation](#system-validation)** - Conduct comprehensive stress testing and qualification
4. **[Performance benchmarks](#performance-benchmarks)** - Validate compute, memory, and interconnect performance

The system is accepted when all criteria below are successfully validated.

### Prerequisites validation

Ensure all system requirements are met before proceeding with validation. See the [Prerequisites documentation](../common/prerequisites.md) and [System setup](../common/system-setup.md) for more details.

- ✅ Supported operating system version installed with kernel ≥ 5.18 (Zen 4 support)
- ✅ Compatible ROCm version installed (verify: `cat /opt/rocm/.info/version`); see the [ROCm System Requirements](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html) for the current supported version matrix
- ✅ BIOS configured per [BIOS settings](../common/bios-settings.md), with MI300A-specific values per platform vendor (IOMMU off, memory interleaving, NPS)
- ✅ Required kernel parameters present: `pci=realloc=off transparent_hugepage=always numa_balancing=disable`
- ✅ Sysctl tunings applied: `vm.compaction_proactiveness=20` and `vm.compact_unevictable_allowed=1` (required on RHEL-based systems; enabled by default on Ubuntu) — see [Enable memory compaction](#enable-memory-compaction)
- ✅ `amdttm` page pool configured for full 128 GiB HBM access per APU — see [Increase allocatable GPU memory](#increase-allocatable-gpu-memory-amdttm)
- ✅ Environment variables (where applicable):
  - `HSA_OVERRIDE_CPU_AFFINITY_DEBUG=0`
  - `GPU_MAX_ALLOC_PERCENT` and `GPU_SINGLE_ALLOC_PERCENT` tuned per workload — see [Limit GPU memory allocation percentage](#limit-gpu-memory-allocation-percentage)
- ✅ Minimum 4 × 128 GB = 512 GB unified HBM3 visible to the OS usable host-visible memory (note: MI300A's HBM is unified with CPU)
- ✅ Latest applicable firmware applied consistently across nodes
- ✅ ROCm Validation Suite (RVS) installed

### Basic health checks

These checks ensure fundamental system health and proper APU detection. For detailed procedures, see [Health Checks](../common/health-checks.md).

| Test | Command | Pass/Fail criteria |
|------|---------|-------------------|
| [Check OS distribution](../common/health-checks.md#check-os-distribution) | `cat /etc/os-release` | **Pass**: OS version listed in compatibility matrix; kernel ≥ 5.18<br>**Fail**: Otherwise |
| [Check kernel boot arguments](../common/health-checks.md#check-kernel-boot-arguments) | `cat /proc/cmdline` | **Pass**: Contains `pci=realloc=off transparent_hugepage=always numa_balancing=disable`<br>**Fail**: Missing any required param |
| Verify IOMMU is off (MI300A-specific) | `sudo apt install acpica-tools && sudo acpidump \| grep IVRS && sudo acpidump \| grep DMAR` | **Pass**: Both commands return no output (IOMMU disabled)<br>**Fail**: Either command returns output |
| [Check for driver errors](../common/health-checks.md#check-for-driver-errors) | `sudo dmesg -T \| grep amdgpu \| grep -i error` | **Pass**: Null<br>**Fail**: Errors reported |
| [Check available memory](../common/health-checks.md#check-for-available-system-memory) | `lsmem \| grep "Total online memory"` | **Pass**: ≥ 4 × 128 GB = 512 GB unified HBM3 visible to the OS<br>**Fail**: Less than 4 × 128 GB = 512 GB unified HBM3 visible to the OS |
| [Check GPU presence](../common/health-checks.md#check-gpu-presence) | `sudo lspci -d 1002:74a0` | **Pass**: 4 MI300A APUs found<br>**Fail**: Otherwise |
| [Check GPU link speed and width](../common/health-checks.md#check-gpu-pcie-bus-link-speed-and-width) | `sudo lspci -d 1002:74a0 -vvv \| grep -e DevSta -e LnkSta` | **Pass**: Speed PCIe Gen 5 (32 GT/s), width `x16`, no `FatalErr+`<br>**Fail**: Otherwise |
| [Monitor utilization metrics](../common/health-checks.md#monitor-utilization-metrics) | `amd-smi monitor -putm` | **Pass**: Idle metrics as specified<br>**Fail**: Otherwise |
| [Check system kernel logs for errors](../common/health-checks.md#check-system-kernel-logs) | `sudo dmesg -T \| grep -i 'error\|warn\|fail\|exception'` | **Pass**: Null<br>**Fail**: Otherwise |

### System validation

Comprehensive validation ensures system stability under load. For detailed procedures, see [System Validation](../common/system-validation.md).

| Test | Command | Pass/Fail criteria |
|------|---------|-------------------|
| [Compute/GPU properties](../common/system-validation.md#gpu-properties) | `rvs -c ${RVS_CONF}/gpup_single.conf` | **Pass**: All APUs listed with no errors<br>**Fail**: Missing APUs or errors |
| [GPU stress test (GST)](../common/system-validation.md#gpu-stress-test) | `rvs -c ${RVS_CONF}/MI300A/gst_single.conf` | **Pass**: `met: TRUE` in logs<br>**Fail**: Target GFLOP/s not met |
| [Input energy delay product (IET)](../common/system-validation.md#input-energy-delay-product) | `rvs -c ${RVS_CONF}/MI300A/iet_single.conf` | **Pass**: `met: TRUE` for all actions<br>**Fail**: Otherwise |
| [Memory test (MEM)](../common/system-validation.md#mem) | `rvs -c ${RVS_CONF}/mem.conf -l mem.txt` | **Pass**: All tests passed; bandwidth ≥ 2.0 TB/s per APU<br>**Fail**: Any test failed or low bandwidth |
| [PCIe bandwidth benchmark (PEBB)](../common/system-validation.md#pcie-bandwidth-benchmark) | `rvs -c ${RVS_CONF}/MI300A/pebb_single.conf` | **Pass**: All distances and bandwidths displayed<br>**Fail**: Missing data |
| [PCIe qualification tool (PEQT)](../common/system-validation.md#pcie-qualification-tool) | `rvs -c ${RVS_CONF}/peqt_single.conf` | **Pass**: All actions true<br>**Fail**: Otherwise |
| [P2P benchmark and qualification tool (PBQT)](../common/system-validation.md#p2p-benchmark-and-qualification-tool) | `rvs -c ${RVS_CONF}/pbqt_single.conf` | **Pass**: `peers:true` lines and non-zero throughput across all xGMI peers<br>**Fail**: Otherwise |

### Performance benchmarks

Performance validation ensures the system meets MI300A specifications. For detailed procedures, see [Performance Benchmarking](../common/system-validation.md#performance-benchmarking).

:::{card} Command: `TransferBench a2a`
[TransferBench all-to-all](../common/system-validation.md#transferbench)
^^^
**Pass:** ≥ 700 GB/s aggregate
+++
**Fail:** otherwise
:::

:::{card} Command: `TransferBench p2p`
[TransferBench peer-to-peer](../common/system-validation.md#transferbench)
^^^

| Test | Pass criteria |
|------|--------------|
| UniDir | ≥ 80 GB/s |
| BiDir | ≥ 155 GB/s |

+++
**Fail:** otherwise
:::

:::{card} Command: `build/all_reduce_perf -b 8 -e 8G -f 2 -g 4`
[RCCL Allreduce](../common/system-validation.md#rccl-allreduce)
^^^
**Pass:** ≥ 230 GB/s busbw (peak, at 8 GiB message size)
+++
**Fail:** otherwise
:::

:::{card} Command: `rocblas-bench` (see code block below)
[rocBLAS FP32](../common/system-validation.md#rocblas-gemm-benchmarks)
^^^

```bash
rocblas-bench -f gemm \
  -r s -m 4000 -n 4000 -k 4000 \
  --lda 4000 --ldb 4000 --ldc 4000 \
  --transposeA N --transposeB T
```

**Pass:** ≥ 60 TFLOPS per APU
+++
**Fail:** otherwise
:::

:::{card} Command: `mpiexec -n 4 wrapper.sh`
[BabelStream](../common/system-validation.md#babelstream)
^^^

| Kernel | Threshold (MB/s) |
|--------|-----------------|
| Copy  | ≥ 2,900,000 |
| Mul   | ≥ 3,000,000 |
| Add   | ≥ 3,250,000 |
| Triad | ≥ 3,250,000 |
| Dot   | ≥ 2,200,000 |

+++
**Fail:** otherwise
:::