Running LLMs on AMD Ryzen AI NPU via Linux

AMD ships a neural processing unit (NPU) inside its Ryzen AI processors, and the company maintains an open-source Linux driver that makes the NPU usable for AI workloads. The project lives at https://github.com/amd/xdna-driver and centers on a kernel module called amdxdna.ko paired with a runtime shim library. Together they let a Linux machine that has an XDNA device run inference on the NPU rather than relying solely on the CPU or GPU.

What the XDNA driver provides

The repository describes two pieces that work together. The first is the kernel driver, amdxdna.ko. The second is a plugin built against AMD’s XRT (Xilinx Runtime). With both XRT and the plugin installed on a system that contains an XDNA device, applications can dispatch work to the NPU.

The driver documentation is explicit about scope. The NPU is built for inference, not training. The project states directly that the NPU can accelerate machine learning inference but is not designed for machine learning training. That distinction matters for anyone planning to run a large language model locally, because it frames the NPU as a target for running already-trained models rather than fine-tuning or building them.

The repository also notes that two driver trees exist. One is an upstream staging driver located under the kernel’s accelerator subsystem, and the other is an out-of-tree legacy driver maintained in the project itself. These produce amdxdna.ko and amdxdna_legacy.ko respectively, giving users a choice depending on their kernel and distribution.

Kernel and operating system requirements

The driver has firm baseline requirements. It needs Linux kernel version 6.10 or newer, and it depends on two kernel configuration options being enabled: CONFIG_AMD_IOMMU and CONFIG_DRM_ACCEL. Kernel 6.10 is significant because, according to the project, it is the version where AMD IOMMU shared virtual addressing became officially supported. The project also warns that the driver does not always continue supporting older kernels, since kernel API changes can break compatibility.

On the distribution side, the project lists Ubuntu 22.04 or later and Arch Linux as supported environments. It calls out specific kernel versions that ship with recent Ubuntu releases: Ubuntu 25.04 includes kernel 6.14 with the amdxdna module already built in, while Ubuntu 24.10 ships kernel 6.11. Ubuntu 24.04 may require a kernel update through the hardware enablement stack to reach a compatible version. Even on releases where the kernel module is bundled, the shim library from the repository is still required.

Build setup and validation

The project separates the machine that builds the package from the machine that runs it. The build machine can use any x86 processor, with AMD chips recommended, while the test machine that actually executes workloads needs a Ryzen AI processor with the NPU present.

After installation, the driver places firmware under a system firmware directory and runtime libraries under the XRT installation path. The documentation provides a basic validation step that sources the XRT setup script and then runs an xrt-smi validate command to confirm the NPU is detected and functioning.

The README does not catalog specific model frameworks or ready-made language model examples, so anyone targeting the NPU will be working close to the XRT runtime layer rather than through a polished high-level toolchain. The foundation for NPU-accelerated inference on Linux is in place, but it currently rewards users who are comfortable with kernel versions, build steps, and runtime configuration.