Understanding IOMMU Groups and Domains in Linux

The Input/Output Memory Management Unit (IOMMU) is a hardware component that enables memory protection and address translation for Direct Memory Access (DMA) operations initiated by peripheral devices. By enforcing isolation between devices and system memory, the IOMMU enhances both security and performance.

In Linux, IOMMU groups represent the minimal unit of device isolation from the IOMMU’s perspective. Devices within the same IOMMU group share a common DMA address space and are subject to identical IOMMU policies. Consequently, they cannot access memory assigned to other groups, ensuring strong isolation boundaries.

Each IOMMU group maps to a single IOMMU domain, which maintains its own page table (PGD) for IO virtual address (IOVA) to physical address translation. Multiple groups may belong to the same domain, but all devices in a given group must reside in the same domain. This design ensures that peer devices within a group can access each other's mapped memory regions, while remaining isolated from external groups.

Determining Group Membership

Group assignment primarily depends on two factors:

  1. Device Topology: On PCIe systems, functions sharing the same bus and device number (i.e., different functions of a multifunction device) typically belong to the same group.
  2. ACS Capability: The PCIe Access Control Services (ACS) feature controls whether peer-to-peer (P2P) transactions are allowed between devices. If ACS is fully enabled along the path from a device to the Root Complex, the device may be placed in its own group. Otherwise, all devices under a non-ACS-compliant switch or briddge are grouped to gether to prevent unsafe P2P traffic that bypasses IOMMU translation.

For example, conventional PCI devices behind a PCIe-to-PCI bridge often share a single group because they all appear with the same Requester ID, making them indistinguishable to the IOMMU.

Inspecting IOMMU Groups

The following script lists all IOMMU groups and their associated devices:

for grp in /sys/kernel/iommu_groups/*; do
    gid=$(basename "$grp")
    echo "IOMMU Group $gid"
    for dev in "$grp"/devices/*; do
        dev_id=$(basename "$dev")
        echo -e "\t$(lspci -nns "$dev_id")"
    done
done

This output reflects how the kernel organizes devices based on hardware topology and ACS support.

Domain and Mapping Initialization

When a device is assigned to a group, the kernel calls iommu_group_create_direct_mappings() to establish identity mappings (or custom mappings, depending on use case) between IOVAs and physical pages. Each mapping is registered via iommu_map(), ensuring that DMA requests from the device are correctly translated through the domain’s page table.

In configurations where the Intel IOMMU is disabled, the DMA address equals the physical address, effectively bypassing translation. However, when enabled, every DMA operasion undergoes IOMMU validation and remapping.

Role of ACS in Isolation

ACS prevents unauthorized P2P communication by disabling transaction redirection at PCIe switches and multifunction endpoints. Without ACS, a DMA write intended for a guest VM’s buffer might inadvertently target a host-owned device if the switch forwards the transaction before it reaches the IOMMU. Thus, ACS enforcement is critical for maintaining isolation in virtualized environments.

The kernel uses pci_acs_path_enabled() to check ACS support along the entire upstream path. If ACS is not universally supported, devices are conservatively grouped to mitigate P2P risks.

Ultimately, an IOMMU group defines the smallest collection of devices that can be safely managed as a single unit for DMA isolation—essential for technologies like VFIO that rely on strict device ownership semantics.

Tags: Linux IOMMU kernel PCIe virtualization

Posted on Fri, 29 May 2026 23:30:59 +0000 by jefrat72