[PW_SID:1142375] iommu/riscv: Enable MSI remapping, IOMMU_DMA and VFIO - #2433
[PW_SID:1142375] iommu/riscv: Enable MSI remapping, IOMMU_DMA and VFIO#2433linux-riscv-bot wants to merge 20 commits into
Conversation
struct iommufd_sw_msi_maps currently uses a fixed 64-bit bitmap, capping the number of distinct SW MSI mappings a context or hwpt can track. An upcoming caller needs one mapping per possible CPU. Convert the fixed bitmap to a pointer plus size that grows on demand via iommufd_sw_msi_maps_ensure(). Add iommufd_sw_msi_maps_test_bit() alongside it: unlike the __set_bit() call sites, which always follow their own successful iommufd_sw_msi_maps_ensure() call on the same id, iommufd_group_setup_msi() tests an id from the fd-global sw_msi_list against a specific group's required_sw_msi map, which may not have been grown to cover that id yet. The bitmap is arbitrarily capped at 16K entries in order to simplify arithmetic validation (i.e. simple bounds checks vs. integer wrap control). Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Add a descriptor-free counterpart to iommufd_sw_msi(). The existing function is tied to a struct msi_desc and stores the result in the descriptor. This variant returns the IOVA directly so callers can pre-map MSI targets before any descriptor has been allocated. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Add a descriptor-free counterpart to iommu_dma_sw_msi(). The existing function is tied to a struct msi_desc and stores the result in the descriptor. This variant returns the IOVA directly so callers can pre-map MSI targets before any descriptor has been allocated. Callers may pass a required mapping size so MSI doorbells that must not share a larger IOMMU leaf, such as RISC-V IMSIC files, can fail before a mapping is installed. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Add iommu_dma_map_msi() to map an MSI doorbell and return the IOVA and granule shift to callers that need their own PA->IOVA lookup, dispatching to the DMA-IOMMU or iommufd cookie implementation as appropriate. Callers may pass a required mapping size so MSI doorbells that must not share a larger IOMMU leaf, such as RISC-V IMSIC files, can fail before a mapping is installed. iommu_dma_map_msi() requires the caller to hold @dev's iommu group mutex, but struct iommu_group is private to drivers/iommu/iommu.c, so a caller outside the core (e.g. an interrupt-remapping driver building a table of mappings ahead of any MSI descriptor existing) has no way to take it. Add iommu_group_mutex_lock()/iommu_group_mutex_unlock() to bridge that gap, next to the existing iommu_group_mutex_assert(). Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Add a bus token for MSI domains that remap interrupts, needed by the upcoming RISC-V IOMMU interrupt-remapping domain to distinguish itself from NEXUS domains. The token is generic because remapping itself is the only property that needs to be conveyed -- there is nothing RISC-V-specific about it. Such a domain implements init_dev_msi_info() via msi_parent_init_dev_msi_info(), which leaves 'domain' pointing at the NEXUS domain while 'real_parent' points at the remap domain itself. Accept that combination in msi_lib_init_dev_msi_info(); no other msi-lib changes are needed. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Reviewed-by: Nutty Liu <nutty.liu@hotmail.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
imsic_irq_set_affinity() wrote the new IMSIC target straight to the device via imsic_msi_update_msg(), bypassing the irqdomain hierarchy. That skips any intermediate IOMMU irqdomain sitting between IMSIC and the device, so a remapped device would keep the pre-remap physical address after every affinity change. Compose MSI updates via irq_chip_compose_msi_msg() starting from the top of the hierarchy instead, so an intermediate remap domain gets to translate the target before the device is written. For the temporary vector used by non-atomic affinity updates, compose the current vector through the hierarchy before d->chip_data is switched to the new vector, then replace only msg.data with the new local ID. This preserves the required old-address/new-data temporary target while also allowing an intermediate remap domain to refresh or clear any MSI IOVA state cached on the device's MSI descriptor. Moving the final write to after d->chip_data = new_vec is required because irq_chip_compose_msi_msg() reads d->chip_data. Keep it before updating effective affinity and migrating vector state so the device is retargeted before the old vector is disabled. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Create a per-device MSI parent irqdomain as the hierarchy hook for future interrupt remapping. The remapping tables will be owned by the attached paging domain since the MSI IOVA mappings live in its page tables. The domain is installed from probe_device() and removed from release_device(). This is only the initial skeleton: it does not remap interrupts yet, and non-paging IOMMU domains will fall back to the raw IMSIC physical address when remapping is added. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
An upcoming compose_msi_msg() implementation reads info->domain from potentially atomic context, concurrent with domain switches. Apply RCU in preparation. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
iommufd's SW MSI path (iommufd_sw_map_msi) requires an IOMMU_RESV_SW_MSI reservation region to know the IOVA base (sw_msi_start) for MSI page mappings. Without it, sw_msi_start stays at PHYS_ADDR_MAX and iommufd_sw_map_msi silently returns passthrough. Add a get_resv_regions callback that, when IMSIC-based MSI is in use, reserves an IOVA range of num_possible_cpus() pages starting at RISCV_IOMMU_MSI_IOVA_BASE. This is large enough to hold one 4 KiB IMSIC supervisor page per hart. RISCV_IOMMU_MSI_IOVA_BASE is set to 128 MiB, matching the ARM SMMU convention. That value is arbitrary for ARM too; its introducing commit (f3ebee8) calls it an "arbitrary MSI IOVA window", so there is nothing riscv-specific lost by reusing it here. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Prepare for irq_compose_msi_msg() to redirect IMSIC-targeted MSIs through the IOMMU with an O(1) lookup. Unlike fixed-target MSI controllers, IMSIC target PAs change on irq_set_affinity(), which may run in atomic context, so compose cannot map the selected target on demand. Pre-map each populated IMSIC MSI target page into a domain-local IOVA table when the device's paging domain first allocates IMSIC-remapped IRQs, so every possible target IOVA is available at lookup time. The table is indexed by a transformed IMSIC physical address using the same transformation defined in the IOMMU specification for the MSI table (and the function will be used for the MSI table when that support is added later). The table is sized by riscv_iommu_ir_msi_iova_count(), i.e. BIT(group_index_bits + hart_index_bits), rather than num_possible_cpus(). This is because riscv_iommu_ir_msi_iova_idx() derives its index from hart_index_bits and group_index_bits, which are sized for the group with the most harts, i.e. there may be a sparse mapping. Select IRQ_MSI_IOMMU so iommu_dma_map_msi() and iommufd_sw_map_msi() are available to the driver. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
iommufd may replace one paging domain with another on a device while leaving the device otherwise operational, e.g. when converting a device's HWPT. If the old domain already had a populated MSI IOVA table then the new domain must inherit that table rather than wait to build its own, since the MSI IOVAs are only built on the next irq_domain_alloc_irqs(), which does not happen on a domain replacement. Only copy from an old domain that is a genuine RISC-V paging domain; iommu.c already has riscv_iommu_paging_domain_ops in scope to check this, so do the check there and pass NULL down on a mismatch. This keeps riscv_iommu_ir_attach_paging_domain() simple: a non-NULL old is always safe to cast with iommu_domain_to_riscv(). Only iommufd cookie domains are copied from and to, since VFIO type1 and DMA API domains build their own tables from irq_domain_alloc_irqs() before any device is attached, and are never replaced while a device is live. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Reject identity-boundary domain transitions while a device still has allocated MSIs. The RISC-V IOMMU MSI compose path can run concurrently with domain attach and derives the MSI address rewrite policy from info->domain. Across an identity boundary, a stale decision can turn into a wrong addressing mode (IOVA vs PA), so require nr_msis == 0 when crossing between paging and identity domains. This is just a defensive guard since MSI state should be quiesced before crossing the identity boundary anyway. Do not gate paging <-> blocking: blocking is the fail-stop mode and must become available immediately, even with live MSIs. Do not gate paging -> paging: that path already uses nr_msis to ensure new-domain MSI table readiness before install. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Wire up irq_compose_msi_msg() on the IOMMU-IR irq chip to look up the pre-mapped IOVA for the target IMSIC page and rewrite the composed message to use it, redirecting MSI writes through the IOMMU instead of landing at the raw IMSIC physical address. The lookup is O(1), since the "extract" function used to build the table's index is also used here, which is necessary since the number of IMSICs may eventually be in the hundreds and irq_compose_msi_msg() may run in atomic context. domain->msi_iova is read here without domain->mutex, since compose can run in atomic context, and RCU only protects info->domain itself. Pair smp_store_release()/smp_load_acquire() on domain->msi_iova to ensure it's observably filled before publishing its pointer. That guarantees a non-NULL domain->msi_iova observed here is safe to use. A NULL domain->msi_iova falls back to the raw IMSIC physical address, same as when info->domain itself is NULL, since a live paging domain should not normally reach compose without a populated table. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Enable IOMMU_DMA for 64-bit RISC-V now that the RISC-V IOMMU driver supports MSI remapping. Signed-off-by: Tomasz Jeznach <tjeznach@rivosinc.com> Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Report RISC-V IOMMU capability required by the VFIO subsystem to enable PCIe device assignment. Signed-off-by: Tomasz Jeznach <tjeznach@rivosinc.com> Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Reviewed-by: Nutty Liu <nutty.liu@hotmail.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Enable VFIO support on RISC-V architecture, now that the RISC-V IOMMU driver reports the IOMMU_CAP_CACHE_COHERENCY capability VFIO_TYPE1 and iommufd both require before allowing a device to be bound. Signed-off-by: Tomasz Jeznach <tjeznach@rivosinc.com> Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Reviewed-by: Nutty Liu <nutty.liu@hotmail.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Enable KVM/VFIO support on RISC-V architecture, now that VFIO device assignment is available on RISC-V through VFIO_IOMMU_TYPE1, so a RISC-V KVM guest can be notified about VFIO-assigned devices. Signed-off-by: Tomasz Jeznach <tjeznach@rivosinc.com> Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Reviewed-by: Nutty Liu <nutty.liu@hotmail.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
Now that the RISC-V IOMMU driver supports MSI remapping through iommufd enable IOMMUFD and VFIO. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
The generic VFIO and iommufd selftests can now run on RISC-V. Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
|
Patch 1: "[v3,01/19] iommufd: Convert struct iommufd_sw_msi_maps to a growable bitmap" |
|
Patch 1: "[v3,01/19] iommufd: Convert struct iommufd_sw_msi_maps to a growable bitmap" |
|
Patch 1: "[v3,01/19] iommufd: Convert struct iommufd_sw_msi_maps to a growable bitmap" |
|
Patch 1: "[v3,01/19] iommufd: Convert struct iommufd_sw_msi_maps to a growable bitmap" |
|
Patch 1: "[v3,01/19] iommufd: Convert struct iommufd_sw_msi_maps to a growable bitmap" |
|
Patch 1: "[v3,01/19] iommufd: Convert struct iommufd_sw_msi_maps to a growable bitmap" |
|
Patch 1: "[v3,01/19] iommufd: Convert struct iommufd_sw_msi_maps to a growable bitmap" |
|
Patch 1: "[v3,01/19] iommufd: Convert struct iommufd_sw_msi_maps to a growable bitmap" |
|
Patch 1: "[v3,01/19] iommufd: Convert struct iommufd_sw_msi_maps to a growable bitmap" |
|
Patch 1: "[v3,01/19] iommufd: Convert struct iommufd_sw_msi_maps to a growable bitmap" |
|
Patch 17: "[v3,17/19] RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch" |
|
Patch 17: "[v3,17/19] RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch" |
|
Patch 17: "[v3,17/19] RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch" |
|
Patch 17: "[v3,17/19] RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch" |
|
Patch 17: "[v3,17/19] RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch" |
|
Patch 17: "[v3,17/19] RISC-V: KVM: Enable KVM_VFIO interfaces on RISC-V arch" |
|
Patch 18: "[v3,18/19] riscv: defconfig: Enable IOMMUFD and VFIO" |
|
Patch 18: "[v3,18/19] riscv: defconfig: Enable IOMMUFD and VFIO" |
|
Patch 18: "[v3,18/19] riscv: defconfig: Enable IOMMUFD and VFIO" |
|
Patch 18: "[v3,18/19] riscv: defconfig: Enable IOMMUFD and VFIO" |
|
Patch 18: "[v3,18/19] riscv: defconfig: Enable IOMMUFD and VFIO" |
|
Patch 18: "[v3,18/19] riscv: defconfig: Enable IOMMUFD and VFIO" |
|
Patch 18: "[v3,18/19] riscv: defconfig: Enable IOMMUFD and VFIO" |
|
Patch 18: "[v3,18/19] riscv: defconfig: Enable IOMMUFD and VFIO" |
|
Patch 18: "[v3,18/19] riscv: defconfig: Enable IOMMUFD and VFIO" |
|
Patch 18: "[v3,18/19] riscv: defconfig: Enable IOMMUFD and VFIO" |
|
Patch 18: "[v3,18/19] riscv: defconfig: Enable IOMMUFD and VFIO" |
|
Patch 18: "[v3,18/19] riscv: defconfig: Enable IOMMUFD and VFIO" |
|
Patch 19: "[v3,19/19] selftests/vfio: Allow building on RISC-V" |
|
Patch 19: "[v3,19/19] selftests/vfio: Allow building on RISC-V" |
|
Patch 19: "[v3,19/19] selftests/vfio: Allow building on RISC-V" |
|
Patch 19: "[v3,19/19] selftests/vfio: Allow building on RISC-V" |
|
Patch 19: "[v3,19/19] selftests/vfio: Allow building on RISC-V" |
|
Patch 19: "[v3,19/19] selftests/vfio: Allow building on RISC-V" |
|
Patch 19: "[v3,19/19] selftests/vfio: Allow building on RISC-V" |
|
Patch 19: "[v3,19/19] selftests/vfio: Allow building on RISC-V" |
|
Patch 19: "[v3,19/19] selftests/vfio: Allow building on RISC-V" |
|
Patch 19: "[v3,19/19] selftests/vfio: Allow building on RISC-V" |
|
Patch 19: "[v3,19/19] selftests/vfio: Allow building on RISC-V" |
|
Patch 19: "[v3,19/19] selftests/vfio: Allow building on RISC-V" |
3 participants
PR for series 1142375 applied to workflow__riscv__fixes
Name: iommu/riscv: Enable MSI remapping, IOMMU_DMA and VFIO
URL: https://patchwork.kernel.org/project/linux-riscv/list/?series=1142375
Version: 3