ATS: Caching a Translation Locally

Overview

Transparent by Default established that this engine has no architectural awareness of a system-level IOMMU or SMMU at all. Address Translation Services – ATS, a PCIe Base Specification extension maintained by PCI-SIG, introduced at PCIe 4.0 – is the one real, standard mechanism that changes that on request: a device requests a translation explicitly, caches the answer locally, and uses the cached result directly on every later access to the same address, without repeating IOTLB Misses and the Latency This Engine Can’t See’s own round trip each time.

Locator

Block diagram

All 11 steps

1. Two layers, by default. A WRITE arrives carrying an rkey and a virtual address the remote side chose. Target Selection
2. The Key Cache checks the rkey’s permission and bounds and translates the virtual address. The RNIC’s own result is an IOVA, which it treats as the final address. Key Caching
3. The write leaves the RNIC across PCIe with that IOVA. Nothing in the RNIC changes whether a system IOMMU is present. Transparent by Default
4. The IOMMU’s IOTLB holds this page, so the IOVA becomes a host physical address and the write lands in DRAM. With passthrough guests, this second check is what keeps one tenant out of another’s memory. Transparent by Default
5. An IOTLB miss. A second WRITE targets a page the IOTLB doesn’t hold. The Key Cache translates it as before; the miss happens past PCIe. IOTLB Misses and the Latency This Engine Can't See
6. The IOMMU walks its own page tables. Tiemann et al. (2022) measured about 330 ns added per IOTLB miss on their platform. IOTLB Misses and the Latency This Engine Can't See
7. The walk fills the IOTLB and the write goes on to DRAM. IOTLB Misses and the Latency This Engine Can't See
8. Nothing. There is no error and no signal; the completion looks the same as for a hit. The RNIC only sees its own DMA slot staying busy longer. IOTLB Misses and the Latency This Engine Can't See
9. With ATS. A device that supports ATS asks the IOMMU for a translation explicitly and keeps the answer in its own Address Translation Cache. No source this book found confirms shipped RNICs do this. ATS: Caching a Translation Locally
10. Later writes to that page look up the IOVA in the ATC instead of sending it across. ATS: Caching a Translation Locally
11. The RNIC issues the already-translated address and the IOTLB isn’t consulted. If the mapping changes, the IOMMU sends an Invalidate Request, and the device must answer with an Invalidate Completion only once the entry is actually cleared. ATS: Caching a Translation Locally

Deep walkthrough

What ATS actually adds: a request-and-cache round trip, paid once instead of every time. A device supporting ATS asks the system-level translation layer for a specific IOVA’s translation explicitly, receives a Translated Address in response, and stores it in its own Address Translation Cache (ATC, sometimes called a Dev-TLB). Every later access to that same IOVA resolves from the device’s own ATC directly – the device issues the already-translated address itself, skipping the transparent-retranslation path Transparent by Default already described, and skipping IOTLB Misses and the Latency This Engine Can’t See’s own round trip on every one of those later accesses.

A different question from the one Mid-Transaction Burst Teardown left open. That chapter asked whether this engine holds a private copy of its own Key-Cache-resolved translation for a burst’s duration, or re-touches the shared Key Cache mid-burst – a question about this engine’s own MR/Key Table layer, still unresolved. An ATC is a real, standard-defined mechanism at a different layer entirely: caching the system-level translation, not this engine’s own. Don’t read this chapter as answering that earlier open question – it’s the same shape of problem recurring one layer out, with its own, separately-specified answer this time, not a resolution of the first one.

Coherency isn’t left to chance – ATS defines its own invalidation protocol. A cached translation can go stale the same way any cache entry can: the mapping it reflects changes underneath it. ATS’s own answer is a real, defined request-completion exchange – the system-level translation layer sends an Invalidate Request downstream to the device’s ATC, and the device is required to respond with an Invalidate Completion only once it has actually cleared the entry, not merely acknowledged the request. This is a real, standard-defined answer to the same shape of question Mid-Transaction Deregistration left open for this engine’s own Key Cache – what happens when the thing a cached translation depends on changes while that translation is still in use – just settled explicitly, one layer out, by a mechanism this engine’s own layer doesn’t have.

The other real benefit ATS enables, named without extending this book’s own scope to it. ATS pairs with a separate PCIe mechanism, the Page Request Interface (PRI), that lets a device tolerate a translation not being resident yet – requesting it on demand rather than requiring everything pinned in advance. That’s architecturally the same shape of relaxation Registering a Memory Region’s own On-Demand Paging already described for this engine’s Key Table layer, one layer further out and governed by PCI-SIG rather than IBTA. This chapter names the parallel and stops there – PRI’s own mechanics belong to PCIe’s own specification, not this book’s.

Interface reference

References

• PCI-SIG, PCI Express Base Specification, Address Translation Services (ATS) extension, introduced at PCIe 4.0 – governs the request/response translation-caching mechanism and the Invalidate Request/Completion coherency protocol this chapter describes.
• No IBTA or RoCEv2 standard governs ATS or any system-level caching mechanism – entirely PCIe/host-architecture territory, named here only for what it means to this engine’s own default transparency, not re-derived from PCI-SIG’s own specification text.