Skip to content
DMA Engine

DMA Engine

Nearly every chapter in this book eventually hands off to the same hardware: the DMA Engine is where a resolved Key Cache translation and an accepted RX PSN Check result both actually become bytes moving into or out of host memory. This part covers that engine directly – which address a transfer actually resolves against, the gather and scatter paths that move bytes once it has, the ordering guarantees software is entitled to assume (and the one it isn’t), and what happens when this shared engine’s own capacity is contended or the ground under a running transfer shifts mid-execution. This engine is also the deepest departure this book has taken from Queue Pair Management’s own template: it has no QP-indexed table of its own, no installation step, and no state machine – a shared, stateless-between-transfers engine every Queue Pair calls into, not another instance of the table-plus-cache-plus-eviction shape this book has drawn three times already.

Chapters

In-Flight Transfer State
Every per-QP context this book has covered so far survives between packets – installed once, held, updated in place. This engine keeps nothing like that. What it tracks exists only for the lifetime of one transfer, belongs to whichever QP asked for it, and disappears the instant that transfer completes.
Target Selection
A normal NIC’s DMA engine only ever writes into a buffer software chose ahead of time. This engine sometimes does that too – and sometimes places data at an address the wire packet itself names, resolved through a key software never touched at the moment the transfer runs. The opcode decides which. This is the one demand ordinary DMA hardware never had to meet.
Address Translation Handoff
This chapter is short on purpose. Every hard question about how a key becomes a physical address already has an owner two Parts back – this chapter’s only job is naming exactly where in this engine’s own flow that lookup gets called, and what this engine is and isn’t allowed to assume about its answer.
The Gather Path
Three opcodes need bytes read out of local memory before this adapter can send anything: a SEND building its own message, a WRITE building the payload the far end will place, and an RDMA READ response gathering exactly what the requester asked for. All three walk the same path – and the payload has to actually be in hand before Packet Build can assemble anything around it.
The Scatter Path
Every arriving payload lands in memory this side can address directly – the only question is who decided where. A RECEIVE lands where this side’s own software already posted a buffer. A WRITE lands where the wire packet itself said to. This chapter is the mechanism RX Pipeline has been calling DMA writeback all along, given its own name and its own depth.
Ordering and Outstanding-DMA Tracking
A completion notification is a weaker promise than it sounds like. IBTA guarantees a QP’s own requests execute in order – it does not guarantee a completed write is actually visible in memory yet, a gap real enough that a later IBTA extension added an entire new operation just to close it. Separately: this engine’s own concurrent-transfer capacity is a resource no QP-level attribute already covers.
Single-Burst Fault Handling
Every fault this book has named so far for a remote memory access happens before a transfer starts – a bad key, an out-of-range address, a permission this operation never had. This chapter is about the narrower case left over: a transfer that was correctly authorized, already running, and still can’t finish.
Multi-QP Burst Arbitration
Ordering and Outstanding-DMA Tracking named the resource: a fixed number of transfers this engine can run at once, shared across every Queue Pair calling into it. This chapter is what happens the instant that capacity is already spoken for and one more QP’s transfer needs a turn.
Mid-Transaction Burst Teardown
Two already-published chapters named this exact resource and left the same honest question open: what happens when the ground a running transfer depends on – the QP that owns it, or the Memory Region backing its translation – gets pulled out from under it mid-execution. This chapter doesn’t resolve either question more cleanly than they already declined to. It states them together, and says why.
Last updated on