diff --git a/2025-05-99-how-to-implement-zero-copy-communication/index.md b/2025-05-99-how-to-implement-zero-copy-communication/index.md new file mode 100644 index 0000000..b067115 --- /dev/null +++ b/2025-05-99-how-to-implement-zero-copy-communication/index.md @@ -0,0 +1,164 @@ +--- +title: Implementing True Zero-Copy Communication with iceoryx2 +date: 2025-05-17 +taxonomies: + tags: + - iceoryx2 + - zerocopy + authors: + - christian-eltzschig +--- + +### What Is iceoryx2 + +**iceoryx2** is a decentralized, service-based inter-process communication +(IPC) library designed for mission-critical, high-efficiency systems. It +achieves low latency and high throughput using a communication paradigm known +as zero-copy communication. + +In this article, we'll explore the core concept of zero-copy communication and +walk through how iceoryx2 implements it using a practical example from its +publish-subscribe API. + +### The Concept of Zero-Copy Communication + +The principle of zero-copy communication is similar to how shared pointers or +references work in programming: instead of copying data, we share access to a +single memory location. + +In the same spirit, zero-copy inter-process communication avoids unnecessary +data copies between processes. Data is produced once in a memory region that +all endpoints can access, and a reference or offset to that memory is shared. +This allows all participants to read the data directly, just like sharing a +pointer to a struct on the heap - but across process boundaries. + +In practice, this shared memory region could be: + +* POSIX shared memory +* GPU memory +* Memory-mapped hardware buffers + +The critical aspect: the sender writes the data directly into shared memory. +Then, the offset or index to that data is transferred to every receiver. +No copies are involved after the data is created. + +
+ +shared-memory-idea + +
+ +### Hands-On: iceoryx2 in Action + +We’ll now walk through the +[iceoryx2 publish-subscribe example](https://github.com/eclipse-iceoryx/iceoryx2/tree/main/examples/rust/publish_subscribe), +where data is sent by a publisher to multiple subscribers using zero-copy. + +Sensor APIs often provide a pointer to the latest data: + +```rust +let data_ptr = sensor_get_next_frame(); +``` + +However, this is not zero-copy: the data resides in sensor-managed memory, and +must be copied into shared memory before sending. For true zero-copy, we +need the sensor to write directly into shared memory, provided by the +zero-copy communication framework. + +* **1. Produce Data into Shared Memory** + First, we acquire an uninitialized sample from the publisher. This is a chunk + of memory from the publisher’s data segment, reserved for the user's payload. + Under the hood, an allocator manages this memory block: + + ```rust + let mut uninitialized_sample = + publisher.loan_slice_uninit(MAX_SIZE_OF_SENSOR_DATA_FRAME)?; + ``` + + Next, the sensor writes directly into this memory: + + ```rust + sensor_get_next_frame(uninitialized_sample.payload_mut()); + ``` + + * **Deep Dive:** + When the publisher is created: + 1. A shared memory object is created using `shm_open`. + 2. It is resized via `ftruncate` to reserve enough space. + 3. The segment is mapped into the process using `mmap`.

+ + In iceoryx2 the memory of the data segment is managed by a pool allocator. + It partitions memory into fixed-size regions, which avoids fragmentation + and allows predictable allocation times - essential for real-time and + safety-critical systems. + + The call `loan` allocates a chunk of memory from this pool allocator and + returns it to the user. + +* **2. Send Pointer To Data** + Once the data is produced, the sample is marked as initialized and published: + + ```rust + let sample = unsafe { uninitialized_sample.assume_init(); } + sample.send()?; + ``` + + * **Deep Dive:** + Only an offset to the memory location is sent to the subscribers, not the + actual data. + Each process has its own virtual memory layout, so we cannot directly + share pointers. Instead: + * The publisher calculates an offset from the base of the shared memory segment. + * The subscriber adds this offset to its local mapping of that segment. + * A valid pointer to the received data is reconstructed.

+ + To transfer the offset, an inter-process communication mechanisms like + message queues, pipes, unix domain sockets, or shared memory queues can be + used. + +
+ + offset + +
+ + +

+ +* **3. Read Data** + Upon receiving the offset, the subscriber can reconstruct a pointer and + access the data directly: + + ```rust + if let Some(sample) = subscriber.receive()? { + process_sensor_data(sample.payload()); + } + ``` + + * **Deep Dive:** + Before a subscriber can consume the received data: + 1. It must map the publisher’s shared memory using `shm_open` and `mmap`. + 2. It listens for incoming offsets from the publisher. + 3. On receipt, it reconstructs the pointer and reads the data — no copying + involved.

+ + This process ensures the subscriber can consume the data efficiently and + with minimal latency. + +### Summary + +The key difference between traditional communication (e.g., sockets or +serialization) and zero-copy inter-process communication lies in who owns and +provides the memory. + +* In classical inter-process communication, the user provides a buffer to the +system. +* In zero-copy inter-process communication, the framework provides the memory +which is shared between all endpoints. + +With iceoryx2, publishers produce data directly into shared memory. Subscribers +receive only an offset, which they resolve into local pointers. + +This guarantees true zero-copy communication. No serialization, no redundant +allocations, no data duplication - just high-performance, low-latency data +exchange. diff --git a/2025-05-99-how-to-implement-zero-copy-communication/offset.drawio b/2025-05-99-how-to-implement-zero-copy-communication/offset.drawio new file mode 100644 index 0000000..35e101a --- /dev/null +++ b/2025-05-99-how-to-implement-zero-copy-communication/offset.drawio @@ -0,0 +1,85 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/2025-05-99-how-to-implement-zero-copy-communication/offset.png b/2025-05-99-how-to-implement-zero-copy-communication/offset.png new file mode 100644 index 0000000..1d4ffb5 Binary files /dev/null and b/2025-05-99-how-to-implement-zero-copy-communication/offset.png differ diff --git a/2025-05-99-how-to-implement-zero-copy-communication/shared-memory-idea.drawio b/2025-05-99-how-to-implement-zero-copy-communication/shared-memory-idea.drawio new file mode 100644 index 0000000..d2963f3 --- /dev/null +++ b/2025-05-99-how-to-implement-zero-copy-communication/shared-memory-idea.drawio @@ -0,0 +1,51 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/2025-05-99-how-to-implement-zero-copy-communication/shared-memory-idea.png b/2025-05-99-how-to-implement-zero-copy-communication/shared-memory-idea.png new file mode 100644 index 0000000..fcce6d3 Binary files /dev/null and b/2025-05-99-how-to-implement-zero-copy-communication/shared-memory-idea.png differ