Skip to content

zero copy blog article #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
elfenpiff wants to merge 1 commit into
base: main
Choose a base branch
from
Open

zero copy blog article #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
164 changes: 164 additions & 0 deletions 2025-05-99-how-to-implement-zero-copy-communication/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
---
title: Implementing True Zero-Copy Communication with iceoryx2
date: 2025-05-17
taxonomies:
tags:
- iceoryx2
- zerocopy
authors:
- christian-eltzschig
---

### What Is iceoryx2

**iceoryx2** is a decentralized, service-based inter-process communication
(IPC) library designed for mission-critical, high-efficiency systems. It
achieves low latency and high throughput using a communication paradigm known
as zero-copy communication.

In this article, we'll explore the core concept of zero-copy communication and
walk through how iceoryx2 implements it using a practical example from its
publish-subscribe API.

### The Concept of Zero-Copy Communication

The principle of zero-copy communication is similar to how shared pointers or
references work in programming: instead of copying data, we share access to a
single memory location.

In the same spirit, zero-copy inter-process communication avoids unnecessary
data copies between processes. Data is produced once in a memory region that
all endpoints can access, and a reference or offset to that memory is shared.
This allows all participants to read the data directly, just like sharing a
pointer to a struct on the heap - but across process boundaries.

In practice, this shared memory region could be:

* POSIX shared memory
* GPU memory
* Memory-mapped hardware buffers

The critical aspect: the sender writes the data directly into shared memory.
Then, the offset or index to that data is transferred to every receiver.
No copies are involved after the data is created.

<table width=100% height=100% border=0><tr><td align=center>

<img src="shared-memory-idea.png" alt="shared-memory-idea" style="max-width: 50%"/>

</td></tr></table>

### Hands-On: iceoryx2 in Action

We’ll now walk through the
[iceoryx2 publish-subscribe example](https://github.com/eclipse-iceoryx/iceoryx2/tree/main/examples/rust/publish_subscribe),
where data is sent by a publisher to multiple subscribers using zero-copy.

Sensor APIs often provide a pointer to the latest data:

```rust
let data_ptr = sensor_get_next_frame();
```

However, this is not zero-copy: the data resides in sensor-managed memory, and
must be copied into shared memory before sending. For true zero-copy, we
need the sensor to write directly into shared memory, provided by the
zero-copy communication framework.

* **1. Produce Data into Shared Memory**
First, we acquire an uninitialized sample from the publisher. This is a chunk
of memory from the publisher’s data segment, reserved for the user's payload.
Under the hood, an allocator manages this memory block:

```rust
let mut uninitialized_sample =
publisher.loan_slice_uninit(MAX_SIZE_OF_SENSOR_DATA_FRAME)?;
```

Next, the sensor writes directly into this memory:

```rust
sensor_get_next_frame(uninitialized_sample.payload_mut());
```

* **Deep Dive:**
When the publisher is created:
1. A shared memory object is created using `shm_open`.
2. It is resized via `ftruncate` to reserve enough space.
3. The segment is mapped into the process using `mmap`.<br><br>

In iceoryx2 the memory of the data segment is managed by a pool allocator.
It partitions memory into fixed-size regions, which avoids fragmentation
and allows predictable allocation times - essential for real-time and
safety-critical systems.

The call `loan` allocates a chunk of memory from this pool allocator and
returns it to the user.

* **2. Send Pointer To Data**
Once the data is produced, the sample is marked as initialized and published:

```rust
let sample = unsafe { uninitialized_sample.assume_init(); }
sample.send()?;
```

* **Deep Dive:**
Only an offset to the memory location is sent to the subscribers, not the
actual data.
Each process has its own virtual memory layout, so we cannot directly
share pointers. Instead:
* The publisher calculates an offset from the base of the shared memory segment.
* The subscriber adds this offset to its local mapping of that segment.
* A valid pointer to the received data is reconstructed.<br><br>

To transfer the offset, an inter-process communication mechanisms like
message queues, pipes, unix domain sockets, or shared memory queues can be
used.

<table width=100% height=100% border=0><tr><td align=center>

<img src="offset.png" alt="offset" style="max-width: 80%"/>
Copy link

@elBoberido elBoberido Jun 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It took me some time to understand the image. What do you think of something like this

                 Shared
          ^      Memory      ^ 
          |                  |
   0x4000 |  +------------*  | 0xD000
          |  |            |  |
          |  +------------*  |
          |  | payload    |  |
          |  +------------* <--------------
          |  |            |  |            | offset to payload
   0x3000 |--+------------*--| 0xC000 -----
          |                  |
          |                  |
          |                  |
         ---                ---
     Address Space      Address Space
       Publisher          Subscriber


</td></tr></table>


<br><br>

* **3. Read Data**
Upon receiving the offset, the subscriber can reconstruct a pointer and
access the data directly:

```rust
if let Some(sample) = subscriber.receive()? {
process_sensor_data(sample.payload());
}
```

* **Deep Dive:**
Before a subscriber can consume the received data:
1. It must map the publisher’s shared memory using `shm_open` and `mmap`.
2. It listens for incoming offsets from the publisher.
3. On receipt, it reconstructs the pointer and reads the data — no copying
involved.<br><br>

This process ensures the subscriber can consume the data efficiently and
with minimal latency.

### Summary

The key difference between traditional communication (e.g., sockets or
serialization) and zero-copy inter-process communication lies in who owns and
provides the memory.

* In classical inter-process communication, the user provides a buffer to the
system.
* In zero-copy inter-process communication, the framework provides the memory
which is shared between all endpoints.

With iceoryx2, publishers produce data directly into shared memory. Subscribers
receive only an offset, which they resolve into local pointers.

This guarantees true zero-copy communication. No serialization, no redundant
allocations, no data duplication - just high-performance, low-latency data
exchange.
85 changes: 85 additions & 0 deletions 2025-05-99-how-to-implement-zero-copy-communication/offset.drawio
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
<mxfile host="Electron" agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/25.0.2 Chrome/128.0.6613.186 Electron/32.2.5 Safari/537.36" version="25.0.2">
<diagram name="Page-1" id="7PI90tU65dkm2o4vBss-">
<mxGraphModel dx="1122" dy="703" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
<root>
<mxCell id="0" />
<mxCell id="1" parent="0" />
<mxCell id="33yOEqOoE3A8E4z9Wf7y-2" value="Process 1 / Publisher" style="rounded=1;whiteSpace=wrap;html=1;verticalAlign=middle;fillOpacity=40;fillColor=#dae8fc;strokeColor=#6c8ebf;fontSize=10;" parent="1" vertex="1">
<mxGeometry x="30" y="170" width="130" height="20" as="geometry" />
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-3" value="Process 2 / Subscriber" style="rounded=1;whiteSpace=wrap;html=1;verticalAlign=middle;fillOpacity=40;fillColor=#dae8fc;strokeColor=#6c8ebf;fontSize=10;" parent="1" vertex="1">
<mxGeometry x="30" y="200" width="130" height="20" as="geometry" />
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-4" value="" style="endArrow=none;dashed=1;html=1;rounded=0;fontColor=#CCCCCC;strokeColor=#CCCCCC;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" parent="1" target="33yOEqOoE3A8E4z9Wf7y-5" edge="1">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="180" y="230" as="sourcePoint" />
<mxPoint x="180" y="120" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-5" value="start address" style="text;html=1;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontColor=#CCCCCC;" parent="1" vertex="1">
<mxGeometry x="140" y="80" width="80" height="20" as="geometry" />
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-6" value="&amp;nbsp;0xC000" style="text;html=1;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontColor=#CCCCCC;fontSize=10;" parent="1" vertex="1">
<mxGeometry x="180" y="170" width="40" height="20" as="geometry" />
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-7" value="&amp;nbsp;0x3000" style="text;html=1;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontColor=#CCCCCC;fontSize=10;" parent="1" vertex="1">
<mxGeometry x="180" y="200" width="40" height="20" as="geometry" />
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-8" value="offset = 0x00FF" style="text;html=1;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontColor=#CCCCCC;fontSize=10;" parent="1" vertex="1">
<mxGeometry x="180" y="240" width="100" height="10" as="geometry" />
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-9" value="" style="endArrow=none;dashed=1;html=1;rounded=0;fontColor=#CCCCCC;strokeColor=#CCCCCC;entryX=0.628;entryY=1.073;entryDx=0;entryDy=0;entryPerimeter=0;" parent="1" edge="1">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="280.39" y="229.26999999999998" as="sourcePoint" />
<mxPoint x="279.99999999999994" y="160.7299999999999" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-1" value="Shared Memory" style="rounded=1;whiteSpace=wrap;html=1;glass=0;fillColor=#d5e8d4;strokeColor=#82b366;fillOpacity=40;verticalAlign=top;fontSize=10;" parent="1" vertex="1">
<mxGeometry x="180" y="120" width="160" height="40" as="geometry" />
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-10" value="" style="endArrow=none;html=1;rounded=0;strokeColor=#CCCCCC;" parent="1" edge="1">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="180" y="260" as="sourcePoint" />
<mxPoint x="180" y="240" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-11" value="" style="endArrow=none;html=1;rounded=0;strokeColor=#CCCCCC;" parent="1" edge="1">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="280" y="240" as="sourcePoint" />
<mxPoint x="280" y="260" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-12" value="" style="endArrow=none;html=1;rounded=0;strokeColor=#CCCCCC;" parent="1" edge="1">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="180" y="250" as="sourcePoint" />
<mxPoint x="280" y="250" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-13" value="&amp;nbsp;= 0xC0FF" style="text;html=1;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontColor=#CCCCCC;fontSize=10;" parent="1" vertex="1">
<mxGeometry x="280" y="175" width="60" height="10" as="geometry" />
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-14" value="&amp;nbsp;= 0x30FF" style="text;html=1;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontColor=#CCCCCC;fontSize=10;" parent="1" vertex="1">
<mxGeometry x="280" y="205" width="60" height="10" as="geometry" />
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-15" value="" style="endArrow=none;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;strokeColor=#CCCCCC;" parent="1" source="33yOEqOoE3A8E4z9Wf7y-13" edge="1">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="240" y="200" as="sourcePoint" />
<mxPoint x="350" y="190" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-16" value="" style="endArrow=none;html=1;rounded=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;strokeColor=#CCCCCC;" parent="1" target="33yOEqOoE3A8E4z9Wf7y-14" edge="1">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="350" y="200" as="sourcePoint" />
<mxPoint x="360" y="200" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-17" value="memory location in local process address space" style="text;html=1;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontColor=#CCCCCC;fontSize=10;" parent="1" vertex="1">
<mxGeometry x="350" y="190" width="110" height="10" as="geometry" />
</mxCell>
<mxCell id="33yOEqOoE3A8E4z9Wf7y-19" value="payload" style="rounded=0;whiteSpace=wrap;html=1;fontSize=10;strokeColor=#999999;fillOpacity=50;" parent="1" vertex="1">
<mxGeometry x="280" y="140" width="40" height="20" as="geometry" />
</mxCell>
</root>
</mxGraphModel>
</diagram>
</mxfile>
Binary file added 2025-05-99-how-to-implement-zero-copy-communication/offset.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
51 changes: 51 additions & 0 deletions 2025-05-99-how-to-implement-zero-copy-communication/shared-memory-idea.drawio
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
<mxfile host="Electron" agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/25.0.2 Chrome/128.0.6613.186 Electron/32.2.5 Safari/537.36" version="25.0.2">
<diagram name="Page-1" id="R9FgCAP87NmguzydoKBM">
<mxGraphModel dx="832" dy="696" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
<root>
<mxCell id="0" />
<mxCell id="1" parent="0" />
<mxCell id="8joSoVzIjDCAu3Ffybnu-10" value="3. read&lt;div&gt;data&lt;/div&gt;" style="text;html=1;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontColor=#CCCCCC;" vertex="1" parent="1">
<mxGeometry x="285" y="73" width="60" height="35" as="geometry" />
</mxCell>
<mxCell id="8joSoVzIjDCAu3Ffybnu-6" value="1. produce&lt;div&gt;data&lt;/div&gt;" style="text;html=1;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontColor=#CCCCCC;" vertex="1" parent="1">
<mxGeometry x="115" y="72" width="60" height="35" as="geometry" />
</mxCell>
<mxCell id="8joSoVzIjDCAu3Ffybnu-8" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.75;entryY=0;entryDx=0;entryDy=0;strokeColor=#CCCCCC;" edge="1" parent="1" source="8joSoVzIjDCAu3Ffybnu-3" target="8joSoVzIjDCAu3Ffybnu-1">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="8joSoVzIjDCAu3Ffybnu-3" value="Process 2" style="rounded=1;whiteSpace=wrap;html=1;verticalAlign=bottom;fillColor=#dae8fc;strokeColor=#6c8ebf;fillOpacity=40;" vertex="1" parent="1">
<mxGeometry x="300" y="110" width="90" height="50" as="geometry" />
</mxCell>
<mxCell id="8joSoVzIjDCAu3Ffybnu-2" value="Process 1" style="rounded=1;whiteSpace=wrap;html=1;verticalAlign=bottom;fillOpacity=40;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="1">
<mxGeometry x="70" y="110" width="90" height="50" as="geometry" />
</mxCell>
<mxCell id="8joSoVzIjDCAu3Ffybnu-1" value="Shared Memory" style="rounded=1;whiteSpace=wrap;html=1;glass=0;fillColor=#d5e8d4;strokeColor=#82b366;fillOpacity=40;" vertex="1" parent="1">
<mxGeometry x="120" y="120" width="220" height="20" as="geometry" />
</mxCell>
<mxCell id="8joSoVzIjDCAu3Ffybnu-4" value="" style="endArrow=blockThin;html=1;rounded=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.25;entryY=0;entryDx=0;entryDy=0;endFill=1;strokeColor=#CCCCCC;" edge="1" parent="1" source="8joSoVzIjDCAu3Ffybnu-2" target="8joSoVzIjDCAu3Ffybnu-1">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="250" y="380" as="sourcePoint" />
<mxPoint x="300" y="330" as="targetPoint" />
<Array as="points">
<mxPoint x="115" y="90" />
<mxPoint x="175" y="90" />
</Array>
</mxGeometry>
</mxCell>
<mxCell id="8joSoVzIjDCAu3Ffybnu-11" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.75;exitY=1;exitDx=0;exitDy=0;entryX=0.25;entryY=1;entryDx=0;entryDy=0;strokeColor=#CCCCCC;" edge="1" parent="1" source="8joSoVzIjDCAu3Ffybnu-2" target="8joSoVzIjDCAu3Ffybnu-3">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="250" y="380" as="sourcePoint" />
<mxPoint x="300" y="330" as="targetPoint" />
<Array as="points">
<mxPoint x="138" y="180" />
<mxPoint x="323" y="180" />
</Array>
</mxGeometry>
</mxCell>
<mxCell id="8joSoVzIjDCAu3Ffybnu-12" value="2. send pointer to data" style="text;html=1;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontColor=#CCCCCC;" vertex="1" parent="1">
<mxGeometry x="160" y="160" width="140" height="20" as="geometry" />
</mxCell>
</root>
</mxGraphModel>
</diagram>
</mxfile>
Binary file added 2025-05-99-how-to-implement-zero-copy-communication/shared-memory-idea.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.