Remote Memory Copy

OKL4 2.1 introduced a new feature enabling threads to request the kernel to perform a memory-copy operation between different address spaces. This increases the efficiency of sharing memory across address spaces of regions greater than the maximum IPC payload but smaller than a page size (4 KB).

Motivation

Historically, OKL4 provided two fundamental mechanisms for cross-address-space communication. The first such mechanism is OKL4 IPC which allows threads to efficiently communicate by passing small data payloads to each other. The second mechanism revolves around sharing pages of memory between different address spaces. Although this mechanism can be efficient when sharing large amounts of data, for some uses cases, its usefulness is somewhat limited.

One limitation is that a shared buffer has to be allocated ahead of time, resulting in extra work for the programmer. Also this may not even be possible due certain constraints in particular libraries. The use of a shared buffer may also lead to double copying. The sender must first copy the memory into the shared buffer then the receiver may copy out of the shared buffer if the buffer is to be reused.

In certain situations, the existing mechanism also leads to unacceptable memory overheads as memory can only be securely shared in multiples of page sizes.

Usage

Remote memory copy is invoked by a new OKL4 system call, L4_MemoryCopy(). This system call requires a capability that refers to a memory region, an address local to the caller, a size and a direction flag.

A remote memory copy is initiated between 2 threads, thread A and thread B, by having A initiate an IPC call operation to B.

The message tag (message register 0) indicates a request for B to issue a copy operation. Subsequent message registers should contain the following information:

• The buffer address local to A.

• The size of A's copy buffer.

• A direction flag indicating whether A's buffer can be copied to, copied from, or both.

When B receives a memory copy request from A, it may invoke the memory copy system call. The remote copy system call accepts the following parameters:

• The IPC reply capability that B received from the IPC call.

• The buffer address local to B.

• The number of bytes to copy between A and B.

• A direction flag indicating the direction of the copy relative to B.

The kernel performs security checks to confirm whether the memory copy request is valid. In the event of a pagefault, the kernel will return the number of bytes copied up to the error condition. The thread making the system call will be notified of this condition.

In the event that the memory copy overflows the bounds of the destination memory region, the kernel will return the number of bytes copied up to the error condition. The thread making the system call will be notified of this condition.

Further Reading

The Remote Memory Copy system call is further described in Section A-11.7 MemoryCopy of the OKL4 Kernel Programming Manual.