What is the most efficient way to exchange high volume data between 2 process?

Question

Recently I am building the software for a DVR.

It will be installed on a x86 pc server with 2 or more PCIE x4 video encoder card plugged in.

We have 2 seperated proces, one has to get encoded video data from these encoder card, the other has to save these data to hard driver. Why we have 2 process? Some histroy problem.

So, what kind of IPC should we use? Socket, Pipe, or shared memory?

Currently we are using socket.

Answer 1

With pipes on linux, you could use the splice functionality to get zero-copy moves of the data from one process to the other. Depending on how the driver for your encoder card works, potentially you could arrange things to get zero-copy all the way from the driver to the disk - the encoder card DMAs into memory and the disk DMAs it back out, without the CPU ever needing to look at it.

Answer 2

Use IPC:

• One shared memory region: A low overhead shared buffer between the two processes (see shmat()).
• One semaphore: Its counter would be the numbers of frames available (see semop()). The process that pumps video data from the camera would put a frame in the shared memory region, and put() on the semaphore. The process that record the frames to the disk would get() on the semaphore, and a frame would be in the shared memory.

It's a bit like implementing a queue with a semphore as a counter.