What is the best way for interprocessor communication in Linux?

Question

I have two CPUs on the chip and they have a shared memory. This is not a SMP architecture. Just two CPUs on the chip with shared memory.

There is a Unix-like operating system on the first CPU and there is a Linux operating system on the second CPU.

The first CPU does some job and the result of this job is some data. After first CPU finishes its job it should say to another CPU that job is finished and the second CPU have to process this data.

What is the way to handle interprocessor communication? What algorithm should I use to do that?

Any reference to an article about it would be greatly appreciated.

Answer 1

A really good method is to just send IP packets back and forth (using sockets). this has the advantage that you can test stuff off-chip - as in, run a test version of one process on a PC, if you have networking.

Answer 2

How about using the shared mem?

I don't have a good link right now, but if you google for IPC + shared mem I bet you find some good info :)

Answer 3

Are you sure you need to do this? In my experience you're better off letting your compiler & operating system manage how your process uses multiple CPUs.

Answer 4

It all depends on the hardware. If all you have is shared memory, and no other way of communication, then you have to use a polling of some sort.

Are both of your processor running linux ? How do they handle the shared memory ? A good solution is to use a linked list as a fifo. On this fifo you put data descriptor, like adress and size.

For example, you can have an input and output fifo, and go like this :

• Processor A does some calculation
• Processor A push the data descriptoron the output fifo
• Processor A wait for data descriptor on the input fifo

• loop

• Processor B wait for data descriptor on the output fifo

• Processor B works with data
• Processor B push used data descriptor on the input fifo
• loop

Of course, the hard part is in the locking. May be you should reformulate your question to emphasize this is not 'standard' SMP.

If you have no atomic test and set bit operation available on the memory, I guess you have to go with a scheme where some zone of memory is write only for one processor, and read only for the other.

Edit : See Hasturkun answer, for a way of passing messages from one processor to the other, using ordered write instead of atomicity to provide serialized access to some predefined data.

Answer 5

If both processors are managed by a single os, then you can use any of the standard IPC to communicate with each other as OS takes care of everything. If they are running on different OSes then sockets would be your best bet.

Answer 6

EDIT

Quick unidirectional version:

• Ready flag
• Done flag

init:

init()
{
    ready = 0;
    done = 1;
}

writer:

send()
{
    while (!done)
        sleep();

    /* copy data in */
    done = 0;
    ready = 1;
}

reader:

poll()
{
    while (1)
    {
        if (ready)
        {
            recv();
        }
        sleep();
    }
}

recv()
{
    /* copy data out */
    ready = 0;
    done = 1;
}

---

Build a message passing system via the shared mem (which should be coherent, either by being uncached for both processors, or by use of cache flush/invalidate calls).

Your shared memory structure should have (at least) the following fields:

• Current owner
• Message active (as in, should be read)
• Request usage fields

Flow will probably be like this: (assumed send/recv synchronized not to run at same time)

poll()
{
    /* you're better off using interrupts instead, if you have them */
    while(1)
    {
        if (current_owner == me)
        {
            if (active)
            {
                recv();
            }
            else if (!request[me] && request[other])
            {
                request[other] = 0;
                current_owner = other;
            }
        }
        sleep();
    }
}

recv()
{
    /* copy data... */
    active = 0;
    /* check if we still want it */
    if (!request[me] && request[other])
    {
        request[other] = 0;
        current_owner = other;
    }
}

send()
{
    request[me] = 1;
    while (current_owner != me || active)
    {
        sleep();
    }

    request[me] = 0;

    /* copy data in... */

    /* pass to other side */
    active = 1;
    current_owner = other;
}

Answer 7

Ok. I understand the question.I have worked on this kind of an issue.

Now first thing that you need to understand is the working of the shared memory that exists between the 2 CPUs. Because these shared memory can be accessed in different ways, u need to figure out which one suits u the best.

Most times hardware semaphores will be provided in the shared memory along with the hardware interrupt to notify the message transfer from one processor to the other processor.

So have a look at this first.