How do I do dynamic data transfer and memory management across threads in C?

Question

Platform: ARM9

Programming Language C

Requirements - plain C and no external libs and no boost.

OS - REX RTOS

I have two threads running in an embedded platform -

1. one is at driver level handling all the comms and data transfer with the hardware.
2. the second thread runs the application that uses the data to/from the hardware.

The idea is to decouple the app thread from the driver thread so we can change the hardware and implementation in the hardware driver thread but have minimal impact on the application thread.

My challenge is that the data received from the hardware may be dynamic i.e. we do not know upfront how much memory the application thread should set aside for each request to/from the hardware as this is determined at run-time.

I was thinking the driver thread could inform the application thread that there is so much data to read. The application thread then allocates the memory and requests the driver thread to read the data. It is then up to the application thread to process the data accordingly. This way, all the memory management is within the application thread.

Answer 1

Couple options come to mind:

1) malloc the memory in the driver, free it in the app. But... we tend to avoid malloc use in anything that approaches a real time requirement. If you have access to malloc/free, and there are no "real time" concerns or memory fragmentation issues (i.e. your heap is large enough), then this is a fairly easy approach. The driver just sends the allocated pointer to the app thread via a message queue and the app free's the memory when done. Watch out for memory leaks.

2) Ring or circular buffers. The driver completely manages a fixed size ring buffer and simply sends a message to the application when a buffer is ready. See here for some details: Circular buffer. Then the application marks the data "available" again via a driver API, which helps hide the ring buffer details from the app thread. We use this approach for one of our drivers that has a very similiar set of requirements as you describe. In this case, you need to be concerned with determining the "best" size for the ring buffer, overflow handling in the driver, etc.

good luck!

Answer 2

You don't specify an OS, but you somehow have "threads". Except that one of them is at driver level (interrupt handler) and the other sounds like an application (userland/kernel). But that doesn't match up either, because your driver and app are communicating before the data is even processed.

Your terminology is confusing and not encouraging. Is this a homebrew (RT)OS or not?

If you have a real OS, there are established methods for writing drivers and handing data to userland. Read the documentation or use one of the existing drivers as reference.

If this is a custom OS, you can still refer to other open source drivers for ideas, but you clearly won't have things set up as conveniently. Preallocate all the memory in the driver code, fill it with data as it arrives, and hand it off to the application code. The amount of memory will be a function of how fast your app can process data, the largest amount of data you plan to accept, and how much internal data queuing is needed to support your app.

Answer 3

This being C, I have ended up having to make the app register a callback with the driver. The purpose of the callback is to process the data after the driver reads it from the device. The driver manages the memory i.e. allocates memory, invokes the callback and finally frees memory. Additionally, the callback only has read permission on the memory. Therefore, the app should ideally just copy the contents of the buffer to its own memory and exit from the callback right away. Then it is free to process the data when and how it wishes.

I updated the documentation to make it clear to uses of the app callback that it is assumed when the callback returns, the memory should no longer be considered valid. If the callback is used any other way, the behavior is undefined.

Answer 4

My first thought would be to use circular buffers. Here is some example code. Feel free to adapt this to your own uses. You probably wouldn't want global variables. And you might not want #defines:

#define LENGTH (1024)
#define MASK (LENGTH-1)
uint8 circularBuffer[ LENGTH ];
int circularBuffer_add = 0;
int circularBuffer_rmv = 0;

void copyIn( uint8 * circularBuffer, uint8 * inputBuffer, int n ) {
    int i;
    for( i = 0; i < n; i++ ) {
        circularBuffer[ circularBuffer_add ] = inputBuffer[ i ];
        circularBuffer_add = ( circularBuffer_add + 1 ) & MASK;
    } 
}

void copyOut( uint8 * circularBuffer, uint8 * outputBuffer, int n ) {
    int i;
    for( i = 0; i < n; i++ ) {
        outputBuffer[ i ] = circularBuffer[ circularBuffer_rmv ];
        circularBuffer_rmv = ( circularBuffer_rmv + 1 ) & MASK;
    } 
}

---

Also the above code assumes that your unit of data is datatype "uint8". You can change it so that it uses some other datatype. Or you can even make it generic and use memcpy() to to copy into the circularBuffer.

The main feature of this code is how it handles the add and rmv ptr.

---

Once you get things working with the above code. I suggest at some point switching over all your reads from the hardware to use your platform's direct-memory-access API.

It is important to switch to direct-memory-access because the above code uses a lot of cycles relative to DMA which uses almost zero cycles.