How best to handle large buffers in a layered protocol stack?

Question

Hi,

I am working on a simple protocol stack for a small embedded system (multidrop, rs485 type stuff). In this stack, losely models after OSI layers:

1. Application
2. Network
3. Datalink
4. physical (serial driver)

Each layer has its own header / footer portion that wraps the payload of the layer above it.

I will be using my own buffer pool of statically allocated fixed sized blocks to store the binary packets. (No malloc / free in this app.)

In other API's I have seen that the data is usually passed as a const pointer with an associated length. In this way the data would need a copy operation at each layer as the payload of the layer above is placed in a newly allocated buffer for the current layer.

For a three layer stack this would be 2 copy operations and 3 allocated buffers.

Is there a better way to do this and still maintain clean separation of the protocol layers?

To better anchor the discussion, lets say the packets are typically around 2k and the processor is a small 8 bit micro running at 8Mhz.

Answer 1

In other API's I have seen that the data is usually passed as a const pointer with an associated length. In this way the data would need a copy operation at each layer as the payload of the layer above is placed in a newly allocated buffer for the current layer.

I'm guessing you're giving an example of an API for a transmit buffer.

I think you can keep the same API, if you add the restriction that whoever invokes this API is not allowed to use or touch that buffer again until after they receive a subsequent notification that the transmit operation has completed: so that invoking the API is implicitly transfering ownership of that buffer.

Answer 2

You could avoid the copies by having each layer request an empty buffer from the next lower layer, rather than allocating one itself:

• Application Layer requests buffer length LA from Network Layer.
• Network Layer requests buffer length LA+LN from Datalink Layer.
• Datalink Layer requests buffer length LA+LN+LD from Physical Layer.
• Physical Layer pulls a buffer from the buffer pool.
• Physical Layer returns buffer + phdr_len to Datalink Layer.
• Datalink Layer returns buffer + phdr_len + dhdr_len to Network Layer.
• Network Layer returns buffer + phdr_len + dhdr_len + nhdr_len to Application Layer.
• Application Layer fills out data in provided buffer, and calls Network Layer to transmit.
• Network Layer prepends header and calls Datalink Layer to transmit.
• Datalink Layer prepends header and calls Physical Layer to transmit.
• Physical Layer prepends header and passes to hardware.

Answer 3

Create a buffer structure. With knowledge of the maximum size at the bottom layer, allocate enough buffer space at the top layer to prepend each successive layer as it goes down the stack. Each layer moves the pointer in the buffer structure as a layer is added.

At the bottom layer the start of the buffer is recorded in the pointer in the buffer structure. The data to be sent is in a contiguous buffer. No data was copied at each layer.

Going from bottom to top you peel off layers within the buffer structure.