on a project I am working on, we are seeing out-of-order issues in certain circumstances on a SMP system when we are reading a UDP stream from the network. We can see it arrives from the network in order by sniffing off a hub connected between the sender and receiver. However sometimes it appears to arrive out of order when read from the socket. Is there any guarantee for UDP packets in this case or should the application implement a re-order buffer? We are not setting the CPU affinity here, I suspect that might help but ideally I would like all the CPU/hw threads to handle the network traffic.
views:
75answers:
2
+8
A:
UDP does not guarantee any ordering whatsoever. It is the responsibility of the application. In fact it does not even guarantee that the packets won't be repeated/ dropped etc. I suggest you read: http://en.wikipedia.org/wiki/User_Datagram_Protocol
It does not make sense for the kernel to make any such guarantees, especially if the incoming packets themselves could be out of order, as the kernel can (reasonably) expect that the application will deal with it, if the application requires ordering.
Moron
2010-06-25 16:37:15
I understand that, and understand that there should be a reorder buffer, but what I am saying is that they are coming in ordered, but it appears that the kernel is scrambling the order. Is this acceptable for the kernel?
tylernol
2010-06-25 16:56:59
@tylernol: Yes, it is acceptable for kernel to be able to give the app the udp packets in any order. Why should we even expect that? The protocol itself does not guarantee it, and having the kernel ensure that it provides the packets in the order it gets would only complicate matters (in the kernel) without any use whatsoever.
Moron
2010-06-25 17:05:22
@tylernol: Maybe this will help you: http://lkml.indiana.edu/hypermail/linux/net/0211.2/0036.html
Moron
2010-06-25 17:12:38
yup this is exactly what I was looking for:http://lkml.indiana.edu/hypermail/linux/net/0211.2/0037.html
tylernol
2010-06-25 19:08:06
A:
You can't be guaranteed that a UDP packet won't get dropped during transmission, so you can't have any ordering guarantees. When the system receives, for example, packet #14 and packet #16, it has no way of knowing if it should wait for packet #15 to come in before delivering packet #16, or if packet #15 was dropped and will never come in. The system will just hand you a bunch of packets, and it's up to you to put them in order.
bta
2010-06-25 16:48:36
yes that is what we get with UDP, it hands it off as it gets them, but should the kernel be introducing out-of-order sequencing as it handles the packets coming off the network?
tylernol
2010-06-25 17:02:18
@tylernol- Does it matter if the kernel changes the order? Your app doesn't know the order in which the kernel received the packets, so the question is irrelevant. All your app knows is that it should be prepared to re-order the packets, regardless of the reason why they might be out-of-order.
bta
2010-06-25 17:32:28
very true,now if I can only persuade the author of this particular interface in the app to do what they are supposed to do..thanks!
tylernol
2010-06-25 19:10:52