Raw Socket Receive Buffer

Question

Hi All, We are currently testing a Telecom application over IP. We open a Raw Socket and receives messages from the remote side (msgrate@750+msgs/second approx size of 180 bytes excluding IP).

On top of the Raw socket sits a layer called SCTP (just like TCP) which is indicating every now and then that it is missing some packets. Now, we are running Wireshark on the receive node and we can see that packet in Wireshark.

It looks to me that the receive buffer of the socket is small causing IP(?) to drop messages. However, IP Pegs(netstat -sv) show NO dropped packets. We have tried setting the socket receive queue to 40000 without any success.

I would appreciate any pointers as to what option, if any, of IP layer should we be configuring or is there any specific socket option that we need to set.

Thanks in advance.

Answer 1

I have a few questions and a few things to think about. 1) Which implementation of SCTP are you using and on which OS. Some SCTP implementations are more robust than others. 2) Is SCTP negatively acknowledging the dropped packets? Search for a gap acks in wireshark. 3) And where you see the dropped packets in wireshark are you sure that these are not retransmissions? 4) Where in the system is wireshark monitoring? If it is not on the same wire as your application then it may be seeing messages which your application doesn't. 5) What exactly is the indication SCTP is giving?

If you believe that the IP socket rx buffer is overflowing then you could consider reducing the size of the SCTP RX window; this is often configurable in sctp stacks. The Rx window limits the amount of data that can be outstanding waiting for acknowledgement and consequently restricts the amount of data which could be in the IP buffer. You could also try raising the priority of your SCTP task so that it more quickly reads messages out of the IP buffers (This may be the easiest thing to try and in my opinion a good thing to do).

Regards

Answer 2

Thanks for your inputs. However, we have been able to "solve" this problem. Earlier, I described how we read messages. Once select returns, we run a loop (to the tune of number of Raw Messages to read which was >1 in our case). 1) we call ioctl(FIONREAD) to find the number of bytes to read; 2) read that many bytes by calling recvfrom 3) send the bytes upto the user 4) go into loop again and call ioctl(FIONREAD) and then repeat the steps

However, at point 4, ioctl(FIONREAD) use to return 0. Our code had a defensive check. It was expecting, a 0 bytes from ioctl(FIONREAD) means that the sender has send an IP header with 0 payload. Therefore, it use to call recvfrom(bytes to read=0) to flush out the IP header lest the select will set again on this.

At time t0, ioctl(FIONREAD) returns 0 as number of bytes to read At time t1, recvfrom(bytes to read=0) is called. Sometimes, between t0 and t1, actual data use to get queued in the socket receive queue and use to get discarded as we were calling recvFrom(bytes=0).

Setting, the number of rawMsgsToRead=1 has "solved" this problem. However, my guess is it will impact our performance. Is their any ioctl call which can differentiate between octets in the queue as 0 and IP header with payload 0