ansaurus

Question

TCP Socket Sending Delays and Retransmission

Answer 1

A:

This only shows the TCP packets on the Linux machine, but I'd recommend to look at the ip stats with the 'netstat -s' command. One reason for the retransmissions might be socket buffer overflows, which will be shown with this command.

steve 2009-10-26 18:30:17

Here is the output...TCP Statistics for IPv4 Active Opens = 14801 Passive Opens = 1168 Failed Connection Attempts = 116 Reset Connections = 33 Current Connections = 46 Segments Received = 64272975 Segments Sent = 20797063 Segments Retransmitted = 486UDP Statistics for IPv4 Datagrams Received = 25465 No Ports = 52039 Receive Errors = 7098 Datagrams Sent = 26223

Michael Covelli 2009-10-26 18:59:37

IPv4 Statistics Packets Received = 64340244 Received Header Errors = 0 Received Address Errors = 3230 Datagrams Forwarded = 0 Unknown Protocols Received = 0 Received Packets Discarded = 38 Received Packets Delivered = 64329718 Output Requests = 20827053

Michael Covelli 2009-10-26 19:06:06

Routing Discards = 0 Discarded Output Packets = 0 Output Packet No Route = 0 Reassembly Required = 15207 Reassembly Successful = 5051 Reassembly Failures = 2587 Datagrams Successfully Fragmented = 0 Datagrams Failing Fragmentation = 0 Fragments Created = 0

Michael Covelli 2009-10-26 19:06:43

Thanks for taking a look, sorry for the poor formatting on the netstat -s output. As for the log from wireshark, I believe this shows both directions. The IP on the left is the source and the ip on the right is the destination.

Michael Covelli 2009-10-26 19:08:46

Michael, these stats are cumulative , meaning when the machine was started. You need to measure the diff, so we get more accurate data. From the stats it is clear that there are retransmits.Did you increase the socket receive buffer size? Is the nagle algorithm enabled?

steve 2009-10-26 19:40:19

Nagle is disabled for this connection (TcpNoDelay option). I will try to clear this and just capture one of these in the netstat output.I can try increasing the buffer size. Is changing it with the GetSocketOption/SetSocketOption generally preferred? Or changing it in the registry? What sizes are typical?

Michael Covelli 2009-10-26 19:52:11

Where would a socket buffer overflow show up in the netstat statistics?

Michael Covelli 2009-10-26 19:52:45

Answer 2

A:

Let's start by pruning some of that Wireshark output. We can toss the ARPs in packets 2, 3, 5 and 6. Looking at the rest, you have two sets of traffic in there. Packets 8 and 9 are two different connections, so you can't compare them. 7, 8 and 10, however, are part of one connection so let's examine those.

Packet 7 is 458 bytes of data being sent to the Linux box with a TCP sequence number of 1. However, the ACK that the Linux box returns is 4294964377. This means that Wireshark is showing relative TCP values and that the Linux box is not sending an ACK for packet 7, but for an earlier packet. Your PC then waits for a follow-up ACK and, when it doesn't get one, retransmits the required data. In this case the 458 bytes from packet 7 along with a previous 1002 bytes. That's why the sequence number from packet 10 matches the ACK from packet 8.

Unfortunately this doesn't tell you why data is being dropped. Packet 8 shows the Linux box indicating it still has a full 32k of input buffer available for this connection ("Win=32767").

Chris Ryding 2009-10-26 21:22:48

Thanks so much. My mistake was forgetting that I was sending two packets at the same time to that machine, one on port 20600 and one on 20700. So I had confused the two conversations. Now I guess my question is: how can I make the re-transmission faster? Why the long pause between the duplicate ack for an earlier packet and the re-transmission from my end? How can I make this faster?

Michael Covelli 2009-10-27 18:46:31

Can you run a capture on both your PC and the Linux box at the same time? This looks like a classic case of a packet not reaching its destination. I assume this connection is over a LAN with some sort of switch. If that's the case, maybe you have some other tools to monitor overall network congestion.I'm not sure if you can change the retransmit timeout, but in any event that seems to be treating symptoms and not the cause.

Chris Ryding 2009-10-27 20:42:30

Actually, I did manage to drastically reduce the number of transmissions today by increasing the ARP cache timeout. I noticed whenever more than 2 minutes elapsed between sends on the socket, the probability of a re-transmit increased dramatically. I looked for anything with a period of 2 minutes and found the ARP cache and increased it and there weren't any issues for the past several hours. But that still leaves two question - 1) Why does increasing the ARP cache timeout fix it? 2) Is there a way to shorten the re-transmit time for those times when it still does happen.

Michael Covelli 2009-10-27 21:22:50

Ah, so the delay is tied to your PC performing an ARP resolution when the ARP cache entry expires. Basically the PC has to figure out the hardware address of the Linux box again when the entry expires and will hold any pending transmit until that happens. There's a couple of kludges you can try to minimize issues: ping the Linux box once a minute or so, add a static ARP entry on your PC or increase the cache timeout to some silly amount like 8 hours. Not good for actual deployments, though.

Chris Ryding 2009-10-28 21:55:15

Answer 3

A:

I don't recall if Windows has it, but on UNIX you'd enable TCP_NODELAY.

This disables TCP's Nagle Algorithm which makes the system wait for a small time in case more data is going to be added to the transmit buffer.

int nodelay = 1;
setsockopt(s, IPPROTO_TCP, TCP_NODELAY, &nodelay, sizeof(nodelay));

Alnitak 2009-10-26 21:30:50

Nagling is dis-abled. Syntax is very similar in .NETsock = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp); sock.NoDelay = true;

Michael Covelli 2009-10-27 17:08:49

Answer 4

A:

http://support.microsoft.com/kb/214397

Moritz 2010-06-29 14:24:17

ansaurus

tags:

views:

answers:

TCP Socket Sending Delays and Retransmission

related questions