Who's Messing Up this TCP Connection?

Question

I'm responsible for some embedded software that has to work with a customer's proprietary TCP interface (also embedded, but running under a well known and well regarded RTOS), but it's not getting through the three-way handshake, even though the HTTP interface, etc., all work fine, and I can communicate using the custom protocol with a program running on my PC.

Looking at the WireShark captures, his side initiates by sending a SYN, I send a SYN-ACK, and then he immediately sends a RST, so it looks like the problem is on his end. Is my analysis correct?

Here's a typical three packet example of the problem, with the MAC IDs anonymized (the real MAC IDs are valid). Sorry about pasting the raw hex, if anybody's got a better idea of how to put the WireShark capture up, I'm certainly amenable.

63  2009-06-29 13:07:49.685057 10.13.91.2 10.13.92.3 TCP 1024 > 49151 [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=0 TSV=194 TSER=0

0000   f1 f1 f1 00 03 09 ab ab ab 60 10 89 08 00 45 00  
0010   00 3c 00 68 40 00 40 06 6f 35 0a 0d 5b 02 0a 0d  
0020   5c 03 04 00 bf ff 7d b3 81 44 00 00 00 00 a0 02  
0030   20 00 9c 2f 00 00 02 04 05 b4 01 03 03 00 01 01  
0040   08 0a 00 00 00 c2 00 00 00 00  

64  2009-06-29 13:07:49.685375 10.13.92.3 10.13.91.2 TCP 49151 > 1024 [SYN, ACK] Seq=0 Ack=1 Win=1460 Len=0

0000   ab ab ab 60 10 89 f1 f1 f1 00 03 09 08 00 45 00  
0010   00 28 00 02 00 00 64 06 8b af 0a 0d 5c 03 0a 0d  
0020   5b 02 bf ff 04 00 d4 ff ff ff 7d b3 81 45 50 12  
0030   05 b4 47 07 00 00 00 00 00 00 00 00  

65  2009-06-29 13:07:49.685549 10.13.91.2 10.13.92.3 TCP 1024 > 49151 [RST] Seq=1 Win=0 Len=0

0000   f1 f1 f1 00 03 09 ab ab ab 60 10 89 08 00 45 00  
0010   00 28 00 6a 00 00 40 06 af 47 0a 0d 5b 02 0a 0d  
0020   5c 03 04 00 bf ff 7d b3 81 45 00 00 00 00 50 04  
0030   00 00 21 c9 00 00 00 00 00 00 00 00

Answer 1

First of all, those aren't valid MAC addresses; a high-order byte & 0x1 means it's a multicast MAC. See http://en.wikipedia.org/wiki/MAC_address

Answer 2

If you're not using fancy stuff on your side like custom tcp stack or raw sockets, I'd suspect the "proprietary TCP interface".

Has this ever worked with that client? Does it work with other clients?

Answer 3

If both of you are using standard RTOS implementations, it is unlikely the TCP stack has a problem. Or, did you say the TCP is locally implemented?

If his client sends a SYN properly, and you can reply with a SYN+ACK,
it would appear that either your SYN+ACK is not well formed
(but, I could not see anything wrong yet), or,
like you suspect, his TCP stack did not accept the SYN+ACK properly.
However, if these are standard implementations, that is unlikely.

So, what more can you do?

• Since it is the TCP handshake we are checking, you can just make him connect to any other machine at your end that is listening on the desired port

  • This will check his implementation (its good if the 3-way completes).

• You can check your TCP stack with a TELNET connect to the port from another local machine

  • This will check your implementation (good if 3-way completes).

• If both these things are fine, we need to suspect the network path

  • For example, could there be some firewall not allowing the communication and actively sending a RST to you?