How is.NET's NetworkStream delimiting multiple messages in the same packet?

Question

So I've been tasked with creating a tool for our QA department that can read packets off the wire and reassemble the messages correctly (they don't trust our logs... long story).

The application whose communication I'm attempting to listen in on is using .NET's TcpListener and TcpClient classes to communicate. Intercepting the packets isn't a problem (I'm using SharpPcap). However, correctly reassembling the packets into application level messages is proving slightly difficult.

Some packets have the end of one message and the beginning of the next message in them and I can't figure out how the NetworkStream object in .NET is able to tell where one application level message ends and the other begins.

I have been able to figure out that any packet that contains the end of an application level message will have the TCP header "PSH" (Push) flag turned on. But I can't figure out how .NET knows where exactly the end of the message is inside that packet.

The data of one packet might look like:

/></Message><Message><Header fromSystem=http://blah

How does the stream know to only send up to the end of </Message> to the application and store the rest until the rest of the message is complete?

There are no IP level flags set for fragmentation, and the .NET sockets have no knowledge of the application level protocol. So I find this incredibly vexing. Any insight would be appreciated.

Answer 1

The stream doesn't know the end of anything - it should be part of the application protocol.

NetworkStream doesn't have anything built into it to convert the data into an object. What makes you think it does? What does the code which manages to use the NetworkStream look like? Are you perhaps doing some form of XML deserialization, and the reading code automatically stops when it reaches the closing tag?

Basically:

• If your protocol doesn't have any message delimiter or length prefix, it probably should have
• NetworkStream itself is highly unlikely to be doing anything clever - but if you could tell us what you're observing, we can maybe work out what's going on.