views:

461

answers:

5

I wrote a simple server and client apps, where I can switch between TCP, DCCP and UDP protocols. The goal was to transfer a file from the one to the other and measure the traffic for each protocol, so I can compare them for different network setups (I know roughly what the result should be, but I need exact numbers/graphs). Anyway after starting both apps on different computers and starting tcpdump I only get in the tcpdump-log the first few MBs (~50MB) from my 4GB file. The apps are written in a standard C/C++ code, which could be found anywhere on the web. What may be the problem or what could I be doing wrong here?

-- Edit

The command line I use is:

tcpdump -s 1500 -w mylog

tcpdump captures then packets only the first ~55 sec. That's the time the client needs to send the file to the socket. Afterwards it stops, even though the server continues receiving and writing the file to the hard drive.

-- Edit2

Source code:

client.cpp
server.cpp
common.hpp
common.cpp

-- Edit final

As many of you pointed out (and as I suspected) there were several misconceptions/bugs in the source code. After I cleaned it up (or almost rewrote it), it works as needed with tcpdump. I will accept the answer from @Laurent Parenteau but only for point 5. as it was the only relevant for the problem. If someone is interested in the correct code, here it is:

Source code edited

client.cpp
server.cpp
common.hpp
common.cpp

+7  A: 

There are many things wrong in the code.

  1. The file size / transfer size is hardcoded to 4294967295 bytes. So, if the file supplied isn't that many bytes, you'll have problems.
  2. In the sender, you aren't checking if the file read is successful or not. So if the file is smaller than 4294967295 bytes, you won't know it and send junk data (or nothing at all) over the network.
  3. When you use UDP and DDCP, the packets order isn't guarantee, so the data received may be out of order (ie. junk).
  4. When you use UDP, there's no retransmission of lost packet, so some data may never be received.
  5. In the receiver, you aren't check how many bytes you received, you always write MAX_LINE bytes to the file. So even if you receive 0 bytes, you'll still be writing to the file, which is wrong.
  6. When you use UDP, since you're sending in a thigh loop, even if the write() call return the same amount of bytes sent that what you requested, a lot of data will probably be dropped by the network stack or the network interface, since there's no congestion control in place. So, you will need to put some congestion control in place yourself.

And this is just from a quick scan of the code, there is probably more problems in there...

My suggestion is : Try the transfer with TCP, do a md5sum of the file you read/send, and a md5sum of the file you receive/save, and compare the 2 md5sum. Once you have this case working, you can move to testing (still using the md5sum comparison) with UDP and DCCP...

For the tcpdump command, you should change -s 1500 for -s 0, which means unlimited. With that tcpdump command, you can trust it that data not seen by it hasn't been sent/received. Another good thing to do is to compare the tcpdump output of the sender with the receiver. This way you'll know if some packet lost occurred between the two network stacks.

Laurent Parenteau
1. and 2. are reasonable, but it's just for test purposes and keeps the source code compact. 3., 4. and 6. are also known, the purpose of the tests is not to implement reliable transfer application, but to test protocol performance (mostly bandwidth). 5. is good point, I'll change that. I already tried the tests with TCP and the file arrives OK and independently of what value I've put for the snaplen tcp won't log behind the ~55sec.
stefita
And for how long after that the receiver is still receiving data? I am asking that since the data may well be received, but still in the stack's buffers (not yet read by the application). This could explain a small difference between the transfer time on the network, and the time it takes to read everything in the application and write that to the file.
Laurent Parenteau
it takes longer then a minute to write the rest of the file to the disk. tcpdump captures only handfull of the communication. If I start an ftp transfer than it captures all packets...
stefita
Regarding your previous comment "the purpose of the tests is not to implement reliable transfer application, but to test protocol performance (mostly bandwidth)". If you have a lot of packet drop, the performance will seems better than what it really is. Unless, of course, the final application doesn't care about packet drop and a reliable communication...
Laurent Parenteau
Regarding the md5sum of the sent and received file... Is that file a repeated pattern, or mostly random data?
Laurent Parenteau
It's random data.
stefita
@stefita - `tcpdump` is capturing all the packets being sent. The numerous bugs in your code are obscuring the fact that it's largely not sending most of the data you think it's sending. The receiver seems to be continuing to 'receive' because bugs in your code cause it to write data to the file even when it's not receiving any. `tcpdump` is functioning perfectly.
Omnifarious
+2  A: 

Do you have x term access? Switch to Wireshark instead and try with that - its free, open source, and probably more widely used than tcpdump today. (It was formerly known as Ethereal.)

Also, do try the following tcpdump options:

  • -xx print the link header and data of the packet as well (does -w write data?)
  • -C specify the max file size explicitly.
  • -U to write packet by packet to the file instead of flushing the buffer.
  • -p dont put the nic in promiscuous mode
  • -O dont use the packet matching optimizer as yours is a new app level protocol.
  • Are you using verbose output in tcpdump? This can make the buffers fill quickly so redirect stdout/err to a file when you run it.

Are these Gigabit ethernet card on both ends?

tholomew
Wireshark uses tcpdump/libpcap also, so it wouldn't make any difference.
stefita
Well libpcap is a packet capture library and tcpdump and wireshark are two distinct applications that use it. I'm guessing its an application level or configuration problem rather than a library one.
tholomew
@stefita: wouldn't or doesn't? Did you try it?
Default
A: 

I know this might sound silly, but are you sure it is not a problem of flush() of the file? I.e. the data are still in memory and not yet written to disk (because they do not amount to a sufficient quantity).

Try sync or just wait a bit until you are certain that enough data have been transmitted.

lorenzog
+1  A: 

tcpdump is used as a diagnostic and forensics tool by 10s of thousands (at least) programmers and computer security professionals worldwide. When a tool like this seems to be mishandling a very common task the first thing to suspect is the code you wrote, and not the tool.

In this particular case your code has a wide variety of significant errors. In particular, with TCP, your server will continue to write data to the file regardless of whether or not the client is sending any.

This code has race conditions that will result in non-deterministic behavior in some situations, improperly treats '\0' as being a special value in network data, ignores error conditions, and ignores end-of-file conditions. And that's just a brief reading.

In this case I am nearly certain that tcpdump is functioning perfectly and telling you that your application does not do what you think it does.

Omnifarious
+1  A: 

"That's the time the client needs to send the file to the socket. Afterwards it stops, even though the server continues receiving and writing the file to the hard drive."

This sound really weird. The socket buffers are way too small to allow this to happen. I really think that your server code only seems to receive data, while the sender actually has already stopped sending data.

IanH