How to determine total data upload+download in TCP/IP

Question

Hi, I need to calculate total data transfer while transferring a fixed size data from client to server in TCP/IP. It includes connecting to the server, sending request,header, receiving response, receiving data etc.

More precisely, how to get total data transfer while using POST and GET method?

Is there any formula for that? Even a theoretical one will do fine (not considering packet loss or connection retries etc)

FYI I tried RFC2616 and RFC1180. But those are going over my head.

Any suggestion?

Thanks in advance.

Answer 1

i'd say on average that request and response have about 8 lines of headers each and about 30 chars per line. Then allow for the size increase of converting any uploaded binary to Base64.

You didn't say if you also want to count TCP packet headers, in which case you could assume an MTU of about 1500 so add 16 bytes (tcp header) per 1500 data bytes

Finally, you could always setup a packet sniffer and count actual bytes for a sample of data.

oh yeah, and you may need to allow for deflate/gzip encoding as well.

Answer 2

You can't know the total transfer size in advance, even ignoring retransmits. There are several things that will stop you:

• TCP options are negotiated between the hosts when the connection is established. Some options (e.g., timestamp) add additional data to the TCP header
• "total data transfer size" is not clear. Ethernet, for example, adds quite a few more bits on top of whatever IP used. 802.11 (wireless) will add even more. So do HDLC or PPP going over a T1. Don't even think about frame relay. Some links may use compression (which will reduce the total size). The total size depends on where you measure it, even for a single packet.
• Assuming you're just interested in the total octet size at layer 2, and you know the TCP options that will be negotiated in advance, you still can't know the path MTU. Which may change, even while the connection is in progress. Or if you're not doing path MTU discovery (which would be wierd), then the packet may get fragmented somewhere, and the remote end will see a different amount of data transfer than you.

I'm not sure why you need to know this, but I suggest that:

• If you just want an estimate, watch a typical connection in Wireshark. Calculate the percent overhead (vs. the size of data you gave to TCP, and received from TCP). Use that number to estimate: it will be close enough, except in pathological situations.
• If you need to know for sure how much data your end saw transmitted and received, use libpcap to capture the packet stream and check.