TCP Socket file transfer

Question

Hello, I'm trying to write a secure transfer file program using Python and AES and i've got a problem i don't totally understand. I send my file by parsing it with 1024 bytes chunks and sending them over but the server side who receive the data crashes ( I use AES CBC therefore my data length must be a multiple of 16 bytes ) and the error i get says that it is not.

I tried to print the length of the data sent by the client on the client side and the length of the data received on the server and it shows that the client is sending exactly 1024 bytes each time like it's supposed to, but the server side shows that at some point in time, a received packet is not and so less than 1024 bytes ( for example 743 bytes ).

I tried to put a time.sleep(0.5) between each socket send on the client side and it seems to work. Is it possible that it is some kind of socket buffer failure on the server side ? That too much data is being send too fast by the client and that it breaks somehow the socket buffer on the server side so the data is corrupted or vanish and the recv(1024) only receive a broken chunk? That's the only thing i could think of, but this may also be completely false, if anyone has an idea of why this is not working properly it would be great ;)

Following my idea i tried :

    self.s.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 32768000)
    print socket.SO_RCVBUF

I tried to put a 32mbytes buffer on the server side but On Windows XP it shows 4098 on the print and on linux it shows only 8. I don't know how i must interpret this, the only thing i know is that it seems that it doesn't have a 32mbytes buffer so the code doesn't work.

Well it's been a really long post, i hope some of you had the courage to read it all to here ! i'm totally lost there so if anyone has any idea about this please share it :D

Thanks to Faisal my code is here :

Server Side: ( count is my filesize/1024 )

while 1:
    txt=self.s.recv(1024)
    if txt == " ":
        break       
    txt = self.cipher.decrypt(txt)
    if countbis == count:
        txt = txt.rstrip()
    tfile.write(txt)
    countbis+=1

Client side :

while 1:
    txt= tfile.read(1024)
    if not txt:
        self.s.send(" ")
        break
    txt += ' ' * (-len(txt) % 16)
    txt = self.cipher.encrypt(txt)
    self.s.send(txt)

Thanks in advance,

Nolhian

Answer 1

TCP is a stream protocol, it doesn't conserve message boundaries, as you have just discovered.

Answer 2

Welcome to network programming! You've just fallen into the same mistaken assumption that everyone makes the first time through in assuming that client sends & server recives should be symmetric. Unfortunately, this is not the case. The OS allows reception to occur in arbitrarily sized chunks. It's fairly easy to work around though, just buffer your data until the amount you've read in equals the amount you wish to receive. Something along the lines of this will do the trick:

buff=''
while len(buff) < 1024:
    buff += s.recv( 1024 - len(buff) )

Answer 3

What TCP can guarantee is that all your data arrives, in the right order, at some point. (Unless something unexpected happens, by which it won't arrive.) But it's very possible that the data you send will still arrive in chunks. Much of it is because of limited send- and receive-buffers. What you should do is to continue doing your recv calls until you have enough data to process it. You might might have to call send multiple times; use its return value to keep track of how much data has been sent/buffered so far.

When you do print socket.SO_RCVBUF, you actually print the symbolic SO_RCVBUF contant (except that Python doesn't really have constants); the one used to tell setsockopt what you want to change. To get the current value, you should instead call getsockopt.

Answer 4

As others have pointed out you're probably processing an incomplete message. You need to either have fixed sized messages or have a delimiter (don't forget to escape your data!) so you know when a complete message has been received.

Answer 5

For many applications, the complexities of TCP are neatly abstracted by Python's asynchat module.