Transfer file in blocks

Question

In C (UNIX), how can I transfer and receive a file in multiple blocks using a socket?

For example, if I had a file of 1234 bytes, and a block size of 500, I would transfer:

• 500 bytes,
• then 500 bytes,
• then 234 bytes

I have attempted this using fseek, read, write, but I just cannot get the logic right. Even a good reference would be much appreciated.

My socket routines are:

int readn(sd, chunk, bytesToRead);

int writen(sd, chunk, bytesToWrite);

Answer 1

If you're using TCP then all you need to do is send your data block (I assume you have some kind of protocol which tells you how many bytes are in the block, such as a header?) and when you get your block at the other end simply write it to the file that you are writing to. TCP will deal with making sure everything is arriving in the expected order so you should just be able to walk your way through the file reading in X bytes at a time and sending them and then on the recv side you simply recv your data and write it to the file... Just remember that every read you issue on your socket can return anywhere between 1 and "block size" bytes and that your protocol should be able to tell you how many to expect and that you should then loop until you have actually got as many bytes as you'd expect...

If you're using UDP then things get a little more fun as you need to track which block a particular datagram represents...

Homework question?

Answer 2

You have the right idea. The main thing to get right is to make sure that you're IO calls actually do what you tell them to... That means checking their return codes to make sure they wrote or read the amount you expected... Also, it looks like you've wrapped read and write... Perhaps you should start by making sure they are totally correct...

Answer 3

The read will only get as many bytes as are actually buffered at that time or the maximum toy specify. So if you read frequently you will get far smaller chunks. If this matters (and I cannot see why it should), you need to collate the data in a secondary buffer of your own until you have the number you expect.

Of course there are perfectly good file transfer protocols such as FTP that you could use instead.

Answer 4

Of the top of my head (not compiled, so see it as pseudocode):

bytesRemaining = 1234;
chunkSize = 500;
offset = 0;
bytesWritten = 0;

while(bytesRemaining > 0)
  {
    if(bytesRemaining < chunkSize)
      bytesToWrite = bytesRemaining;
    else
      bytesToWrite = chunkSize;

    writen(sd, chunk+offset, bytesToWrite);
    offset += bytesToWrite;
    bytesRemaining -= bytesToWrite;
  }

or if writen() return the number of bytes written:

bytesRemaining = 1234;
chunkSize = 500;
offset = 0;
bytesWritten = 0;

while(bytesRemaining > 0)
  {
    if(bytesRemaining < chunkSize)
      bytesToWrite = bytesRemaining;
    else
      bytesToWrite = chunkSize;

    bytesWritten = writen(sd, chunk+offset, bytesToWrite);
    offset += bytesWritten;
    bytesRemaining -= bytesWritten;
  }

The same logic can be applied to the readn() case.