views:

310

answers:

2

I need a hint on how to write multiple NSData chunks to single file. Downloading a file using NSURLConnection in chunks. Each chunk is downloaded in a separate NSOperation thread. As the chunks finish downloading they need to be written to a file so combined result is the file downloaded.

What would be the best way to manage the NSData that is returned and writing it to a single file?

A: 

Assuming that you know what the final data size will be, and you want to stick to Foundation classes/Objective-C, you could create an instance of NSMutableData that can be shared across these operations. When an operation completes its chunk, it should lock some shared mutex, write its completed download to the appropriate place in the NSMutableData object and then unlock the shared mutex.

Once all the operations are joined, you can simply write the mutable data to a file using the writeToFile: convenience methods on the NSData class. If you're more proficient in C/BSD, you could also create the file as an mmap and simply write to it. Since all the data are going to discrete seconds in the mapping, you can write without locking a mutex. Once all the operations are joined, you can remove the mmap and close the file.

Jason Coco
C/BSD approach might be one way of doing it but i'm looking for Objective-C answer to this. What you gave will work but i'm concerned with the file size and keeping it all in memory until finish downloading. The file size will exceed 100MB.
MikeM
@user326943: If you are worried about file size, I think that mmap is the way to go. The other option would be to simply have a file handle open and seek/write the file. This option would require locking obviously and there may be a lot of contention as threads are joined since writing to the flash device is waaaay slow.
Jason Coco
@user326943: Another all Cocoa option might to be to have the operation write a temporary file named with something meaningful like the chunk's sequence in the main file. Once all the operations are joined, you can combine all these temporary files into the final, large file, reading and writing them in chunks on a single thread.
Jason Coco
A: 

Write each chunk to a separate file. Then when the last chunk is downloaded, concatenate each file in the correct order into a single large file.

lucius
Can you give more details on how that can be done. How do i append everything to a single file after i have the multiple files?
MikeM
You can use a loop with NSInputStream and initWithFileAtPath: to read a small buffer (4 kb is ideal) at a time, and use +[NSOutputStream outputStreamToFileAtPath:append:] to write to the new file.
lucius