Ruby File.size? seems to cache file sizes.

Question

I'm working on an app that will transcode videos that are sent into a drop box. Because videos could take anywhere from a few minutes to a few hours to be sent to the dropbox, I want to be sure the file is done being written before my app begins processing it.

I thought the easiest way to do this would be to check the size of the file with every pass of the daemon that indexes the dropbox. If the file size stays the same for two or more passes, the app can assume that the file is done being written.

However when I mess with this in the Rails Console for my app, it seems like the file size is being cached. I see this when I'm exporting a video file to the drop box directory. In OS X 10.6 finder, the File Size is being updated constantly as the video is being written. How ever if I run File.size? on the file being written, I get the same size repeatedly, until it randomly updates. Below is some sample output, I'm running the File.size? method about once a second.

Is there something I'm missing?

Thanks.

File.size?("/Volumes/FCP_Edit/ProgramingDropBox/test.mov")
=> 95053324
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/test.mov")
=> 95053324
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 23769068
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 23769068
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 23769068
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 23769068
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 23769068
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 23769068
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 64888832
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 64888832
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 64888832
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 64888832
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 64888832
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 64888832
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 64888832
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 64888832
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 123609088
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 336691200
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 336691200
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 336691200
>> File.size?("/Volumes/FCP_Edit/ProgramingDropBox/dumb.mov")
=> 336691200

Answer 1

I'm not used to Ruby, but it looks like it's reading the file size as the space it has in the disk, not the actual size in bytes.

Because every file usually occupies at least one sector of the disk, no matter if the actual file size if one byte.

I'd recommend to search through the documentation a better solution to read the actual file size.

Answer 2

So I still don't know why the File.size? method doesn't get the correct size, but I found that by calling the systems du <filename> that I each time the command is called I get an updated file size.

So basically what I'm doing is:

IO.popen("du <filename>").readlines.first

If this command returns the same string twice, I know the file is not being written to anymore.

If anyone has a better idea how to solve this problem I would love to hear it.