tags:

views:

160

answers:

2

I need to read a file in MB chunks, is there a cleaner way to do this in Ruby:

FILENAME="d:\\tmp\\file.bin"
MEGABYTE = 1024*1024
size = File.size(FILENAME)
open(FILENAME, "rb") do |io| 
  read = 0
  while read < size
    left = (size - read)
    cur = left < MEGABYTE ? left : MEGABYTE
    data = io.read(cur)
    read += data.size
    puts "READ #{cur} bytes" #yield data
  end
end
+4  A: 

Adapted from the Ruby Cookbook page 204:

FILENAME="d:\\tmp\\file.bin"
MEGABYTE = 1024*1024

class File
  def each_chunk(chunk_size=MEGABYTE)
    yield read(chunk_size) until eof?
  end
end

open(FILENAME, "rb") do |f|
  f.each_chunk() {|chunk| puts chunk }
end

Disclaimer: I'm a ruby nub and haven't tested this.

mbarkhau
Yes, this works. However, I thought that IO.read would throw if the number of bytes left was less than chunk size. I thought that because I had read about IO.readbyte, which will throw TruncatedDataError. Looks like that does NOT apply to read. An oversite on my part.Thanks!
A: 
FILENAME="d:/tmp/file.bin"

class File
  MEGABYTE = 1024*1024

  def each_chunk(chunk_size=MEGABYTE)
    yield self.read(chunk_size) until self.eof?
  end
end

open(FILENAME, "rb") do |f|
  f.each_chunk {|chunk| puts chunk }
end

It works, mbarkhau. I just moved the constant definition to the File class and added a couple of "self"s for clarity's sake.

Mereghost