rubyj handling large number of files

Question

I believe File.foreach('input.txt') would read file one line at a time. Could not fine any documentation on that though. Can anyone confirm that?

Also I wanted to create a gigantic file to test the difference betwween File.forach and File.open . If the file is really large then File.open should fail and File.foreach should succeed. Anyone knows any nifty *nix trick to create a gigantic file really fast?

Update:

On further reading I found following different ways to read a file. Not sure which one would try to read all of them at the same time. Will try some cases and will update this post.

f = File.open('input.txt')
a = f.readlines

f = File.open('input.txt')
a = f.get

f = File.open('input.txt')
f.each_line{ |s| puts s}

Answer 1

File.foreach("file") will create an enumerator for the lines in the file, so yes it does what you believe it to do. In order to create a large file fast use this (this creates a 1 GB file, you can change the count parameter if you need something larger):

dd if=/dev/random of=myfile.dat bs=$(( 1024 * 1024  )) count=1000

Answer 2

There's no particular reason that open should fail on a really large file. That is, open merely opens a file (for reading, writing or appending). It doesn't immediately slurp the file into memory.

In terms of Ruby documentation for I/O, I would start here:

http://ruby-doc.org/core/classes/IO.html