Ruby: Is there a better way to iterate over multiple (big) files?

Question

Here's what I'm doing (sorry for the variable names, I'm not using those in my code):

File.open("out_file_1.txt", "w") do |out_1|
  File.open("out_file_2.txt", "w") do |out_2|
    File.open_and_process("in_file_1.txt", "r") do |in_1|
      File.open_and_process("in_file_2.txt", "r") do |in_2|
        while line_1 = in_1.gets do
          line_2 = in_2.gets        #input files have the same number of lines
          #process data and output to files
        end
      end
    end
  end
end

The open_and_process method is just to open the file and close it once it's done. It's taken from the pickaxe book.

Anyway, the main problem is that the code is nested too deeply. I can't load all the files' contents into memory, so I have to iterate line by line. Is there a better way to do this? Or at least prettify it?

Answer 1

I'm not sure that this version is all that much better for the two-file case, but it's certainly less deeply nested.

outfiles = [1,2].map {|n| File.open("outfile#{n}.txt", 'w') }
infiles =  [1,2].map {|n| File.open("infile#{n}.txt", "r")}
while (lines = infiles.map {|f| f.gets})).all?
  lines.each_with_index {|l, n| outfiles[n].puts("processed #{l}")}
end
(outfiles + infiles).each {|f| f.close}

Answer 2

You don't need to use open's block syntax in cases where it makes no sense to do so

http://ruby-doc.org/core/classes/IO.html#M002239