Loop from contents of a file in ruby.

Question

Okay, so I am new to Ruby and I have a strong background in bash/ksh/sh.

What I am trying to do is use a simple for loop to run a command across several servers. In bash I would do it like:

for SERVER in `cat etc/SERVER_LIST`
do
    ssh -q ${SERVER} "ls -l /etc"
done

etc/SERVER_LIST is just a file that looks like:

server1
server2
server3
etc

I can't seem to get this right in Ruby. This is what I have so far:

 #!/usr/bin/ruby
### SSH testing
#
#

require 'net/ssh'

File.open("etc/SERVER_LIST") do |f|
        f.each_line do |line|
                Net::SSH.start(line, 'andex') do |ssh|
                        result = ssh.exec!("ls -l")
                        puts result
                end
        end
end

I'm getting these errors now:

andex@master:~/sysauto> ./ssh2.rb
/usr/lib64/ruby/gems/1.8/gems/net-ssh-2.0.23/lib/net/ssh/transport/session.rb:65:in `initialize': newline at the end of hostname (SocketError)
        from /usr/lib64/ruby/gems/1.8/gems/net-ssh-2.0.23/lib/net/ssh/transport/session.rb:65:in `open'
        from /usr/lib64/ruby/gems/1.8/gems/net-ssh-2.0.23/lib/net/ssh/transport/session.rb:65:in `initialize'
        from /usr/lib64/ruby/1.8/timeout.rb:53:in `timeout'
        from /usr/lib64/ruby/1.8/timeout.rb:93:in `timeout'
        from /usr/lib64/ruby/gems/1.8/gems/net-ssh-2.0.23/lib/net/ssh/transport/session.rb:65:in `initialize'
        from /usr/lib64/ruby/gems/1.8/gems/net-ssh-2.0.23/lib/net/ssh.rb:179:in `new'
        from /usr/lib64/ruby/gems/1.8/gems/net-ssh-2.0.23/lib/net/ssh.rb:179:in `start'
        from ./ssh2.rb:10
        from ./ssh2.rb:9:in `each_line'
        from ./ssh2.rb:9
        from ./ssh2.rb:8:in `open'
        from ./ssh2.rb:8

The file is sourced correctly, I am using the relative path, as I am sitting in the directory under etc/ (not /etc, I'm running this out of a scripting directory where I keep the file in a subdirectory called etc.)

Answer 1

The most common construct I see when doing by-line iteration of a file is:

File.open("etc/SERVER_LIST") do |f|
    f.each_line do |line|
      # do something here
    end
end

To expand on the above with some more general Ruby info... this syntax is equivalent to:

File.open("etc/SERVER_LIST") { |f|
    f.each_line { |line|
      # do something here
    }
}

When I was first introduced to Ruby, I had no idea what the |f| and |line| syntax meant. I knew when to use it, and how it worked, but not why they choose that syntax. It is, in my opinion, one of the magical things about Ruby. That simple syntax above is actually hiding a very advanced programming concept right under your nose. The code nested inside of the "do"/"end" or { } is called a block. And you can consider it an anonymous function or lambda. The |f| and |line| syntax is in fact just the handle to the parameter passed to the block of code by the executing parent.

In the case of File.open(), the anonymous function takes a single argument, which is the handle to the underyling File IO object.

In the case of each_line, this is an interator function which gets called once for every line. The |line| is simply a variable handle to the data that gets passed with each iteration of the function.

Oh, and one nice thing about do/end with File.open is it automatically closes the file at the end.

Edit:

The error you're getting now suggests the SSH call doesn't appreciate the extra whitespace (newline) at the end of the string. To fix this, simply do a

Net::SSH.start(line.strip, 'andex') do |ssh|
end

Answer 2

Use File.foreach:

require 'net/ssh'

File.foreach('etc/SERVER_LIST', "\n") do |line|
        Net::SSH.start(line, 'andex') do |ssh|
          result = ssh.exec!("ls -l")
          puts result
        end
end

Answer 3

File.open("/etc/SERVER_LIST", "r") do |file_handle|
  file_handle.each do |server|
    # do stuff to server here
  end
end

The first line opens the file for reading and immediately goes into a block. (The block is the code between do and end. You can also surround blocks with just { and }. The rule of thumb is do..end for multi-line blocks and {...} for single-line blocks. Blocks are very common in Ruby. Far more idiomatic than a while or for loop.) The call to open receives the filehandle automatically, and you give it a name in the pipes.

Once you have a hold of that, so to speak, you can call each on it, and iterate over it as if it were an array. Again, each iteration automatically passes you a line, which you call what you like in the pipes.

The nice thing about this method is that it saves you the trouble of closing the file when you're finished with it. A file opened this way will automatically get closed as you leave the outer block.

One other thing: The file is almost certainly named /etc/SERVER_LIST. You need the initial / to indicate the root of the file system (unless you are intentionally using a relative value for the path to the file, which I doubt). That alone may have kept you from getting the file open.

Update for new error: Net::SSH is barfing up over the newline. Where you have this:

 Net::SSH.start(line, 'andex') do |ssh|

make it this:

 Net::SSH.start(line.chomp, 'andex') do |ssh|

The chomp method removes any final newline character from a string.

Answer 4

Reading in lines from a file is a common operation, and Ruby has an easy way to do this:

servers = File.readlines('/etc/SERVER_LIST')

The readlines method will open the file, read the file into an array, and close the file for you (so you don't have to worry about any of that). The variable servers will be an array of strings; each string will be a line from the file. You can use the Array::each method to iterate through this array and use the code you already have. Try this:

servers = File.readlines('/etc/SERVER_LIST')
servers.each {|s|
    Net::SSH.start(s, 'andex') {|ssh| puts ssh.exec!("ls -l") }
}

Answer 5

I think this is what you want for the in 'initialize': newline at the end of hostname (SocketError) error:

Net::SSH.start(line.chomp, 'andex')

The each_line method includes the "\n", and the chomp will remove it.