parsing commandline arguments as wildcards

Question

Hi everyone,

I wrote a little ruby script using optparse. I'd like to pass a list of files to my script using optparse. I would like to add a couple of files using wildcards (e.g. /dir/*). (Background: simple script, that writes all given arguments to a single text file, separated by newline.)

I tried this:

opts = OptionParser.new 
opts.on('-a', '--add FILE') do |s| 
  puts "DEBUG: before #{s}"
  @options.add = s
  puts "DEBUG: after #{@options.add}"
end
...
def process_arguments
  @lines_to_add = Dir.glob @options.add
end

put when I add files like this:

./script.rb -a /path/*

I always get only the first file in the directory. (All the debug outputs show only the first file of directory (seems as if optparse does some magic interpretations))

Anybody knows how to handle my problem?

Answer 1

Whats happening here is that the shell is expanding the wildcard before ruby gets to it. So really you are processing:

./script.rb -a /path/file1 /path/file2 ......

put quotes around /path/* to avoid the shell expansion and pass the wildcard to ruby:

./script.rb -a '/path/*'

Answer 2

You didn't mention which operating system you are using (it matters).

On Windows, whatever you type on the command line gets passed to the program without modification. So if you type

./script.rb -a /path/*

then the arguments to the program contain "-a" and "/path/*".

On Unix and other systems with similar shells, the shell does argument expansion that automatically expands wildcards in the command line. So when you type the same command above, the shell looks to find the files in the /path/* directory and expands the command line arguments before your program runs. So the arguments to your program might be "-a", "/path/file1", and "/path/file2".

An important point is that the script cannot find out whether argument expansion happened, or whether the user actually typed all those filenames out on the command line.

Answer 3

The shell is expanding the argument before it gets passed to your program. Either keep consuming filenames until you reach another option, or have the user escape the wildcards (e.g. ./script.rb -a '/path/*') and glob them yourself.

Answer 4

As mentioned above, the command-line is being parsed before the OS hands off the command to Ruby. The wildcard is being expanded into a list of space-delimited filenames.

You can see what will happen if you type something like echo * at the command-line, then, instead of hitting Return, instead hit Esc then *. You should see the * expanded into the list of matching files.

After hitting Return those names will be added to the ARGV array. OptionParser will walk through ARGV and find the flags you defined, grab the following elements if necessary, then remove them from ARGV. When OptionParser is finished any ARGV elements that didn't fit into the options will remain in the ARGV array where you can get at them.

In your code, you are looking for a single parameter for the '-a' or '--add FILE' option. OptionParser() has an Array option which will grab comma-separated elements from the command line but will subsequent space-delimited ones.

require 'optparse'

options = []
opts = OptionParser.new 
opts.on('-a', '--add FILE', Array) do |s| 
  options << s
end.parse!

print "options => ", options.join(', '), "\n"
print "ARGV => ",    ARGV.join(', '),    "\n"

Save that to a file and try your command line with -a one two three, then with -a one,two,three. You'll see how the Array option grabs the elements differently depending on whether there are commas or spaces between the parameters.

Because the * wildcard gets replaced with space delimited filenames you'll have to post-process ARGV after OptionParser has run against it, or programmatically glob the directory and build the list that way. ARGV has all the files except the one picked up in the -a option so, personally, I'd drop the -a option and let ARGV contain all the files.

You will have to glob the directory if * has to too many files and exceeds the buffer size. You'll know if that happens because the OS will complain.