Command to find all files in a directory and concat them for a parameter?

Question

Hi,

I'm looking for a command which finds all files in a directory using a specific partern, lets say "*.txt" and create a list of parameters from it in BASH.

So if the dir contains: file1.txt file2.txt file3.txt nonsense.c

I need the string "file1.txt file2.txt file3.txt"

I knew there was a BASH/Unix command for this, but I can't remember it :-S. "find" didn't work for me...

Thanks!

Yvan Janssens

Answer 1

It sounds like you want ls and xargs.

Answer 2

$ echo "*.txt"
1.txt 2.txt ...

Answer 3

You have asked for one of the most basic features of any Unix shell. It's known as "globbing", for some reason, and it's built-in.

$ echo *.txt

Answer 4

If you are just looking for files in one directory, not in nested directories, you can use a glob argument to your command, which will be interpreted by the shell and produce one argument to your command for each matching filename:

$ echo *.txt
file1.txt file2.txt file3.txt

If you are looking for files nested arbitrarily deeply within subdirectories, find is indeed what you are looking for. You need to pass in the directory you are looking in, and use the -name parameter with a quoted glob expression to find the appropriate files (if the glob is not quoted, the shell will try to interpret it, and substitute a list of files matching that name in the current directory, which is not what you want):

$ find . -name "*.txt"
file1.txt. file2.txt file3.txt subdir/file.txt

If you want to pass these in as arguments to another function, you can use $() (which is equivalent to the more familiar backquotes ``),

echo $(find . -name "*.txt")

Or if you are going to get a list that is too long for a single argument list, or need to support spaces in your filename, you can use xargs, which divides its input up into chunks, each of which is small enough to be passed into a command.

find . -name "*.txt" | xargs echo

If your filenames contain whitespace, then xargs will split them up into two pieces, treating each as a separate argument, which is not what you want. You can avoid this by using the null character, 0, as the delimiter, with the -print0 argument to find and the -0 argument to xargs.

find . -name "*.txt" -print0 | xargs -0 echo

Answer 5

If you want to run a command on files matched by a wildcard, you don't need extra baggage:

mycommand *.txt

What you remember is probably the xargs command. It takes a list of file names on its standard input, and runs a command on these files. For example echo *.txt | xargs mycommand is a complicated, and unreliable, way of writing mycommand *.txt. xargs is useful when the list of files that mycommand must act on is the output of some other command.

The reason I said xargs is unreliable is that it expects its input to be quoted in a peculiar way: all whitespace (not just newlines) separate names, backslashes must be doubled, and ' and " delimit literal strings (in which only backslash and the end quote are special). Since few commands produce output in the xargs input format, this limits xargs's usefulness to those rare cases where it is known that file names will not contain any of the special characters.

The find command is often used in combination with xargs, but this should, and can, be avoided. Instead of find ... | xargs mycommand, write

find ... -exec mycommand {} +

Answer 6

The simple answer is

mylist=$(ls *.txt)

NOTE: ls reports an error if nothing matches. Errors can be redirected to /dev/null.

For commands other than ls and find, if there are no matches, then wildcard usage usually returns the wildcard as the result; so it is necessary to handle the special case of no matches.

The following sets the variable "list" to all .txt files, or sets it to the empty string if no such files exist. The advantage over ls(1) is that if invokes no external command.

list=$(echo *.txt)
if [ "$list" = "*.txt" ] ; then list=""; fi

Example usage to run my command, but only if list is not empty:

test -n "$list" && my-command $list

Now, if your files may have spaces in their names, then an array is better:

List=()
let n=0
for x in *.txt
do
    test -e "$x" || exit
    List[n++]="$x"
done

Then use the list with

test ${#List[@]} -gt 0 && my-command "${List[@]}"

or

for value in "$List[@]" ; do my-command "$value" ; done 

All this may be a little different for ksh, yet it would be very similar.