How to use > in an xargs command?

Question

I want to find a bash command that will let me grep every file in a directory and write the output of that grep to a separate file. My guess would have been to do something like this

ls -1 | xargs -I{} "grep ABC '{}' > '{}'.out"

but, as far as I know, xargs doesn't like the double-quotes. If I remove the double-quotes, however, then the command redirects the output of the entire command to a single file called '{}'.out instead of to a series of individual files.

Does anyone know of a way to do this using xargs? I just used this grep scenario as an example to illustrate my problem with xargs so any solutions that don't use xargs aren't as applicable for me.

Answer 1

A solution without xargs is the following:

find . -mindepth 1 -maxdepth 1 -type f -exec sh -c "grep ABC '{}' > '{}.out'" \;

...and the same can be done with xargs, it turns out:

ls -1 | xargs -I{} sh -c "grep ABC '{}' > '{}.out'"

---

Edit: single quotes added after remark by lhunath.

Answer 2

Do not make the mistake of doing this:

sh -c "grep ABC {} > {}.out"

This will break under a lot of conditions, including funky filenames and is impossible to quote right. What you need to do, is this:

sh -c 'grep ABC "$1" > "$1.out"' -- {}

Applies to xargs as well as find.

By the way, never use xargs without the -0 option (unless for very rare and controlled one-time interactive use where you aren't worried about destroying your data).

Also don't parse ls. Ever. Use globbing or find instead: http://mywiki.wooledge.org/ParsingLs

Use find for everything that needs recursion and a simple loop with a glob for everything else:

find /foo -exec sh -c 'grep "$1" > "$1.out"' -- {} \;

or non-recursive:

for file in *; do grep "$file" > "$file.out"; done

Notice the proper use of quotes.

Answer 3

I assume your example is just an example and that you may need > for other things. GNU Parallel http://www.gnu.org/software/parallel/ may be your rescue. It does not need additional quoting as long as your filenames do not contain \n:

ls | parallel "grep ABC {} > {}.out"

If you have filenames with \n in it:

find . -print0 | parallel -0 "grep ABC {} > {}.out"

As an added bonus you get the jobs run in parallel.