tags:

views:

26

answers:

1

Xargs can be used to cut up the contents of standard input into manageable pieces and invoke a command on each such piece. But is it possible to know which piece it is ? To give an example:

seq 1 10 | xargs -P 2 -n 2 mycommand

will call

mycommand 1 2 &
mycommand 3 4 &
mycommand 5 6 &
mycommand 7 8 &
mycommand 9 10 &

But I would like to know in my "mycommand" script that

mycommand 1 2

is processing the first piece/segment, and so on. Is it possible to access that information ?

p.s. In the simple example above I can just look at the numbers and tell. But for arbitrary lists how does one access the information without actually injecting piece# in the input stream ?

A: 

I only see you can do this if you change your input and add the sequence number:

seq 1 10 | perl -ne '$. % 2 and print (($.+1)/2,"\n"); print' | xargs -n3 ...

It is unclear why you need this, but if your final goal is to keep the output in the same order as the input, it may be easier to use GNU Parallel:

seq 1 10 | parallel -j+0 -n2 -k mycommand 

Watch the intro video for GNU Parallel to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ

Ole Tange
Thanks. I had indicated the possibility of injecting the segment id before feeding the sequence to xargs. Since both xargs and the "injecter" needs to know the split_param, it is perhaps better to read it from an exported environment variable. I just wanted a quick and dirty nested paralleliser. Knowing the segment_id would allow me to tag the output-file generated by each xargs segment. But I think Gnu Parallel is the right way to go. I have been avoiding it as I am shit scared of perl syntax :)
srean
Watch the video and you will see that you do not need to know anything about perl to use GNU Parallel. If you know how to use xargs you already know how to use GNU Parallel.
Ole Tange