Debugger for unix pipe commands

Question

As I build *nix piped commands I find that I want to see the output of one stage to verify correctness before building the next stage but I don't want to re-run each stage. Does anyone know of a program that will help with that? It would keep the output of the last stage automatically to use for any new stages. I usually do this by sending the result of each command to a temporary file (i.e. tee or run each command one at a time) but it would be nice for a program to handle this.

I envision something like a tabbed interface where each tab is labeled with each pipe command and selecting a tab shows the output (at least a hundred lines) of applying that command to to the previous result.

Answer 1

Use 'tee' to copy the intermediate results out to some file as well as pass them on to the next stage of the pipe, like so:

cat /var/log/syslog | tee /tmp/syslog.out | grep something | tee /tmp/grep.out | sed 's/foo/bar/g' | tee /tmp/sed.out | cat >>/var/log/syslog.cleaned

Answer 2

tee(1) is your friend. It sends its input to both the specified file and stdout.

Stick it between your pipes. For example:

ls | tee /tmp/out1 | sort | tee /tmp/out2 | sed 's/foo/bar/g'

Answer 3

You can also use pipes if you need bidirectional communication (i.e. with netcat):

mknod backpipe p
nc -l -p 80 0<backpipe | tee -a inflow | nc localhost 81 | tee -a outflow 1>backpipe

(via)

Answer 4

There's also the "pv" command - available in debian / ubuntu repostitories which shows you the throughput of your pipes.

An example from the man page : Transferring a file from another process and passing the expected size to pv:

          cat file | pv -s 12345 | nc -w 1 somewhere.com 3000

Answer 5

http://www.webdesignerdepot.com/rss.htm

I have the same issue. This command:

./somescript.sh > ../log/scriptlog.log

requires the output of a command go to std out. but inside the script

command | mailx -s "Subject" [email protected]

what I would like to do is something like :

command | tee > /dev/stdout | mailx -s "Subject" [email protected]

Where the output of the command goes to stdout( to be redirected into the ..log/scriptlog.log file )

and also into stdin for the mailx command.

Any way to do that?