views:

4273

answers:

3

I have a program whose STDERR output I want to inspect and run grep on etc.

So I could redirect it to STDOUT and use grep, but the problem is, I do not want the original STDOUT content.

So, this one won't do

cmd 2>&1 | grep pattern

because it will mix the original STDOUT and STDERR.

And this one doesn't work since grep doesn't read the STDERR output:

cmd 1>/dev/null | grep pattern

But also, this one won't work:

cmd 1>/dev/null 2>&1 | grep pattern

because the output will be completely empty, since everything gets written to /dev/null.

But there must be a simple way to do it?

+2  A: 

Close STDOUT first:

1>&-, >&-

See here.

Charlie Martin
Uhmm, are you sure that would do what the OP wanted? Doesn't seem to for me. If you close stdout first, there is no way to redirect stderr to it.
Tom Alsberg
Yes, that way (closing it after the redirection) it works, and where supported is sometimes better than using /dev/null because you save on writes - I will update my answer to mention that. Note that this is an extension supported by some shells (including bash), but apparently not standard in sh.
Tom Alsberg
Closing stdout, rather than sending to /dev/null, only works if the command writing to stdout ignores errors on write - and it is a sad commentary on C programming that in fact, most times, the program will ignore those errors.
Jonathan Leffler
You did specifically tag it as bash after all.
Charlie Martin
@Jonathan Leffler - right, as I mentioned - closing stdout will not work if the command actually bails on write errors.As you mention however, most C programs do not. I try to always check for errors, I normally do not check the output of printf if it is just printing messages. Sad? Not sure.
Tom Alsberg
@Charlie Martin. Right, it was tagged bash and the answer is probably good. I do not know too much about bash's extensions so I actually learned something new here :-)
Tom Alsberg
@Jonathan Leffler - as for ignoring return of write - when the write is "important" (has an effect on running on) then of course I agree that it must be checked for errors. But for usual output - consider what happens when piping to head, for example, if your program fails when stdout is closed.
Tom Alsberg
A: 

I would try something simple like:

cmd 2> tmp_file && cat tmp_file | grep pattern && rm -f tmp_file
Martin Cote
Jonathan Leffler
Also consider that the output may be very large, and with cat you will see the grep only at the end, it will take more time, and fail if there is not enough space to write the temporary file. Unless you need random access to the output later it is better to use a pipe "on the fly".
Tom Alsberg
+14  A: 

What does not work:

The reason the last command you quoted:

cmd 1>/dev/null 2>&1 | grep pattern

does not work, stems from a confusion on the order in which redirection works. You expected the last quoted redirection to be applied to the ones before it on every output, so that output the original standard output file descriptor (1) will go to /dev/null, and output to the standard error file descriptor (2) will go to the original standard output.

However, this is not how shell redirection works. Each redirection causes the file descriptors to be "remapped" by closing the "source" and duplicating the "destination" into it (see the man pages of dup(2) and close(2)), in order. This means that in your command standard output is first replaced with /dev/null, and then standard error replaced with standard output, which is /dev/null already.

What works:

Therefore, to obtain the desired effect, you just need to reverse the redirections. Then you will have standard error go to standard output, and the original standard output go to /dev/null:

cmd 2>&1 >/dev/null | grep pattern

(note that the 1 before > is unnecessary - for output redirection standard output is the default)


Addendum: Charlie mentioned redirecting to &- to close a file descriptor. If using an interactive shell which supports that extension (bash and some other implementations do but not all and it is not standard), you can also do it like this:

cmd 2>&1 >&- | grep pattern

This may be better - it can save some time, because when the command tries to write to standard output the call to write may fail immediately without waiting for a context switch into the kernel and the driver handling /dev/null (depending on the system call implementation - some may catch this in the libc function, and some may also have special handling for /dev/null). If there is a lot of output that can be worthwhile, and it's faster to type.

This will mostly work because most programs do not care if they fail to write to standard output (who really checks the return value of printf?) and will not mind that standard output is closed. But some programs can bail out with a failure code if write fails - usually block processors, programs using some careful library for I/O or logging to stdandard output. So if it doesn't work remember that this is a likely cause and try /dev/null.

Tom Alsberg