Redirecting CGI error output from STDERR to a file (python AND perl)

Question

I'm moving a website to Hostmonster and asked where the server log is located so I can automatically scan it for CGI errors. I was told, "We're sorry, but we do not have cgi errors go to any files that you have access to."

For organizational reasons I'm stuck with Hostmonster and this awful policy, so as a workaround I thought maybe I'd modify the CGI scripts to redirect STDERR to a custom log file.

I have a lot of scripts (269) so I need an easy way in both Python and Perl to redirect STDERR to a custom log file.

Something that accounts for file locking either explicitly or implicitly would be great, since a shared CGI error log file could theoretically be written to by more than one script at once if more than one script fails at the same time.

(I want to use a shared error log so I can email its contents to myself nightly and then archive or delete it.)

I know I may have to modify each file (grrr), that's why I'm looking for something elegant that will only be a few lines of code. Thanks.

Answer 1

Python: cgitb. At the top of your script, before other imports:

import cgitb
cgitb.enable(False, '/home/me/www/myapp/logs/errors')

(‘errors’ being a directory the web server user has write-access to.)

Answer 2

In Perl try CGI::Carp

BEGIN { 
use CGI::Carp qw(carpout); 
use diagnostics;
open(LOG, ">errors.txt"); 
carpout(LOG);
close(LOG);
}

use CGI::Carp qw(fatalsToBrowser);

Answer 3

For Perl, just close and re-open STDERR to point to a file of your choice.

close STDERR;
open STDERR, '>>', '/path/to/your/log.txt' 
  or die "Couldn't redirect STDERR: $!";

warn "this will go to log.txt";

Alternatively, you could look into a filehandle multiplexer like File::Tee.

Answer 4

Python has the sys.stderr module that you might want to look into.

>>>help(sys.__stderr__.read)
Help on built-in function read:

read(...)
    read([size]) -> read at most size bytes, returned as a string.

    If the size argument is negative or omitted, read until EOF is reached.
    Notice that when in non-blocking mode, less data than what was requested
    may be returned, even if no size parameter was given.

You can store the output of this in a string and write that string to a file.

Hope this helps

Answer 5

python:

import sys

sys.stderr = open('file_path_with_write_permission/filename', 'w')

Answer 6

In my Perl CGI programs, I usually have

BEGIN {
  open(STDERR,'>>','stderr.log');
}

right after shebang line and "use strict;use warnings;". If you want, you may append $0 to file name. But this will not solve multiple programs problem, as several copies of one programs may be run simultaneously. I usually just have several output files, for every program group.

Answer 7

The solution I finally went with was similar to the following, near the top of all my scripts:

Perl:

open(STDERR,">>","/path/to/my/cgi-error.log")
    or die "Could not redirect STDERR: $OS_ERROR";

Python:

sys.stderr = open("/path/to/my/cgi-error.log", "a")

Apparently in Perl you don't need to close the STDERR handle before reopening it.

Normally I would close it anyway as a best practice, but as I said in the question, I have 269 scripts and I'm trying to minimize the changes. (Plus it seems more Perlish to just re-open the open filehandle, as awful as that sounds.)

In case anyone else has something similar in the future, here's what I'm going to do for updating all my scripts at once:

Perl:

find . -type f -name "*.pl" -exec perl -pi.bak -e 's%/usr/bin/perl%/usr/bin/perl\nopen(STDERR,">>","/path/to/my/cgi-error.log")\n    or die "Could not redirect STDERR: \$OS_ERROR";%' {} \;

Python:

find . -type f -name "*.py" -exec perl -pi.bak -e 's%^(import os, sys.*)%$1\nsys.stderr = open("/path/to/my/cgi-error.log", "a")%' {} \;

The reason I'm posting these commands is that it took me quite a lot of syntactical massaging to get those commands to work (e.g., changing Couldn't to Could not, changing #!/usr/bin/perl to just /usr/bin/perl so the shell wouldn't interpret ! as a history character, using $OS_ERROR instead of $!, etc.)

Thanks to everyone who commented. Since no one answered for both Perl and Python I couldn't really "accept" any of the given answers, but I did give votes to the ones which led me in the right direction. Thanks again!