Easy way to find "not enough arguments..." in python logging library.

Question

Do you know any easy way to find a logging call that throws "not enough argumenst for format string". On my workstation I've modified logging/__init__.py to print the msg so I can easily find the line in the source.

But do you have any idea what to do on the testing environment where you can't change python standard library nor run pdb easily?

Note: The traceback is meaningless, and it is caugth by the logging library. Here is an traceback:

  File "/usr/lib/python2.6/logging/handlers.py", line 71, in emit
    if self.shouldRollover(record):
  File "/usr/lib/python2.6/logging/handlers.py", line 144, in shouldRollover
    msg = "%s\n" % self.format(record)
  File "/usr/lib/python2.6/logging/__init__.py", line 648, in format
    return fmt.format(record)
  File "/usr/lib/python2.6/logging/__init__.py", line 436, in format
    record.message = record.getMessage()
  File "/usr/lib/python2.6/logging/__init__.py", line 306, in getMessage
    msg = msg % self.args
TypeError: not enough arguments for format string

And here is the code in standard library that catch the error

    try:
        if self.shouldRollover(record):
            self.doRollover()
        logging.FileHandler.emit(self, record)
    except (KeyboardInterrupt, SystemExit):
        raise
    except:
        self.handleError(record)

Solution as suggested by Alex: I've wrapped the getMessage to print the msg and args. Here is the code:

def print_log_record_on_error(func):
    def wrap(self, *args, **kwargs):
        try:
            return func(self, *args, **kwargs)
        except:
            import sys
            print >>sys.stderr, "Unable to create log message msg=%r, args=%r " % (
                                getattr(self, 'msg', '?'), getattr(self, 'args', '?'))
            raise
    return wrap
import logging
logging.LogRecord.getMessage = print_log_record_on_error(logging.LogRecord.getMessage)

Answer 1

Generally, the best way to catch an exception (including the one you mention) is to put the suspect code in the try clause of a try/except statement; in the except clause you can use the traceback module or other ways to pinpoint the errant statement.

Otherwise-uncaught exceptions end up in sys.excepthook, which is another way you can use to get a traceback if needed.

However, I don't think the logging module has an architected way to tell it "let exceptions propagate" -- in which case your best bet, dirty as it may be, is probably monkey-patching. In other words, you generally can "change the Python standard library" at runtime -- by setting identifiers therein to your own functions / classes / etc wrapping the standard ones. For example, if you know the problem is with a logging.warning call, a "proof of concept" monkeypatch might be:

import logging, traceback

orgwarn = logging.warning
def mywarn(msg, *a):
  try: res = msg % a
  except TypeError:
    traceback.print_exc()
  return orgwarn(msg, *a)
logging.warning = mywarn

There are cleaner ways, but they may be somewhat cumbersome (e.g., make your own logger class and set it as the root logger). Monkey-patching is not a good way to operate in a production environment (it's fragile to system upgrades, can be hard to understand and maintain, etc), but it can be acceptable for debugging and testing purposes.

Answer 2

To find a logging call that throws "not enough argumenst for format string".

I grep the source for all logging calls. There aren't that many: .info(, .warn(, .debug(, .error(, .critical(, .exception(, .log(.

One of those methods has a format without enough arguments. It's generally easy to spot, since the message will have "%" formatting operators.

If your application is consistent in calling all loggers logger is makes it even easier, since you can more simply grep for logger..