Chained string formatting in Python

Question

In handling a little SQL formatting I was amazed to find I could chain string formatters:

def get_sql(table, limit=True):
    sql = "select report_date from %s"
    if limit:
        result = "%s limit 1" % sql % table
    else:
        result = sql % table
    return result

Is this legit? Any reason not to do this?

Answer 1

It's perfectly legit.

The "single argument" form of the string formatter is really a special case - for multiple items a tuple is normally used and that would lead to a more obvious example of why it's ok

result = "%s limit 1" % (sql % (table,),)

This ^ was originally written to encourage the questioner that supporting multiple-formats was a legitimate language feature but, as Nas Banov comments, it does read like I'm trying to explain how it works (not helped by screwing up the code). It doesn't build the string right to left like this suggests it might, but HAS to build it (be associative) left to right. The operator must take a string on the left and return one, but can take a non-string (a tuple) on the right. Since you can't use % on a pair of tuples it can't possibly work in reverse

>>> "%s %f %s" % ( "%d", 0.1, "%d %d" ) % (1,2,3)
'1 0.100000 2 3'

It may, however, lead to complicated/messy code, so personally I would use it very sparingly.

You could work your example like this:

def get_sql(table, limit=True):
    sql = "select report_date from"
    strlimit = ""
    if limit:
        strlimit = "limit 1"
    return "%s %s"%(sql, strlimit)

Answer 2

It makes sense that it works because a statement like this:

'some value goes here %s' % value

Actually returns a string. It's probably a bit more logical to view it like this:

result = ("%s limit 1" % sql) % table

There's nothing expressly wrong with doing that, but chaining operators can lead to problems with figuring out where an error came from.

So for instance, this works fine:

>>> sql = 'a value of %s'
>>> x = 'some string %s with stuff'
>>> y = 'VALUE'
>>> x % sql % y
'some string a value of VALUE with stuff'

But if there was a formatting error in there (I realize this example is pathological, but it gets the point across):

>>> sql = 'a value of %d'
>>> x = 'some string %d with stuff'
>>> y = 123    
>>> x % sql % y
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: %d format: a number is required, not str

It's really not clear which %d is causing your error. For that reason, I would split it out and just use one % formatter per line if possible because then the traceback will be able to point you to exactly which line and which formatter had the issue.

For the record, by doing it one formatter per line you'll also make life a lot easier for anyone else who has to read your code and try to figure out what's going on.

Answer 3

I would write it this way:

def get_sql(table,limit=True):
    sql = "select report_date from %s"%table
    if limit: sql += " limit 1"
    return sql

PS. The code you posted works like this:

In [49]: "%s limit 1" % sql
Out[49]: 'select report_date from %s limit 1'

In [50]: "%s limit 1" % sql % 'table'
Out[50]: 'select report_date from table limit 1'

Sure you can do it, but I don't think it is particularly clear.

Answer 4

Why yes, it is possible to chain %string formatting like that, even if this is the first time i see it used (and in horrible way, mind you)!

The reason is that operators of the same type group left to right (have "left associativity" - with the notable exceptions of exponentiation ** and comparisons a<b<c).

So in the same way that

>>> 1 - 2 - 3    # equals to (1-2)-3
-4
>>> 16 / 4 / 2   # equals to (16 /4) /2,  NOT 16 / (4 / 2)
2

so does s1 % s2 % s3 equal to (s1 % s2) % s3

Oh and by the way, it does not matter if s1, s2 and s3 are strings or numbers - the compiler does not know that during compilation and only runtime it will be determined that whether % means "remainder from division" (if s1 is number) or string formatting (if s1 is a string).