Most pythonic form for mapping a series of statements?

Question

This is something that has bugged me for some time. I learnt Haskell before I learnt Python, so I've always been fond of thinking of many computations as a mapping onto a list. This is beautifully expressed by a list comprehension (I'm giving the pythonic version here):

result = [ f(x) for x in list ]

In many cases though, we want to execute more than a single statement on x, say:

result = [ f(g(h(x))) for x in list ]

This very quickly gets clunky, and difficult to read.

My normal solution to this is to expand this back into a for loop:

result = []
for x in list:
  x0 = h(x)
  x1 = g(x0)
  x2 = f(x1)
  result.append(x2)

One thing about this that bothers me no end is having to initialize the empty list 'result'. It's a triviality, but it makes me unhappy. I was wondering if there were any alternative equivalent forms. One way may be to use a local function(is that what they're called in Python?)

def operation(x):
  x0 = h(x)
  x1 = g(x0)
  x2 = f(x1)
  return x2
result = [ operation(x) for x in list ]

Are there any particular advantages/disadvantages to either of the two forms above? Or is there perhaps a more elegant way?

Answer 1

There are cases where it's best to go back to the for-loop, yes, but more often I prefer one of these approaches:

Use appropriate line breaks and indentation to keep it readable:

result = [blah(blah(blah(x)))
          for x in list]

Or extract (enough of) the logic into another function, as you mention. But not necessarily local; Python programmers prefer flat to nested structure, if you can see a reasonable way of factoring the functionality out.

I came to Python from the functional-programming world, too, and share your prejudice.

Answer 2

If your only concerned with the last result, your last answer is the best. It's clear for anyone looking at it what your doing.

I often take any code that starts to get complex and move it to a function. This basically serves as a comment for that block of code. (any complex code probably needs a re-write anyway, and putting it in a function I can go back and work on it later)

def operation(x):
  x0 = h(x)
  x1 = g(x0)
  x2 = f(x1)
  return x2
result = [ operation(x) for x in list]

Answer 3

Follow the style that most matches your tastes.
I would not worry about performance; only in case you really see some issue you can try to move to a different style.

Here some other possible suggestions, in addition to your proposals:

result = [f(
              g(
                h(x)
                )
              )
            for x in list]

Use progressive list comprehensions:

result = [h(x) for x in list]
result = [g(x) for x in result]
result = [f(x) for x in result]

Again, that's only a matter of style and taste. Pick the one you prefer most, and stick with it :-)

Answer 4

You can easily do function composition in Python.

Here's a demonstrates of a way to create a new function which is a composition of existing functions.

>>> def comp( a, b ):
    def compose( args ):
     return a( b( args ) )
    return compose

>>> def times2(x): return x*2

>>> def plus1(x): return x+1

>>> comp( times2, plus1 )(32)
66

Here's a more complete recipe for function composition. This should make it look less clunky.

Answer 5

If this is something you're doing often and with several different statements you could write something like

def seriesoffncs(fncs,x):
    for f in fncs[::-1]:
        x=f(x)
    return x

where fncs is a list of functions. so seriesoffncs((f,g,h),x) would return f(g(h(x))). This way if you later in your code need to workout h(q(g(f(x)))) you would simply do seriesoffncs((h,q,g,f),x) rather than make a new operations function for each combination of functions.

Answer 6

A variation of dagw.myopenid.com's function:

def chained_apply(*args):
    val = args[-1]
    for f in fncs[:-1:-1]:
        val=f(val)
    return val

Instead of seriesoffncs((h,q,g,f),x) now you can call:

result = chained_apply(foo, bar, baz, x)

Answer 7

As far as I know there's no built-in/native syntax for composition in Python, but you can write your own function to compose stuff without too much trouble.

def compose(*f):
    return f[0] if len(f) == 1 else lambda *args: f[0](compose(*f[1:])(*args))

def f(x): 
    return 'o ' + str(x)

def g(x): 
    return 'hai ' + str(x)

def h(x, y): 
    return 'there ' + str(x) + str(y) + '\n'

action = compose(f, g, h)
print [action("Test ", item) for item in [1, 2, 3]]

Composing outside the comprehension isn't required, of course.

print [compose(f, g, h)("Test ", item) for item in [1, 2, 3]]

This way of composing will work for any number of functions (well, up to the recursion limit) with any number of parameters for the inner function.