How to Use python map and other functional tools

Question

This is quite n00bish, but I'm trying to learn/understand functional programming in python. The following code:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo, bar):
    print foo, bar

map(maptest, foos, bars)

produces:

1.0 1
2.0 2
3.0 3
4.0 None
5.0 None

Q. Is there a way to use map or any other functional tools in python to produce the following without loops etc.

1.0 [1,2,3]
2.0 [1,2,3]
3.0 [1,2,3]
4.0 [1,2,3]
5.0 [1,2,3]

Just as a side note how would the implementation change if there is a dependency between foo and bar. e.g.

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3,4,5]

and print:

1.0 [2,3,4,5]
2.0 [1,3,4,5]
3.0 [1,2,4,5]
...

**P.S: I know how to do it naively using if, loops and/or generators, but I'd like to learn how to achieve the same using functional tools. Is it just a case of adding an if statement to maptest or apply another filter map to bars internally within maptest? **

Answer 1

Would this do it?

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest2(bar):
  print bar

def maptest(foo):
  print foo
  map(maptest2, bars)

map(maptest, foos)

Answer 2

Here's the solution you're looking for:

>>> foos = [1.0, 2.0, 3.0, 4.0, 5.0]
>>> bars = [1, 2, 3]
>>> [(x, bars) for x in foos]
[(1.0, [1, 2, 3]), (2.0, [1, 2, 3]), (3.0, [1, 2, 3]), (4.0, [1, 2, 3]), (5.0, [
1, 2, 3])]

I'd recommend using a list comprehension (the [(x, bars) for x in foos] part) over using map as it avoids the overhead of a function call on every iteration (which can be very significant). If you're just going to use it in a for loop, you'll get better speeds by using a generator comprehension:

>>> y = ((x, bars) for x in foos)
>>> for z in y:
...     print z
...
(1.0, [1, 2, 3])
(2.0, [1, 2, 3])
(3.0, [1, 2, 3])
(4.0, [1, 2, 3])
(5.0, [1, 2, 3])

The difference is that the generator comprehension is lazily loaded.

UPDATE In response to this comment:

Of course you know, that you don't copy bars, all entries are the same bars list. So if you modify any one of them (including original bars), you modify all of them.

I suppose this is a valid point. There are two solutions to this that I can think of. The most efficient is probably something like this:

tbars = tuple(bars)
[(x, tbars) for x in foos]

Since tuples are immutable, this will prevent bars from being modified through the results of this list comprehension (or generator comprehension if you go that route). If you really need to modify each and every one of the results, you can do this:

from copy import copy
[(x, copy(bars)) for x in foos]

However, this can be a bit expensive both in terms of memory usage and in speed, so I'd recommend against it unless you really need to add to each one of them.

Answer 3

The easiest way would be not to pass bars through the different functions, but to access it directly from maptest:

foos = [1.0,2.0,3.0,4.0,5.0]
bars = [1,2,3]

def maptest(foo):
    print foo, bars

map(maptest, foos)

With your original maptest function you could also use a lambda function in map:

map((lambda foo: maptest(foo, bars)), foos)

Answer 4

import itertools

foos=[1.0, 2.0, 3.0, 4.0, 5.0]
bars=[1, 2, 3]

print zip(foos, itertools.cycle([bars]))

Answer 5

>>> from itertools import repeat
>>> for foo, bars in zip(foos, repeat(bars)):
...     print foo, bars
... 
1.0 [1, 2, 3]
2.0 [1, 2, 3]
3.0 [1, 2, 3]
4.0 [1, 2, 3]
5.0 [1, 2, 3]

Answer 6

Here's an overview of the parameters to the map(function, *sequences) function:

• function is the name of your function.
• sequences is any number of sequences, which are usually lists or tuples. map will iterate over them simultaneously and give the current values to function. That's why the number of sequences should equal the number of parameters to your function.

It sounds like you're trying to iterate for some of function's parameters but keep others constant, and unfortunately map doesn't support that. I found an old proposal to add such a feature to Python, but the map construct is so clean and well-established that I doubt something like that will ever be implemented.

Use a workaround like global variables or list comprehensions, as others have suggested.

Answer 7

Are you familiar with other functional languages? i.e. are you trying to learn how python does functional programming, or are you trying to learn about functional programming and using python as the vehicle?

Also, do you understand list comprehensions?

map(f, sequence)

is directly equivalent (*) to:

[f(x) for x in sequence]

In fact, I think map() was once slated for removal from python 3.0 as being redundant (that didn't happen).

map(f, sequence1, sequence2)

is mostly equivalent to:

[f(x1, x2) for x1, x2 in zip(sequence1, sequence2)]

(there is a difference in how it handles the case where the sequences are of different length. As you saw, map() fills in None when one of the sequences runs out, whereas zip() stops when the shortest sequence stops)

So, to address your specific question, you're trying to produce the result:

foos[0], bars
foos[1], bars
foos[2], bars
# etc.

You could do this by writing a function that takes a single argument and prints it, followed by bars:

def maptest(x):
     print x, bars
map(maptest, foos)

Alternatively, you could create a list that looks like this:

[bars, bars, bars, ] # etc.

and use your original maptest:

def maptest(x, y):
    print x, y

One way to do this would be to explicitely build the list beforehand:

barses = [bars] * len(foos)
map(maptest, foos, barses)

Alternatively, you could pull in the itertools module. itertools contains many clever functions that help you do functional-style lazy-evaluation programming in python. In this case, we want itertools.repeat, which will output its argument indefinitely as you iterate over it. This last fact means that if you do:

map(maptest, foos, itertools.repeat(bars))

you will get endless output, since map() keeps going as long as one of the arguments is still producing output. However, itertools.imap is just like map(), but stops as soon as the shortest iterable stops.

itertools.imap(maptest, foos, itertools.repeat(bars))

Hope this helps :-)

(*) It's a little different in python 3.0. There, map() essentially returns a generator expression.

Answer 8

Functional programming is about creating side-effect-free code.

map is a functional list transformation abstraction. You use it to take a sequence of something and turn it into a sequence of something else.

You are trying to use it as an iterator. Don't do that. :)

Here is an example of how you might use map to build the list you want. There are shorter solutions (I'd just use comprehensions), but this will help you understand what map does a bit better:

def my_transform_function(input):
    return [input, [1, 2, 3]]

new_list = map(my_transform, input_list)

Notice at this point, you've only done a data manipulation. Now you can print it:

for n,l in new_list:
    print n, ll

-- I'm not sure what you mean by 'without loops.' fp isn't about avoiding loops (you can't examine every item in a list without visiting each one). It's about avoiding side-effects, thus writing fewer bugs.