views:

461

answers:

8

What's the best Python equivalent of Common Lisp's maplist function? From the maplist documentation:

maplist is like mapcar except that function is applied to successive sublists of the lists. function is first applied to the lists themselves, and then to the cdr of each list, and then to the cdr of the cdr of each list, and so on.

Example (pseudoy-code, not tested):

>>> def p(x): return x
>>> maplist(p, [1,2,3])
[[1, 2, 3], [2, 3], [3]]

Note: the arguments passed to p in the example above would be the lists [1, 2, 3], [2, 3], [3]; i.e., p is not applied to the elements of those lists. E.g.:

>>> maplist(lambda l: list(reversed(l)), [1,2,3])
[[3, 2, 1], [3, 2], [3]]
+10  A: 

You can write a little function for that

def maplist(func, values):
    return [map(func, values[i:]) for i in xrange(len(values))]

>>> maplist(lambda a: a* 2, [1,2,3])
[[2, 4, 6], [4, 6], [6]]

[Edit]

if you want to apply the function on the sublists you can change the function to this:

def maplist(func, values):
    return [func(values[i:]) for i in xrange(len(values))]

>>> maplist(lambda l: list(reversed(l)), [1,2,3])
[[3, 2, 1], [3, 2], [3]]
Nadia Alramli
You can't slice iterators. So the second parameter is badly named.
Aaron Maenpaa
@Aaron, thanks for pointing that, fixed
Nadia Alramli
I probably chose an ambiguous example in my question, unfortunately; 'maplist' should apply 'func' directly to the sublists [1,2,3], [2,3], [3], not the elements of those sublists.
Jacob Gabrielson
@Jacob, check the edit, I added another function that does what you want.
Nadia Alramli
-1 because that function isn't equivalent. LISP's maplist is a linear time algorithm.
Cybis
Couple of things: This doesn't appear to do what maplist does at all. The examples I've seen of it all show *multiple* lists, and the size of the output is equal to the size of the smallest list. For example: maplist(lambda a: a*2, [[1,2,3], [1,2]]) -> [ [[2,4,6], [2,4]], [[2,4], [2]] ].
Kamil Kisiel
A: 

I think there isn't, but the following function can work:

def maplist(func, l):
    for i in range(len(l)):
        func(l[i:])
Javier Badia
eww, side-effecty :)
offby1
+1  A: 

this works like your example (I've modified reyjavikvi's code)

def maplist(func, l):
    result=[]
    for i in range(len(l)):
        result.append(func(l[i:]))
    return result
Adrian Mester
+1  A: 

You can use nested list comprehensions:

>>> def p(x): return x
>>> l = range(4)[1:]
>>> [p([i:]) for i in range(len(l))]
[[1, 2, 3], [2, 3], [3]]

Which you can use to define maplist yourself:

>>> def maplist(p, l): return [p([i:]) for i in range(len(l))]
>>> maplist(p, l)
[[1, 2, 3], [2, 3], [3]]
Aaron Maenpaa
I probably chose an ambiguous example by just using the identity function; the function "p" should just be passed the sublists themselves, not the sublists' elements. I added a clarification to the question.
Jacob Gabrielson
@Jacob That makes it easier :)
Aaron Maenpaa
A: 

Modifying Aaron's answer:

In [8]: def maplist(p, l): return [p([x for x in l[i:]]) for i in range(len(l))]
   ...: 

In [9]: maplist(lambda x: x + x, [1,2,3])
Out[9]: [[1, 2, 3, 1, 2, 3], [2, 3, 2, 3], [3, 3]]
yangyang
+2  A: 

Eewww... Slicing a list is a linear-time operation. All of the solutions posted thus far have O(n^2) time complexity. Lisp's maplist, I believe, has O(n).

The problem is, Python's list type isn't a linked list. It's a dynamically resizable array (i.e., like C++ STL's "vector" type).

If maintaining linear time complexity is important to you, it isn't possible to create a "maplist" function over Python lists. It would be better to modify your code to work with indices into the list, or convert the list into an actual linked list (still a linear-time operation, but would have a lot of overhead).

Cybis
That is a good point. In the case I was thinking of the lists would be short enough that it wouldn't be a concern. But in the general case it could. Maybe something involving generators would work, too?
Jacob Gabrielson
That's a good idea - if maplist accepted a function which itself accepted generators instead of lists. The implementation would be a bit convoluted, and there would be more overhead than LISP's maplist (from creating all those generators), but I think it could work.
Cybis
I've edited my answer below (hopefully soon above) to add a generator solution
Rick Copeland
+7  A: 

As @Cybis and others mentioned, you can't keep the O(N) complexity with Python lists; you'll have to create a linked list. At the risk of proving Greenspun's 10th rule, here is such a solution:

class cons(tuple):
    __slots__=()

    def __new__(cls, car, cdr):
        return tuple.__new__(cls, (car,cdr))

    @classmethod
    def from_seq(class_, l):
        result = None
        for el in reversed(l):
            result = cons(el, result)
        return result

    @property
    def car(self): return self._getitem(0)

    @property
    def cdr(self): return self._getitem(1)

    def _getitem(self, i):
        return tuple.__getitem__(self, i)

    def __repr__(self):
        return '(%s %r)' % (self.car, self.cdr)

    def __iter__(self):
        v = self
        while v is not None:
            yield v.car
            v = v.cdr

    def __len__(self):
        return sum(1 for x in self)

    def __getitem__(self, i):
        v = self
        while i > 0:
            v = v.cdr
            i -= 1
        return v.car

def maplist(func, values):
    result = [ ]
    while values is not None:
        result.append(func(values))
        values = values.cdr
    return result

Testing yields:

>>> l = cons.from_seq([1,2,3,4])
>>> print l
(1 (2 (3 (4 None))))
>>> print list(l)
[1, 2, 3, 4]
>>> print maplistr(lambda l: list(reversed(l)), cons.from_seq([1,2,3]))
[[3, 2, 1], [3, 2], [3]]

EDIT: Here is a generator-based solution that basically solves the same problem without the use of linked lists:

import itertools

def mapiter(func, iter_):
    while True:
        iter_, iter2 = itertools.tee(iter_)
        iter_.next()
        yield func(iter2)

Testing yields:

>>> print list(mapiter(lambda l: list(reversed(list(l))), [1,2,3]))
[[3, 2, 1], [3, 2], [3]]
Rick Copeland
+1 for showing that not even Python can escape Greenspun's 10th Rule.
Cybis
Your generator solution is giving this error on big lists: RuntimeError: maximum recursion depth exceeded
Nadia Alramli
And is 7 times slower than the regular list comprehension solution when running on a 100 entry list for 1000 times.
Nadia Alramli
@Rick, I tested your both solutions and both of them are very slow even for large lists. The first one is much slower than the second. And the second is 7 times slower than the regular list comprehension solution.
Nadia Alramli
@Nadia, thanks for the comments. I would expect both solutions to be fairly slow (and would expect the mapiter() solution to have a recursion depth problem.) The goal of this answer was simply to replicate the O(N) scaling of maplist, not necessarily to make it faster overall.
Rick Copeland
Most of the speed problems were due to the extravagant use of recursion. I have edited the solution to use iteration instead and now both solutions (linked list and generator-based) should beat list comprehensions on lists of more than 1000 elements.
Rick Copeland
@Rick, are you sure they beat list comprehensions? My results show otherwise. For 10000 elements list: the generator took 0m2.808s while list comprehension took 0m2.083s. I'm not arguing whether O(N) or O(N^2) is better. My point is that there might be hidden python overhead that we should take into account.
Nadia Alramli
The test I ran was using the identity function for "func" (lambda l:l). For my iterator solution, I used a list [None]*int(1e8) and completed 1000 runs in timeit in 0.00134s. When I tried the same for the list comprehension solution, I got a MemoryError, so I reduced the list to [None]*1000 and it still took 9.79s for 1000 runs. My linked list was also a memory hog, so I ran it on the 1000-element list and it completed in 1.4049s. All this is running on Python 2.5.2. To reproduce, here is maplist.py: http://pastebin.com/m54e69b8c and here is my timer script:http://pastebin.com/m7422c943
Rick Copeland
I should mention that all my comparisons are with @Nadia's edited maplist() that applies func on all the sublists, if that makes a difference.
Rick Copeland
I did notice that my iterator timing was off because I didn't actually measure the time to exhaust the returned iterator. Fixing this (http://pastebin.com/m123068a), the generator solution is still faster than list comprehensions at 1.46s/1000 runs on a 1000-element list.
Rick Copeland
+1  A: 

O(N) implementation of maplist() for linked lists

maplist = lambda f, lst: cons(f(lst), maplist(f, cdr(lst))) if lst else lst

See Python Linked List question.

J.F. Sebastian