Python: Find identical items in multiple lists

Question

I have a list of an arbitrary number of lists, for instance:

[[1,2,3], [3,4,5], [5,6,7], [7,8,9]]

Now I would like a list containing all elements that are present in more than one list:

[3,5,7]

How would I do that?

Thanks!

Answer 1

flatten, sort, 1 for loop comparing numbers before and after

Answer 2

The same way as you'd do it by hand:

seen = set()
repeated = set()
for l in list_of_lists:
  for i in set(l):
    if i in seen:
      repeated.add(i)
    else:
      seen.add(i)

By the way, here's the one liner (without counting the import) that some people were seeking (should be less efficient than the other approach)

from itertools import *
reduce(set.union, (starmap(set.intersection, combinations(map(set, ll), 2))))

Answer 3

You can use a set see http://docs.python.org/library/stdtypes.html#set

Answer 4

You can use a dictionary to get the count of each

from collections import defaultdict

init_list = [[1,2,3], [3,4,5], [5,6,7], [7,8,9]]
#defaultdict, every new key will have a int(0) as default value
d = defaultdict(int)
for values in init_list:
  #Transform each list in a set to avoid false positives like [[1,1],[2,2]]
  for v in set(values):
    d[v] += 1

#Get only the ones that are more than once
final_list = [ value for value,number in d.items() if number > 1 ]

Answer 5

l=[[1,2,3], [3,4,5], [5,6,7], [7,8,9]]
d={}
for x in l:
    for y in x:
        if not d.has_key(y):
            d[y]=0
        d[y]+=1
[x for x,y in d.iteritems() if y>1]

Answer 6

reference: http://docs.python.org/library/stdtypes.html#set

#!/usr/bin/python

ll = [[1,2,3], [3,4,5], [5,6,7], [7,8,9]]
ls = [set(l) for l in ll]

su = ls[0]  #union
ssd = ls[0] #symmetric_difference
for s in ls[1:]:
  su = su.union(s)
  ssd = ssd.symmetric_difference(s)

result = su.difference(ssd)
print list(result)

=>

[3, 5, 7]

revise and adopt FP,

ll = [[1,2,3], [3,4,5], [5,6,7], [7,8,9]]

u = reduce(set.union, map(set, ll))
sd = reduce(set.symmetric_difference, map(set, ll))
print u - sd

=>

[3, 5, 7]

Answer 7

Try this:

data = [[1,2,3], [3,4,5], [5,6,7], [7,8,9], [1,2,3]]

res = set()

for i in data:
    for j in data:
        if i is not j:
            res |= set(i) & set(j)

print res

Answer 8

Cleanest way would probably be to use reduce:

def findCommon(L):
    def R(a, b, seen=set()):
        a.update(b & seen)
        seen.update(b)
        return a
    return reduce(R, map(set, L), set())

result = findCommon([[1,2,3], [3,4,5], [5,6,7], [7,8,9]])

Result is a set, but just do list(result) if you really need a list.

Answer 9

>>> sets = [[1,2,3], [3,4,5], [5,6,7], [7,8,9]]
>>> seen = set()
>>> duplicates = set()
>>> 
>>> for subset in map(set, sets) :
...     duplicates |= (subset & seen)
...     seen |= subset
... 
>>> print(duplicates)
set([3, 5, 7])
>>> 

I tried for a one-line answer with map/reduce, but can't quite get it yet.

Answer 10

Here is my go:

seen = set()
result = set()
for s in map(set, [[1,2,3], [3,4,5], [5,6,7], [7,8,9]]):
    result.update(s & seen)
    seen.update(s)
print result

This prints:

set([3, 5, 7])