Comparing multiple dictionaries in Python

Question

I'm new to Python and am running to a problem I can't google my way out of. I've built a GUI using wxPython and ObjectiveListView. In its very center, the GUI has a list control displaying data in X rows (the data is loaded by the user) and in five columns.

When the user selects multiple entries from the list control (pressing CTRL or shift while clicking), the ObjectiveListView module gives me a list of dictionaries, the dictionaries containing the data in the rows of the list control. This is exactly what I want, good!

The returned list looks something like this:

print MyList
[{'id':1023, 'type':'Purchase', 'date':'23.8.2008', 'sum':'-21,90', 'target':'Apple Store'}, {'id':1024, 'type':'Purchase', 'date':'24.8.2008', 'sum':'-21,90', 'target':'Apple Store'}, {'id':23, 'type':'Purchase', 'date':'2.8.2008', 'sum':'-21,90', 'target':'Apple Store'}]

All the dictionaries have the same keys, but the values change. The 'id' value is unique. Here the problems start. I want to get the common values for all the items the user selected. In the above list they would be 'sum':'-21,90' and 'target':'Apple Store'.

I don't know how to properly compare the dicts in the list. One big problem is that I don't know beforehand how many dicts the list contains, since it's decided by the user.

I have a vague idea that list comprehensions would be the way to go, but I only know how to compare two lists with list comprehensions, not n lists. Any help would be appreciated.

Answer 1

Sorry, yes, 'type':'Purchase' is also one of the common values.Should have logged in to edit the question.

Answer 2

>>> mysets = (set(x.items()) for x in MyList)
>>> reduce(lambda a,b: a.intersection(b), mysets)
set([('sum', '-21,90'), ('type', 'Purchase'), ('target', 'Apple Store')])

First, I've created a generator that will convert the list of dicts into an iterable sequence of sets of key,value pairs. You could use a list comprehension here but this way doesn't convert your entire list into yet another list, useful if you don't know how big it will be.

Then I've used reduce to apply a function that finds the common values between each set. It finds the intersection of set 1 & set 2, which is itself a set, then the intersection of that set & set 3 etc. The mysets generator will happily feed each set on demand to the reduce function until its done.

I believe reduce has been deprecated as a built-in in Python 3.0, but should still be available in functools.

You could of course make it a one-liner by replacing mysets in the reduce with the generator expression, but that reduces the readability IMO. In practice I'd probably even go one step further and break the lambda out into its own line as well:

>>> mysets = (set(x.items()) for x in MyList)
>>> find_common = lambda a,b: a.intersection(b)
>>> reduce(find_common, mysets)
set([('sum', '-21,90'), ('type', 'Purchase'), ('target', 'Apple Store')])

And if you need the end result to be a dict, just wrap it like so:

>>> dict(reduce(find_common, mysets))
{'sum': '-21,90', 'type': 'Purchase', 'target': 'Apple Store'}

dict can accept any iterator of key,value pairs, such as the set of tuples returned at the end.

Answer 3

First, we need a function to compute intersection of two dictionaries:

def IntersectDicts( d1, d2 ) :
    return dict(filter(lambda (k,v) : k in d2 and d2[k] == v, d1.items()))

Then we can use it to process any number of dictionaries:

result = reduce(IntersectDicts, MyList)

Answer 4

My answer is identical to Matthew Trevor's, except for one difference:

>>> mysets = (set(x.items()) for x in MyList)
>>> reduce(set.intersection, mysets)
set([('sum', '-21,90'), ('type', 'Purchase'), ('target', 'Apple Store')])

Here I use set.intersection instead of creating a new lambda. In my opinion this is more readable, as this intuitively reads as "reduce is reducing this list using the set intersection operator." This should also be much faster, as set.intersection is a built-in C function.

To fully answer your question, you can extract the values using a list comprehension:

>>> mysets = (set(x.items()) for x in MyList)
>>> result = reduce(set.intersection, mysets)
>>> values = [r[1] for r in result]
>>> values
['-21,90', 'Purchase', 'Apple Store']

This would end up on one line for me. but that's entirely up to you:

>>> [r[1] for r in reduce(set.intersection, (set(x.items()) for x in myList))]
['-21,90', 'Purchase', 'Apple Store']

Answer 5

Since you're only looking for the common set, you can compare the keys in the first dictionary to the keys in all other dictionaries:

common = {}
for k in MyList[0]:
    for i in xrange(1,len(MyList)):
        if MyList[0][k] != MyList[i][k]: continue
        common[k] = MyList[0][k]

>>> common
{'sum': '-21,90', 'type': 'Purchase', 'target': 'Apple Store'}