item frequency count in python

Question

Hi, I'm a python newbie, so maybe my question is very noob. Assume I have a list of words, and I want to find the number of times each word appears in that list. Obvious way to do this is:

words = "apple banana apple strawberry banana lemon"
uniques = set(words.split())
freqs = [(item, words.split.count(item)) for item in uniques]
print(freqs)

But I find this code not very good, because this way program runs through words list twice, once to build the set, and second time counting the number of appearances. Of course, I could write a function to run through list and do the counting, but that wouldn't be so pythonic. So, is there a more efficient and pythonic way?

Answer 1

If you don't want to use the standard dictionary method (looping through the list incrementing the proper dict. key), you can try this:

>>> from itertools import groupby
>>> myList = words.split() # ['apple', 'banana', 'apple', 'strawberry', 'banana', 'lemon']
>>> [(k, len(list(g))) for k, g in groupby(sorted(myList))]
[('apple', 2), ('banana', 2), ('lemon', 1), ('strawberry', 1)]

It runs in O(n log n) time.

Answer 2

defaultdict to the rescue!

from collections import defaultdict

words = "apple banana apple strawberry banana lemon"

d = defaultdict(int)
for word in words.split():
    d[word] += 1

This runs in O(n).

Answer 3

Standard approach:

from collections import defaultdict

words = "apple banana apple strawberry banana lemon"
words = words.split()
result = collections.defaultdict(int)
for word in words:
    result[word] += 1

print result

Groupby oneliner:

from itertools import groupby

words = "apple banana apple strawberry banana lemon"
words = words.split()

result = dict((key, len(list(group))) for key, group in groupby(sorted(words)))
print result

Answer 4

If you are using python 2.7+/3.1+, there is a Counter Class in the collections module which is purpose built to solve this type of problem:

>>> from collections import Counter
>>> words = "apple banana apple strawberry banana lemon"
>>> freqs = Counter(words.split())
>>> print(freqs)
Counter({'apple': 2, 'banana': 2, 'strawberry': 1, 'lemon': 1})
>>>

Since both 2.7 and 3.1 are still in beta it's unlikely you're using it, so just keep in mind that a standard way of doing this kind of work will soon be readily available.

Answer 5

Without defaultdict:

words = "apple banana apple strawberry banana lemon"
my_count = {}
for word in words.split():
    try: my_count[word] += 1
    except KeyError: my_count[word] = 1

Answer 6

freqs = {}
for word in words:
    freqs[word] = freqs.get(word, 0) + 1 # fetch and increment OR initialize

I think this results to the same as Triptych's solution, but without importing collections. Also a bit like Selinap's solution, but more readable imho. Almost identical to Thomas Weigel's solution, but without using Exceptions.

This could be slower than using defaultdict() from the collections library however. Since the value is fetched, incremented and then assigned again. Instead of just incremented. However using += might do just the same internally.