python csv reader - convert string to int on the for line when iterating

Question

I'm interested in not having to write map the int function to the tuple of strings where I currently have it. See the last part of my example:

import os
import csv

filepath =  os.path.normpath("c:/temp/test.csv")

individualFile = open(filepath,'rb')
dialect = csv.Sniffer().sniff(individualFile.read(1000))

individualFile.seek(0)
reader = csv.reader(individualFile,dialect)

names = reader.next()

print names

def buildTree(arityList):
    if arityList == []:
        return 0            
    else:
        tree = {}
        for i in xrange(arityList[0][0],arityList[0][1]+1):
            tree[i] = buildTree(arityList[1:])
        return tree


census = buildTree([(1,12),(1,6),(1,4),(1,2),(0,85),(0,14)])

for m, f, s, g, a, c, t in reader:
    try:
        m,f,s,g,a,c,t = map(int,(m,f,s,g,a,c,t))
        census[m][f][s][g][a][c] += t
    except:
        print "error"
        print m, f, s, g, a, c, t
        break   

What I want to do is something like this:

for m, f, s, g, a, c, t in map(int,reader):
    try:
        census[m][f][s][g][a][c] += t
    except:
        print "error"
        print m, f, s, g, a, c, t
        break    

I try this and I get the following error:

TypeError: int() argument must be a string or a number, not 'list'

I'm having trouble understand this error message. I thought reader was an iterable object - not a list. It returns a list for each iteration, but it itself is not a list, correct? I guess that is more of a side question. What I really want to know is if there is a way to do what I am trying to do. Sorry for the code that doesn't really relate, but I thought I would include my whole example. Feel free to tear it to bits! :) I'm wondering if it might be better to just have one dict where the key is a tuple instead of this nested dictionary stuff, but even so, I'm still interested in figuring out my question.

Answer 1

what you want is something like:

def int_wrapper(reader):
    for v in reader:
        yield map(int, v)

Your code would then look like:

reader = csv.reader(individualFile,dialect)
reader = int_wrapper(reader)

# all that other stuff

for m, f, s, g, a, c, t in reader:
    try:
        census[m][f][s][g][a][c] += t
    except:
        print "error"
        print m, f, s, g, a, c, t
        break    

This is just using a generator function to wrap the reader and convert the input to integers.

The origin of the TypeError is that reader is a generator function which yields lists of values. When you apply map to it, you are applying map to a 'list' of lists. This is different than applying map to a list of values which you do when you write it out the long way.

For illustration, another way to do it is

for m, f, s, g, a, c, t in (map(int, v) for v in reader):
    # code

This just uses an in situ generator expression instead of defining a function. It's a matter of taste.