Unexpected results feeding Django File upload object to Python CSV module

Question

I have no problems getting the file to upload and if I save it to disk, all formatting is intact.

I wrote a function to read in the the file within Django using:

data = csv.reader(f.read())

where f is the Django file object that I get from 'form.cleaned_data['file']' and yes the file is already bound to the form.

When I try to read the file using

for row in data:
    logging.debug(row)

I get an unexpected result in that it appears to be producing small packs of the data almost as if its reading some buffer. for example, for my float fields I get this when I log each row:

['0'] ['.'] ['0'] ['5']['', ''] ['0'] ['.'] ['2'] etc etc... where each item between the square bracket is actually from one row (ie. a newline)

csv.reader requires the object it takes to support the iterator protocol which I believe the Django File object does

So what am I doing wrong?

Answer 1

What happens if you try reading in the whole file, logging its first few bytes, and then passing it into the parser? Perhaps the file isn't being uploaded as you expect? An http form-encoding issue, perhaps?

Answer 2

You're actually passing the wrong iterable to csv.reader(). Try changing that line to:

data = csv.reader(f)

What you're doing is passing the whole contents of the file to the csv.reader() function, which will cause it to iterate over every individual character, treating each of them as a separate line. If you pass the actual file object to the function, it will iterate over the lines in that file, as you expect.