Reading .csv in Python without looping through the whole file?

Question

Hello,

The only way I've seen Python's csv.reader used is in a for loop, which goes through the whole file without saving past values of the read in variables. I only need to work with 2 consecutive lines of the (enormous) file at a time. Using the csv.reader for loop, I only have 1 line at a time.

Is there a way to use Python's csv module for taking in only one line of a csv file without having to finish reading the file to the end?

I need to set variables to the values in the first line, set a second set of variables to the values of the next line, use the two sets of variables simultaneously for computations, then overwrite the first set of variables with the second set, and read a new line to overwrite the second set.

Thank you in advance for your help!

Answer 1

The obvious answer seems to be to just store the previous line on each iteration.

>>> for x in csv.DictReader(stream):
...   print prevLine
...   print x
...   prevLine = x
....

Answer 2

There's nothing forcing you to use the reader in a loop. Just read the first line, then read the second line.

import csv
r = csv.reader(open("data.csv"))
line1=r.next()
line2=r.next()

Answer 3

Blatant stealing from TK... ...mostly the question that remains is, what does the OP want to do with the first and last lines of the file?

prevLine = None

for x in csv.DictReader(stream):
   if prevLine is not None:
       DoWork(prevLine, x)
   else:
       Initialize(x)
   prevLine = x

Finalize(prevLine)

Answer 4

If you're always looking at exactly two consecutive lines, it sounds to me like you might benefit from using the pairwise recipe. From the itertools module:

from itertools import tee, izip
def pairwise(iterable):
   "s -> (s0,s1), (s1,s2), (s2, s3), ..."
   a, b = tee(iterable)
   next(b, None)
   return izip(a, b)

You would use this like so:

for first_dict, second_dict in pairwise(csv.DictReader(stream)):
    # do stuff with first_dict and second_dict