tags:

views:

62

answers:

2

I'm parsing a file and need to keep track of where I am in the file... let's say I have file test.txt and I'm doing a while loop that reads in data constantly as each line is written to the file. In case of a crash I'm marking my position in another file with the tell() method of the file. Is there a way to mark the line and be able to go back to that line position or do I need to keep track of the bytes with tell only?

#this is just example code, not the real thing
f = open('test.txt')
pos = open('pos.txt', 'w')
f.seek(pos)
while 1:
  readline(f)
  pos.write(f.tell())

Update: the files are around 1GB each

+1  A: 

Keep track of what f.tell() reports (usually the number of bytes, but not necessarily), then you can jump directly to the right position in the file.

If you only keep track of the number of lines, you will have to read the file again from the beginning, counting the number of new lines seen.

Mark Byers
It is better to use whatever `f.tell()` returns that might or might not correlate with number of bytes read (the point is the argument for `f.seek()` should be value returned by `f.tell()`).
J.F. Sebastian
@J.F.Sebastian: OK, updated.
Mark Byers
+2  A: 

You may like this better: http://docs.python.org/library/linecache.html

"In case of a crash I'm marking my position in another file with the tell() method of the file."

Good.

"Is there a way to mark the line and be able to go back to that line position"

That's what you are doing. You're marking the line with tell. The tell value is the position used by seek.

If it helps, pretend you don't know what the units are. Pretend that tell is just a "magic" key that finds the proper line.

S.Lott
I'm not sure that `linecache` is optimized for 1GB text files. It is used on .py files which are small text files.
J.F. Sebastian
@J.F. Sebastian: Good point. It makes the simple, obvious `tell` solution look even better.
S.Lott