Using Python Reg Exp to read data from file

Question

I'm having trouble using python reg exp to read data from a file.

The file has data I want and some info I'm not interested in. An example of the info I'm interested in is below. The number of rows will vary

FREQ VM(VOUT)        

1.000E+00  4.760E+01

1.002E+00  4.749E+01
Y

I want to create a list of tuples like:

[(1.000, 47.6),(1.002, 47.49)]

I'm trying to read the file till I find the 'FREQ VM(VOUT)' line and read the data points till I hit the 'Y'.

I have 2 questions:

1. Is it possible to get all the points with one expression or do I need to loop through each line and look for the start end? I can't seem to get the the reg exp to work when I try to find the section and read the points in a single expression.
2. How do I parse a number that is in engineering notation?

I couldn't find an example that was very close to what I'm doing. If it is out there, please point me to it.

Answer 1

I think this will get you what you want. As long as the file is consistent.

from csv import reader
with open('file') as f:
  listoftuples = [(float(row[0]), float(row[1])) 
                  for row in reader(f, delimiter='  ') 
                  if row and row[0] != 'FREQ']

If you want it to break at 'Y', then do this less elegant thingy:

from csv import reader
l = []
with open('file') as f:
  for row in reader(f, delimiter='  '):
    if row[0] == 'Y':
      break
    if row and row[0] != 'FREQ':
      l.append((floar(row[0]), float(row[1])))

Answer 2

Not nearly as elegant as Tor's answer. No regexps either. Bring on the downvotes!

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import decimal
import os

def main():
    we_care = False # start off not caring 

    list_of_tuples = []

    f = open('test.txt','r')

    for line in f:
        if line.startswith('FREQ'):
            we_care = True # we found what we want; now we care
            continue
        if we_care:
            try:
                x,y = (decimal.Decimal(x) 
                        for x in line.rstrip(os.linesep).split())
                list_of_tuples.append((x,y))
            except ValueError:
                pass # we get here when a line doesn't contain two floats
            except decimal.InvalidOperation:
                pass # we get here when a line contains a non-decimal
            if line.startswith('Y'):
                break # break out of processing once you've got your data
    return list_of_tuples

if __name__ == "__main__":
    print main()

Returns:

[(Decimal('1.000'), Decimal('47.60')), (Decimal('1.002'), Decimal('47.49'))]

Answer 3

import decimal
flag=0
result=[]
for line in open("file"):
    line=line.rstrip()
    if line == "Y": flag=0
    if line.startswith("FREQ VM"):
         flag=1
         continue
    if flag and line:
         result.append(map(decimal.Decimal,line.split()))
print result