Python: Import a file and convert to a list

Question

I need help with importing a file and converting each line into a list.

An example of the file would look like:

p wfgh 1111 11111 111111
287 48 0
65626 -1818 0
4654 21512 02020 0

The first line beginning with p is a header and the rest are clauses. Each clause line must begin with a series of at least two integers and finish with a zero

thanks in advance

Answer 1

Hi there,

look at this: Input Output Python Documentation

The Keyword here is "readline".

If you have problems using it...reedit your question

Answer 2

If you want all of the values in a flat list, the code would look as follows:

ls=[]
for line in open( "input.txt", "r" ).readlines():
    for value in line.split( ' ' ):
        ls.append( value )

If you just want the lines in a list then you can stop at readlines().

Answer 3

The following line will create a list where each item is a list. The inner list is one line thats split up into "words".

li = [i.strip().split() for i in open("input.txt").readlines()]

I put the code snippet you posted into a input.txt file in c:\temp and ran this line. Is the output similar to what you want?

C:\temp>python
Python 3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print([i.strip().split() for i in open("input.txt").readlines()])
[['p', 'wfgh', '1111', '11111', '111111'], ['287', '48', '0'], ['65626', '-1818', '0'], ['4654', '21512', '02020', '0']]

Answer 4

fh=open("file")
mylist=[]
header=fh.readline().rstrip()
if not header.startswith("p wncf") :
    print "error"
header=header.split()
mylist.append(header)
if len(header) != 5:
    print "error"
if False in map(str.isdigit, header[2:]):
    print "Error"
for line in fh:
    line=line.rstrip().split()
    if False in map(str.isdigit, line[0:2]):
        print "Error"            
    elif line[-1] != 0: 
        print "Error"
    else:
        mylist.append(line)
fh.close()

Answer 5

fileName=open("d:/foo.bar")
lines = [i for i in fileName.readlines()]

hope that helps :D

Answer 6

p = open('filename')

#List:
linelist = [line for line in p.readlines()]

"""
But I prefer creating a dictionary as I find them more useful at times. Example here is very trivial. You can use the list index as a line number also.
"""

#Dictionary:
linedict = dict([(no, line) for no, line in enumerate(p.readlines())])

Answer 7

To build a list of only the lines in the file that contain at least two integers and end with a zero, use a regular expression:

import re
p = re.compile(r'^((\-?\d*\s+){2,})0$')
with open(filename, 'rb') as f:
    seq = [line.strip() for line in f if p.match(line)]

Answer 8

You're not providing all the details, but i assume that:

• there's only 1 header line at the beginning and you don't need what's in it
• the other lines only contain integers
• you don't need to retain the trailing '0'

I have to also assume your file may be very big, so reading the entire file in memory, or storing the entire resulting list in memory is not a very good idea.

Here's a quick solution which reads the file line-by-line and uses a generator to yield each line as a list. You can use the entire result as one list if you want, like so:

result_list = read_data('foo.dat')

or you can do as i did in the example call and use each result line as it's read out. You can call this file directly if you're on linux, otherwise just associate it with the python interpreter, and call it with the name of the data file as the first argument, and it'll print the results line by line - this will work even if your file is humongous. You can also just import the file as a module and use the read_data method and use the results in other calculations.

Note that it does some error checking (the header line starts with a p and the data lines end with a 0, and only contain integers), and you probably want to either not do this checking at all, or raise a proper exception when they're encountered.

#!/usr/bin/env python
import sys

def read_data(fn):
    """Reads in datafile

    data file is in format:
        p wfgh 1111 11111 111111
        287 48 0
        65626 -1818 0
        4654 21512 02020 0
    where first line begins with p and is a header, and following lines
    are comprised of at least 2 integers plus a tailing 0.
    Pass in the filename, the resulting list of lists of integers will be 
    returned.
    """
    f = open(fn, 'r')
    # check for header line
    assert(f.readline().split()[0]=='p')
    for l in f:
        d = [int(col) for col in l.split()]
        if not d:
            # skip empty lines
            continue
        # check we have at least 2 integers and the last column is 0
        assert(d[-1] == 0 and len(d) >= 3)
        # yield current line
        yield d[:-1]

if __name__ == '__main__':
    for l in read_data(sys.argv[1]):
        print unicode(l)