best way to parse a line in python to a dictionary

Question

I have a file with lines like

account = "TEST1" Qty=100 price = 20.11 subject="some value" values="3=this, 4=that"

There is no special delimiter and each key has a value that is surrounded by double quotes if its a string but not if it is a number. There is no key without a value though there may exist blank strings which are represented as "" and there is no escape character for a quote as it is not needed

I want to know what is a good way to parse this kind of line with python and store the values as key-value pairs in a dictionary

Answer 1

We're going to need a regex for this.

import re, decimal
r= re.compile('([^ =]+) *= *("[^"]*"|[^ ]*)')

d= {}
for k, v in r.findall(line):
    if v[:1]=='"':
        d[k]= v[1:-1]
    else:
        d[k]= decimal.Decimal(v)

>>> d
{'account': 'TEST1', 'subject': 'some value', 'values': '3=this, 4=that', 'price': Decimal('20.11'), 'Qty': Decimal('100.0')}

You can use float instead of decimal if you prefer, but it's probably a bad idea if money is involved.

Answer 2

A recursive variation of bobince's parses values with embedded equals as dictionaries:

>>> import re
>>> import pprint
>>>
>>> def parse_line(line):
...     d = {}
...     a = re.compile(r'\s*(\w+)\s*=\s*("[^"]*"|[^ ,]*),?')
...     float_re = re.compile(r'^\d.+$')
...     int_re = re.compile(r'^\d+$')
...     for k,v in a.findall(line):
...             if int_re.match(k):
...                     k = int(k)
...             if v[-1] == '"':
...                     v = v[1:-1]
...             if '=' in v:
...                     d[k] = parse_line(v)
...             elif int_re.match(v):
...                     d[k] = int(v)
...             elif float_re.match(v):
...                     d[k] = float(v)
...             else:
...                     d[k] = v
...     return d
...
>>> line = 'account = "TEST1" Qty=100 price = 20.11 subject="some value" values=
"3=this, 4=that"'
>>> pprint.pprint(parse_line(line))
{'Qty': 100,
 'account': 'TEST1',
 'price': 20.109999999999999,
 'subject': 'some value',
 'values': {3: 'this', 4: 'that'}}

Answer 3

If you don't want to use a regex, another option is just to read the string a character at a time:

string = 'account = "TEST1" Qty=100 price = 20.11 subject="some value" values="3=this, 4=that"'

inside_quotes = False
key = None
value = ""
dict = {}

for c in string:
    if c == '"':
        inside_quotes = not inside_quotes
    elif c == '=' and not inside_quotes:
        key = value
        value = ''
    elif c == ' ':
        if inside_quotes:
            value += ' ';
        elif key and value:
            dict[key] = value
            key = None
            value = ''
    else:
        value += c

dict[key] = value
print dict

Answer 4

Maybe a bit simpler to follow is the pyparsing rendition:

from pyparsing import *

# define basic elements - use re's for numerics, faster than easier than 
# composing from pyparsing objects
integer = Regex(r'[+-]?\d+')
real = Regex(r'[+-]?\d+\.\d*')
ident = Word(alphanums)
value = real | integer | quotedString.setParseAction(removeQuotes)

# define a key-value pair, and a configline as one or more of these
# wrap configline in a Dict so that results are accessible by given keys
kvpair = Group(ident + Suppress('=') + value)
configline = Dict(OneOrMore(kvpair))

src = 'account = "TEST1" Qty=100 price = 20.11 subject="some value" ' \
        'values="3=this, 4=that"'

configitems = configline.parseString(src)

Now you can access your pieces using the returned configitems ParseResults object:

>>> print configitems.asList()
[['account', 'TEST1'], ['Qty', '100'], ['price', '20.11'], 
 ['subject', 'some value'], ['values', '3=this, 4=that']]

>>> print configitems.asDict()
{'account': 'TEST1', 'Qty': '100', 'values': '3=this, 4=that', 
  'price': '20.11', 'subject': 'some value'}

>>> print configitems.dump()
[['account', 'TEST1'], ['Qty', '100'], ['price', '20.11'], 
 ['subject', 'some value'], ['values', '3=this, 4=that']]
- Qty: 100
- account: TEST1
- price: 20.11
- subject: some value
- values: 3=this, 4=that

>>> print configitems.keys()
['account', 'subject', 'values', 'price', 'Qty']

>>> print configitems.subject
some value