Python: load text as python object

Question

Hi!
I have a such text to load: https://sites.google.com/site/iminside1/paste
I'd prefer to create a python dictionary from it, but any object is OK. I tried pickle, json and eval, but didn't succeeded. Can you help me with this?
Thanks!
The results:

a = open("the_file", "r").read()

json.loads(a)
ValueError: Expecting property name: line 1 column 1 (char 1)

pickle.loads(a)
KeyError: '{'

eval(a)
File "<string>", line 19
from: {code: 'DME', airport: "Домодедово", city: 'Москва', country: 'Россия', terminal: ''},
    ^
SyntaxError: invalid syntax

Answer 1

If you really have to load the bulls... this data is (see my comment), you's propably best of with a regex adding missing quotes. Something like r"([a-zA-Z_][a-zA-Z_0-9]*)\s*\:" to find things to quote and r"\'\1\'\:" as replacement (off the top of my head, I have to test it first).

Edit: After some troulbe with backward-references in Python 3.1, I finally got it working with these:

>>> pattern = r"([a-zA-Z_][a-zA-Z_0-9]*)\s*\:"
>>> test = '{"foo": {bar: 1}}'
>>> repl = lambda match: '"{}":'.format(match.group(1))
>>> eval(re.sub(pattern, repl, test))
{'foo': {'bar': 1}}

Answer 2

Till now with help of delnan and a little investigation I can load it into dict with eval:

pattern = r"\b(?P<word>\w+):"
x = re.sub(pattern, '"\g<word>":',open("the_file", "r").read())
y = x.replace("true", '"true"')
d = eval(y)

Still looking for more efficient and maybe simpler solution.. I don't like to use "eval" for some reasons.

Answer 3

Lifted almost straight from the pyparsing examples page:

# read text from web page
import urllib
page = urllib.urlopen("https://sites.google.com/site/iminside1/paste")
html = page.read()
page.close()

start = html.index("<pre>")+len("<pre>")+3 #skip over 3-byte header
end = html.index("</pre>")
text = html[start:end]
print text

# parse dict-like syntax    
from pyparsing import (Suppress, Regex, quotedString, Word, alphas, 
alphanums, oneOf, Forward, Optional, dictOf, delimitedList, removeQuotes)

LBRACK,RBRACK,LBRACE,RBRACE,COLON,COMMA = map(Suppress,"[]{}:,")
integer = Regex(r"[+-]?\d+").setParseAction(lambda t:int(t[0]))
real = Regex(r"[+-]?\d+\.\d*").setParseAction(lambda t:float(t[0]))
string_ = Word(alphas,alphanums+"_") | quotedString.setParseAction(removeQuotes)
bool_ = oneOf("true false").setParseAction(lambda t: t[0]=="true")
item = Forward()

key = string_
dict_ = LBRACE - Optional(dictOf(key+COLON, item+Optional(COMMA))) + RBRACE
list_ = LBRACK - Optional(delimitedList(item)) + RBRACK
item << (real | integer | string_ | bool_ | Group(list_ | dict_ ))

result = item.parseString(text,parseAll=True)[0]
print result.data[0].dump()
print result.data[0].segments[0].dump(indent="  ")
print result.data[0].segments[0].flights[0].dump(indent="  -  ")
print result.data[0].segments[0].flights[0].flightLegs[0].dump(indent="  -  -  ")
for seg in result.data[6].segments:
    for flt in seg.flights:
        fltleg = flt.flightLegs[0]
        print "%(airline)s %(airlineCode)s %(flightNo)s" % fltleg,
        print "%s -> %s" % (fltleg["from"].code, fltleg["to"].code)

Prints:

[['index', 0], ['serviceClass', '??????'], ['prices', [3504, ...
- eTicketing: true
- index: 0
- prices: [3504, 114.15000000000001, 89.769999999999996]
- segments: [[['indexSegment', 0], ['stopsCount', 0], ['flights', ... 
- serviceClass: ??????
  [['indexSegment', 0], ['stopsCount', 0], ['flights', [[['index', 0], ...
  - flights: [[['index', 0], ['time', 'PT2H45M'], ['minAvailSeats', 9], ...
  - indexSegment: 0
  - stopsCount: 0
  -  [['index', 0], ['time', 'PT2H45M'], ['minAvailSeats', 9], ['flight...
  -  - flightLegs: [[['flightNo', '309'], ['eTicketing', 'true'], ['air... 
  -  - index: 0
  -  - minAvailSeats: 9
  -  - stops: []
  -  - time: PT2H45M
  -  -  [['flightNo', '309'], ['eTicketing', 'true'], ['airplane', 'Boe... 
  -  -  - airline: ?????????
  -  -  - airlineCode: UN
  -  -  - airplane: Boeing 737-500
  -  -  - availSeats: 9
  -  -  - classCode: I
  -  -  - eTicketing: true
  -  -  - fareBasis: IPROW
  -  -  - flightClass: ECONOMY
  -  -  - flightNo: 309
  -  -  - from:   -  -  [['code', 'DME'], ['airport', '??????????'], ... 
  -  -    - airport: ??????????
  -  -    - city: ??????
  -  -    - code: DME
  -  -    - country: ??????
  -  -    - terminal: 
  -  -  - fromDate: 2010-10-15
  -  -  - fromTime: 10:40:00
  -  -  - time: 
  -  -  - to:   -  -  [['code', 'TXL'], ['airport', 'Berlin-Tegel'], ... 
  -  -    - airport: Berlin-Tegel
  -  -    - city: ??????
  -  -    - code: TXL
  -  -    - country: ????????
  -  -    - terminal: 
  -  -  - toDate: 2010-10-15
  -  -  - toTime: 11:25:00
airBaltic BT 425 SVO -> RIX
airBaltic BT 425 SVO -> RIX
airBaltic BT 423 SVO -> RIX
airBaltic BT 423 SVO -> RIX

EDIT: fixed grouping and expanded output dump to show how to access individual key fields of results, either by index (within list) or as attribute (within dict).