Best way to construct a "complex" data structure in Python

Question

Hello guys.

I need to construct a tool that will be used to create field mappings (between tables) in the most automated manner possible.

Here is the deal: imagine a table being appended to other. (lets ignore field type, just for a second...)

CREATE OR REPLACE TABLE fooA(
id,
name,
type,
foo)

CREATE OR REPLACE TABLE otherFooTable(
idFoo,
nameFoo,
spam)

I am thinking to create a structure like this:

fieldMap = {'otherFooTable': [('idFoo','id'),('nameFoo','name'),('spam','foo')]}

I would be able to access this using (for example)

print fieldMap['tabelax'][0][1]

It´s not a very complex structure, but i can run into some problems using it? Is there any suggestions of how to handle this sort of issue? I need to store (for now) at least inputTable (i don´t want to repeat it for each field mapped), inputField,outputField. There is no reason to store outputTable, because that is always known beforehand.

Suggestions and past experiences are deeply appreciated.

PS: perhaps a formal structure (like a class) would be better?

Thanks

Answer 1

I'd honestly just take hints from (or use) SQLAlchemy or Django Models. These are tried and true data representation methods.

Answer 2

Think about this

class Column( object ):
    def __init__( self, name, type_information=None ):
        self.name = name
        self.type_information = type_information
        self.pk = None
        self.fk_ref = None
    def fk( self, column ): 
        self.fk_ref = column

class Table( object ):
    def __init__( self, name, *columns ):
        self.name = name
        self.columns = dict( (c.name, c) for c in columns )
    def column( self, name ):
        return self.columns[ name ]

Table( "FOOA", Column( "id" ), Column( "name" ), Column( "type" ), Column( "foo" ) )

Table( "otherFooTable", Column( "idFoo" ), Column( "nameFoo" ), Column( "spam" ) )

It's not clear at all what you're tying to do or why, so this is as good as anything, since it seems to represent the information you actually have.

Answer 3

Try to avoid accessing your data through fixed numerical indexes as in fieldMap['tabelax'][0][1]. After a year of not looking at your code, it may take you (or others) a while to figure out what it all means in human terms (e.g. "the value of idFoo in table tabelax"). Also, if you ever need to change your data structure (e.g. add another field) then some/all your numerical indexes may need fixing. Your code becomes ossified when the risk of breaking the logic prevents you from modifying the data structure.

It is much better to use a class and use class (accessor) methods to access the data structure. That way, the code outside of your class can be preserved even if you need to change your data structure (inside the class) at some future date.

Answer 4

Here is a little wrapper class for FooB's to mimic FooA's, but still retain their FooB-ishness.

from collections import namedtuple

# use namedtuple to define some simple classes (requires Py2.6 or later)
FooA = namedtuple('FooA', 'id name type foo')
FooB = namedtuple('FooB', 'idfoo namefoo spam')

# create a wrapper class for FooB's to look like a FooA
class FooAMimic(object):
    attrMap = dict(zip(FooA._fields, FooB._fields))
    # or if the fields aren't nicely ordered, declare this mapping explicitly
    #~ attrMap = { 'id' : 'idfoo', 'name' : 'namefoo', 'foo' : 'spam' }
    def __init__(self, obj):
        self.obj = obj
    def __getattr__(self, aname):
        ob = self.obj
        if aname in self.attrMap:
            return getattr(ob, self.attrMap[aname])
        elif  hasattr(ob, aname):
            return getattr(ob, aname)
        else:
            raise AttributeError("no such attribute " + aname)
    def __dir__(self):
        return sorted(set(dir(super(FooAMimic,self)) 
                          + dir(self.obj) 
                          + list(FooA._fields)))

Use it like this:

# make some objects, some FooA, some FooB
fa = FooA('a', 'b', 'c','d')
fb = FooB('xx', 'yy', 'zz')
fc = FooA('e', 'f', 'g','h')

# create list of items that are FooA's, or FooA lookalikes
coll = [fa, FooAMimic(fb), fc]

# access objects like FooA's, but notice that the wrapped FooB
# attributes are still available too
for f in sorted(coll, key=lambda k : k.id):
    print f.id, '=', 
    try:
        print f.namefoo, "(really a namefoo)"
    except AttributeError:
        print f.name

Prints:

a = b
e = f
xx = yy (really a namefoo)