Concatenate sequence from a predefined datastructure

Question

Hello,

I've been struggling a little to build this piece of code, and I was wondering if there are others more simple/efficient way of doing this:

fsSchema = {'published': {'renders': {'SIM': ('fold1', 'fold2'), 'REN': ('fold1', 'fold2')}}}


def __buildPathFromSchema(self, schema, root=''):        
    metaDirs = []
    for dir_ in schema.keys():
        root = os.path.join(root, dir_)
        if isinstance(schema[dir_], dict):
            return self.__buildPathFromSchema(schema[dir_], root)

        if isinstance(schema[dir_], tuple):
            for i in schema[dir_]:
                bottom = os.path.join(root, i)
                metaDirs.append(bottom)
            root = os.sep.join(os.path.split(root)[:-1])
    return metaDirs

Basically what I want to do is generating paths from a predefined structure like fsSchema. Note the latest iteration is always a tuple.

The ouput looks like:

['published\renders\REN\fold1', 'published\renders\REN\fold2', 'published\renders\SIM\fold1', 'published\renders\SIM\fold2']

Thanks!

Answer 1

You can use a recursive function to generate all the paths:

def flatten(data):
   if isinstance(data, tuple):
      for v in data:
         yield v
   else:
      for k in data:
         for v in flatten(data[k]):
            yield k + '\\' + v

This should be able to handle any kind of nested dictionaries:

>>> fsSchema = {'published': {'renders': {'SIM': ('fold1', 'fold2'), 'REN': ('fold1', 'fold2')}}}
>>> list(flatten(fsSchema))
['published\\renders\\REN\\fold1', 'published\\renders\\REN\\fold2', 'published\\renders\\SIM\\fold1', 'published\\renders\\SIM\\fold2']

Note that the paths are generated in "random" order since dictionaries don't have any internal ordering.

Answer 2

Instead of:

for dir_ in schema.keys():
    ...
    if isinstance(schema[dir_], dict):

you can do:

for dir_name, dir_content in schema.iteritems():
    ...
    if isinstance(dir_content, tuple):

It's both faster and more readable.

Answer 3

I would keep doing it recursively like you already are but split the walker off from the path generator:

def walk(data):
    if hasattr(data, 'items'):
        for outer_piece, subdata in data.items():
            for inner_piece in walk(subdata):
                yield (outer_piece, ) + inner_piece
     else:
         for piece in data:
             yield (piece, )

def paths(data):
    for path in walk(data):
        yield os.sep.join(path)

The reason being that it is really two separate pieces of functionality and having them implemented as separate functions is hence easier to debug, maintain, implement and just generally think about.