How can I get fields in an original order?

Question

I have a code like:

class Ordered(object):
    x = 0
    z = 0
    b = 0
    a = 0

print(dir(Ordered))

it prints:

[ ......., a, b, x, z]

How can I get fields in an original order: x, z, b, a? I've seen similar behavior in Django Models.

Answer 1

You can't track the order of the addition of the class variables. These attributes (as well as attributes on objects) are stored internally as a dictionary, which is optimized for fast lookups and does not support ordering.

You can see this fact:

class A(object):
    x = 0
    y = 0
    z = 0

A.__dict__.items()

#  [('__module__', '__main__'), 
#   ('__dict__', <attribute '__dict__' of 'A' objects>), 
#   ('y', 0), ('x', 0), ('z', 0), 
#   ('__weakref__', <attribute '__weakref__' of 'A' objects>), 
#   ('__doc__', None)]

If you want your attributes in a particular order, you could just add another field, containing this inforamtion:

class B(object):
    x = 0
    y = 0
    z = 0
    a = 0
    _ordering = ['x', 'y', 'z', 'a']

print B._ordering
# => ['x', 'y', 'z', 'a']

Sidenote: In python 2.7 and 3.2 ordered dictionaries will be introduced as part of the standard library.

Answer 2

Django's model and form metaclasses work together with the field descriptors to maintain the original order. There is no way of doing what you ask without jumping through a lot of hoops. See the Django source code if you're still interested.

Answer 3

As mentioned above, if you want to keep things simple, just use a eg _ordering attribute, which manually keeps track of ordering. Otherwise, here is a metaclass approach (like the one Django uses), which creates an ordering attribute automatically.

Recording the original ordering

Classes don't keep track of the ordering of the attributes. You can however keep track of which order the field instances were created. For that, you'll have to use your own class for fields (not int). The class keeps track of how many instances have already been made and each instance takes note of its position. Here is how you would do it for your example (storing integers):

class MyOrderedField(int):
  creation_counter = 0

  def __init__(self, val):
    # Set the instance's counter, to keep track of ordering
    self.creation_counter = MyOrderedField.creation_counter
    # Increment the class's counter for future instances
    MyOrderedField.creation_counter += 1

Creating an ordered_items attribute automatically

Now that your fields have a number which can be used to order them, your parent class needs to use that somehow. You can do this a variety of ways, if I remember correctly, Django uses Metaclasses to do this, which is a bit wild for a simple class.

class BaseWithOrderedFields(type):
  """ Metaclass, which provides an attribute "ordered_fields", being an ordered
      list of class attributes that have a "creation_counter" attribute. """

  def __new__(cls, name, bases, attrs):
    new_class = super(BaseWithOrderedFields, cls).__new__(cls, name, bases, attrs)
    # Add an attribute to access ordered, orderable fields
    new_class._ordered_items = [(name, attrs.pop(name)) for name, obj in attrs.items()
                                    if hasattr(obj, "creation_counter")]
    new_class._ordered_items.sort(key=lambda item: item[1].creation_counter)
    return new_class

Using this metaclass

So, how do you use this? First, you need to use our new MyOrderedField class when defining your attributes. This new class will keep track of the order in which the fields were created:

class Ordered(object):
  __metaclass__ = BaseWithOrderedFields

  x = MyOrderedField(0)
  z = MyOrderedField(0)
  b = MyOrderedField(0)
  a = MyOrderedField(0)

Then you can access the ordered fields in our automatically created attribute ordered_fields:

>>> ordered = Ordered()
>>> ordered.ordered_fields
[('x', 0), ('z', 0), ('b', 0), ('a', 0)]

Feel free to change this to an ordered dict or just return the names or whatever you need. Additionally, you can define an empty class with the __metaclass__ and inherit from there.

Don't use this!

As you can see, this approach is a little overcomplicated and probably not suitable for most tasks or python developers. If you're newish to python, you'll probably spend more time and effort developing your metaclass than you would have if you just defined the ordering manually. Defining your own ordering manually is almost always going to be the best approach. Django do it automatically because the complicated code is hidden from the end developer, and Django is used far more often than it itself is written/maintained. So only if you're developing a framework for other developers, then metaclasses may be useful for you.

Answer 4

I was 80% done with this answer when Will posted his, but I decided to post anyway so the effort wouldn't go to waste (our answers basically describe the same thing).

Here's how Django does it. I've chosen to keep the same nomenclature, methodology, and data structures as Django, so that this answer may be also useful for people trying to understand how field names are sorted in Django.

from bisect import bisect

class Field(object):
    # A global creation counter that will contain the number of Field objects
    # created globally.
    creation_counter = 0

    def __init__(self, *args, **kwargs):
        super(Field, self).__init__(*args, **kwargs)
        # Store the creation index in the "creation_counter" of the field.
        self.creation_counter = Field.creation_counter
        # Increment the global counter.
        Field.creation_counter += 1
        # As with Django, we'll be storing the name of the model property
        # that holds this field in "name".
        self.name = None

    def __cmp__(self, other):
        # This specifies that fields should be compared based on their creation
        # counters, allowing sorted lists to be built using bisect.
        return cmp(self.creation_counter, other.creation_counter)

# A metaclass used by all Models
class ModelBase(type):
    def __new__(cls, name, bases, attrs):
        klass = super(ModelBase, cls).__new__(cls, name, bases, attrs)
        fields = []
        # Add all fields defined for the model into "fields".
        for key, value in attrs.items():
            if isinstance(value, Field):
                # Store the name of the model property.
                value.name = key
                # This ensures the list is sorted based on the creation order
                fields.insert(bisect(fields, value), value)
        # In Django, "_meta" is an "Options" object and contains both a
        # "local_fields" and a "many_to_many_fields" property. We'll use a
        # dictionary with a "fields" key to keep things simple.
        klass._meta = { 'fields': fields }
        return klass

class Model(object):
    __metaclass__ = ModelBase

Now let's define some example models:

class Model1(Model):
    a = Field()
    b = Field()
    c = Field()
    z = Field()

class Model2(Model):
    c = Field()
    z = Field()
    b = Field()
    a = Field()

And let's test them:

>>>> [f.name for f in Model1()._meta['fields']]
['a', 'b', 'c', 'z']
>>>> [f.name for f in Model2()._meta['fields']]
['c', 'z', 'b', 'a']

Hope this helps clarify anything that wasn't already clear in Will's answer.