Creation of dynamic model fields in django

Question

This is a problem concerning django. I have a model say "Automobiles". This will have some basic fields like "Color","Vehicle Owner Name", "Vehicle Cost".

I want to provide a form where the user can add extra fields depending on the automobile that he is adding. For example, if the user is adding a "Car", he will extra fields in the form, dynamically at run time, like "Car Milage", "Cal Manufacturer". Suppose if the user wants to add a "Truck", he will add "Load that can be carried", "Permit" etc.

How do I achieve this in django?

There are two questions here:

1. How to provide a form where the user can add new fields at run time?
2. How to add the fields to the database so that it can be retrieved/queried later?

Answer 1

Are you talking about in a front end interface, or in the Django admin?

You can't create real fields on the fly like that without a lot of work under the hood. Each model and field in Django has an associated table and column in the database. To add new fields usually requires either raw sql, or migrations using South.

From a front end interface, you could create pseudo fields, and store them in a json format in a single model field.

For example, create an other_data text field in the model. Then allow users to create fields, and store them like {'userfield':'userdata','mileage':54}

But I think if you're using a finite class like vehicles, you would create a base model with the basic vehicle characteristics, and then create models that inherits from the base model for each of the vehicle types.

class base_vehicle(models.Model):
    color = models.CharField()
    owner_name = models.CharField()
    cost = models.DecimalField()

class car(base_vehicle):
    mileage = models.IntegerField(default=0)

etc

Answer 2

There are a few approaches:

• key/value model (easy, well supported)
• JSON data in a TextField (easy, flexible, can't search/index easily)
• Dynamic model definition (not so easy, many hidden problems)

It sounds like you want the last one, but I'm not sure it's the best for you. Django is very easy to change/update, if system admins want extra fields, just add them for them and use south to migrate. I don't like generic key/value database schemas, the whole point of a powerful framework like Django is that you can easily write and rewrite custom schemas without resorting to generic approaches.

If you must allow site users/administrators to directly define their data, I'm sure others will show you how to do the first two approaches above. The third approach is what you were asking for, and a bit more crazy, I'll show you how to do. I don't recommend using it in almost all cases, but sometimes it's appropriate.

Dynamic models

Once you know what to do, this is relatively straightforward. You'll need:

• 1 or 2 models to store the names and types of the fields
• (optional) An abstract model to define common functionality for your (subclassed) dynamic models
• A function to build (or rebuild) the dynamic model when needed
• Code to build or update the database tables when fields are added/removed/renamed

1. Storing the model definition

This is up to you. I imagine you'll have a model CustomCarModel and CustomField to let the user/admin define and store the names and types of the fields you want. You don't have to mirror Django fields directly, you can make your own types that the user may understand better.

Use a forms.ModelForm with inline formsets to let the user build their custom class.

2. Abstract model

Again, this is straightforward, just create a base model with the common fields/methods for all your dynamic models. Make this model abstract.

3. Build a dynamic model

Define a function that takes the required information (maybe an instance of your class from #1) and produces a model class. This is a basic example:

from django.db.models.loading import cache
from django.db import models


def get_custom_car_model(car_model_definition):
  """ Create a custom (dynamic) model class based on the given definition.
  """
  # What's the name of your app?
  _app_label = 'myapp'

  # you need to come up with a unique table name
  _db_table = 'dynamic_car_%d' % car_model_definition.pk

  # you need to come up with a unique model name (used in model caching)
  _model_name = "DynamicCar%d" % car_model_definition.pk

  # Remove any exist model definition from Django's cache
  try:
    del cache.app_models[_app_label][_model_name.lower()]
  except KeyError:
    pass

  # We'll build the class attributes here
  attrs = {}

  # Store a link to the definition for convenience
  attrs['car_model_definition'] = car_model_definition

  # Create the relevant meta information
  class Meta:
      app_label = _app_label
      db_table = _db_table
      managed = False
      verbose_name = 'Dynamic Car %s' % car_model_definition
      verbose_name_plural = 'Dynamic Cars for %s' % car_model_definition
      ordering = ('my_field',)
  attrs['__module__'] = 'path.to.your.apps.module'
  attrs['Meta'] = Meta

  # All of that was just getting the class ready, here is the magic
  # Build your model by adding django database Field subclasses to the attrs dict
  # What this looks like depends on how you store the users's definitions
  # For now, I'll just make them all CharFields
  for field in car_model_definition.fields.all():
    attrs[field.name] = models.CharField(max_length=50, db_index=True)

  # Create the new model class
  model_class = type(_model_name, (CustomCarModelBase,), attrs)

  return model_class

4. Code to update the database tables

The code above will generate a dynamic model for you, but won't create the database tables. I recommend using South for table manipulation. Here are a couple of functions, which you can connect to pre/post-save signals:

import logging
from south.db import db
from django.db import connection

def create_db_table(model_class):
  """ Takes a Django model class and create a database table, if necessary.
  """
  table_name = model_class._meta.db_table
  if (connection.introspection.table_name_converter(table_name)
                    not in connection.introspection.table_names()):
    fields = [(f.name, f) for f in model_class._meta.fields]
    db.create_table(table_name, fields)
    logging.debug("Creating table '%s'" % table_name)

def add_necessary_db_columns(model_class):
  """ Creates new table or relevant columns as necessary based on the model_class.
    No columns or data are renamed or removed.
    XXX: May need tweaking if db_column != field.name
  """
  # Create table if missing
  create_db_table(model_class)

  # Add field columns if missing
  table_name = model_class._meta.db_table
  fields = [(f.column, f) for f in model_class._meta.fields]
  db_column_names = [row[0] for row in connection.introspection.get_table_description(connection.cursor(), table_name)]

  for column_name, field in fields:
    if column_name not in db_column_names:
      logging.debug("Adding field '%s' to table '%s'" % (column_name, table_name))
      db.add_column(table_name, column_name, field)

And there you have it! You can call get_custom_car_model() to deliver a django model, which you can use to do normal django queries:

CarModel = get_custom_car_model(my_definition)
CarModel.objects.all()

Problems

• Your models are hidden from Django until the code creating them is run. You can however run get_custom_car_model for every instance of your definitions in the class_prepared signal for your definition model.
• ForeignKeys/ManyToManyFields may not work (I haven't tried)
• You will want to use Django's model cache so you don't have to run queries and create the model every time you want to use this. I've left this out above for simplicity
• You can get your dynamic models into the admin, but you'll need to dynamically create the admin class as well, and register/reregister/unregister appropriately using signals.

Overview

If you're fine with the added complication and problems, enjoy! One it's running, it works exactly as expected thanks to Django and Python's flexibility. You can feed your model into Django's ModelForm to let the user edit their instances, and perform queries using the database's fields directly. If there is anything you don't understand in the above, you're probably best off not taking this approach (I've intentionally not explained what some of the concepts are for beginners). Keep it Simple!

I really don't think many people need this, but I have used it myself, where we had lots of data in the tables and really, really needed to let the users customise the columns, which changed rarely.