How can _meta.local_fields not match the table schema in the database?

Question

I'm completely confused about why _meta.local_fields returns more fields than the database table contains. The User model inherits from contrib.auth.models.User.

$ mysql -u user -p database  
Enter password:   
Reading table information for completion of table and column names  
You can turn off this feature to get a quicker startup with -A  

Welcome to the MySQL monitor.  Commands end with ; or \g.  
Your MySQL connection id is 1240032  
Server version: 5.0.77 Source distribution  

mysql> describe auth_user;  
+--------------------+------------------+------+-----+---------+----------------+  
| Field              | Type             | Null | Key | Default | Extra          |  
+--------------------+------------------+------+-----+---------+----------------+  
| id                 | int(11)          | NO   | PRI | NULL    | auto_increment |   
| username           | varchar(30)      | NO   | UNI | NULL    |                |   
| first_name         | varchar(30)      | NO   |     | NULL    |                |   
| last_name          | varchar(30)      | NO   |     | NULL    |                |   
| email              | varchar(75)      | NO   |     | NULL    |                |   
| password           | varchar(128)     | NO   |     | NULL    |                |   
| is_staff           | tinyint(1)       | NO   |     | NULL    |                |   
| is_active          | tinyint(1)       | NO   |     | NULL    |                |   
| is_superuser       | tinyint(1)       | NO   |     | NULL    |                |   
| last_login         | datetime         | NO   |     | NULL    |                |   
| date_joined        | datetime         | NO   |     | NULL    |                |   
| email_isvalid      | tinyint(1)       | NO   |     | NULL    |                |   
| email_key          | varchar(16)      | YES  |     | NULL    |                |   
| reputation         | int(10) unsigned | NO   |     | NULL    |                |   
| gravatar           | varchar(32)      | NO   |     | NULL    |                |   
| gold               | smallint(6)      | NO   |     | NULL    |                |   
| silver             | smallint(6)      | NO   |     | NULL    |                |   
| bronze             | smallint(6)      | NO   |     | NULL    |                |   
| questions_per_page | smallint(6)      | NO   |     | NULL    |                |   
| last_seen          | datetime         | NO   |     | NULL    |                |   
| real_name          | varchar(100)     | NO   |     | NULL    |                |   
| website            | varchar(200)     | NO   |     | NULL    |                |   
| location           | varchar(100)     | NO   |     | NULL    |                |   
| date_of_birth      | date             | YES  |     | NULL    |                |   
| about              | longtext         | NO   |     | NULL    |                |   
+--------------------+------------------+------+-----+---------+----------------+  
25 rows in set (0.00 sec)  

From the Django error page, this is the SQL statement that generates the error: Exception Value: (1110, "Column 'about' specified twice")

'INSERT INTO `auth_user` (`username`, `first_name`, `last_name`, `email`, `password`, `is_staff`, `is_active`, `is_superuser`, `last_login`, `date_joined`,     `email_isvalid`, `email_key`, `reputation`, `gravatar`, `gold`, `silver`, `bronze`, `questions_per_page`, `last_seen`, `real_name`, `website`, `location`, `date_of_birth`, `about`, `email_isvalid`, `email_key`, `reputation`, `gravatar`, `gold`, `silver`, `bronze`, `questions_per_page`, `last_seen`, `real_name`, `website`, `location`, `date_of_birth`, `about`) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)'     

This SQL statement seems to be generated by iterating over User._meta.local_fields.
I don't understand why _meta.local_fields doesn't match the actual User table schema.

$ python2.5 manage.py shell  
Python 2.5.4 (r254:67916, Aug  5 2009, 12:42:40)   
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2  
Type "help", "copyright", "credits" or "license" for more information.  
(InteractiveConsole)  
>>> from forum.models import User  
>>> len(User._meta.local_fields)  
39  
>>> import pprint  
>>> pprint.pprint(User._meta.local_fields)  
[<django.db.models.fields.AutoField object at 0x852c80c>,  
 <django.db.models.fields.CharField object at 0x8528c4c>,  
 <django.db.models.fields.CharField object at 0x8528cac>,  
 <django.db.models.fields.CharField object at 0x8528d0c>,  
 <django.db.models.fields.EmailField object at 0x8528d6c>,  
 <django.db.models.fields.CharField object at 0x8528e2c>,  
 <django.db.models.fields.BooleanField object at 0x8528ecc>,  
 <django.db.models.fields.BooleanField object at 0x8528f6c>,  
 <django.db.models.fields.BooleanField object at 0x852c02c>,  
 <django.db.models.fields.DateTimeField object at 0x852c0ac>,  
 <django.db.models.fields.DateTimeField object at 0x852c0ec>,  
 # here is where customizations to User begin.
 <django.db.models.fields.BooleanField object at 0x861744c>,  
 <django.db.models.fields.CharField object at 0x861732c>,  
 <django.db.models.fields.PositiveIntegerField object at 0x861746c>,  
 <django.db.models.fields.CharField object at 0x861748c>,  
 <django.db.models.fields.SmallIntegerField object at 0x861784c>,  
 <django.db.models.fields.SmallIntegerField object at 0x86178ec>,  
 <django.db.models.fields.SmallIntegerField object at 0x861792c>,  
 <django.db.models.fields.SmallIntegerField object at 0x861796c>,  
 <django.db.models.fields.DateTimeField object at 0x861798c>,  
 <django.db.models.fields.CharField object at 0x86179cc>,  
 <django.db.models.fields.URLField object at 0x8617a0c>,  
 <django.db.models.fields.CharField object at 0x8617a4c>,  
 <django.db.models.fields.DateField object at 0x8617a8c>,  
 <django.db.models.fields.TextField object at 0x8617acc>,  
 # this seems to be a duplicate of the fields added to User
 <django.db.models.fields.BooleanField object at 0x862ab2c>,  
 <django.db.models.fields.CharField object at 0x862a4ac>,  
 <django.db.models.fields.PositiveIntegerField object at 0x862ab6c>,  
 <django.db.models.fields.CharField object at 0x862f6cc>,  
 <django.db.models.fields.SmallIntegerField object at 0x861782c>,  
 <django.db.models.fields.SmallIntegerField object at 0x862fa2c>,  
 <django.db.models.fields.SmallIntegerField object at 0x862fa4c>,  
 <django.db.models.fields.SmallIntegerField object at 0x862fa8c>,  
 <django.db.models.fields.DateTimeField object at 0x862faac>,  
 <django.db.models.fields.CharField object at 0x862faec>,  
 <django.db.models.fields.URLField object at 0x862fb2c>,  
 <django.db.models.fields.CharField object at 0x862fb6c>,  
 <django.db.models.fields.DateField object at 0x862fbac>,  
 <django.db.models.fields.TextField object at 0x862fbec>]  

The additional fields to the model are added thusly:

User.add_to_class('email_isvalid', models.BooleanField(default=False))  
User.add_to_class('email_key', models.CharField(max_length=16, null=True))  
User.add_to_class('reputation', models.PositiveIntegerField(default=1))  
User.add_to_class('gravatar', models.CharField(max_length=32))  
User.add_to_class('email_feeds', generic.GenericRelation(EmailFeed))  
User.add_to_class('favorite_questions', models.ManyToManyField(Question, through=FavoriteQuestion, related_name='favorited_by'))  
User.add_to_class('badges', models.ManyToManyField(Badge, through=Award, related_name='awarded_to'))  
User.add_to_class('gold', models.SmallIntegerField(default=0))  
User.add_to_class('silver', models.SmallIntegerField(default=0))  
User.add_to_class('bronze', models.SmallIntegerField(default=0))  
User.add_to_class('questions_per_page', models.SmallIntegerField(choices=QUESTIONS_PER_PAGE_CHOICES, default=10))  
User.add_to_class('last_seen', models.DateTimeField(default=datetime.datetime.now))  
User.add_to_class('real_name', models.CharField(max_length=100, blank=True))  
User.add_to_class('website', models.URLField(max_length=200, blank=True))  
User.add_to_class('location', models.CharField(max_length=100, blank=True))  
User.add_to_class('date_of_birth', models.DateField(null=True, blank=True))  
User.add_to_class('about', models.TextField(blank=True))  

I think it's possible that the .add_to_class method may be part of this problem.

Answer 1

It would have been helpful if you showed the Django model that is related to this problem. From the field names in your SQL this model appears to inherit from contrib.auth.models.User, is that true? If so, did you happen to duplicate a field name defined in the User model?

Update: To put it more bluntly, I was asking that you edit your question and include the specific model you declared in your models.py file. It's almost a certainty that you have declared two fields with the same name. The last field declared in auth.models.User is date_joined so both definitions of about should show up in your models declaration.

Actually, on closer inspection of your SQL again, you not only have two columns named about, you have about 14 duplicated field names in there. Something ain't right.

To paraphrase Jerry McGuire: Show me the model! :-)

Second Update (Working with a 1 beer handicap (and it's good stuff!))

I have never used add_to_class in this way. Why did you go this route and not use one of the more standard Django model inheritance techniques? Or, since this is User you are dealing with, why not use UserProfile. It can be a little bit clunky but it works just fine.

Third Update

Ah, Domini Nabisco, my son -- I didn't realize you had inherited this from someone else. A word of caution: because this changed a fundamental class in Django (in a decidedly non-standard way) you may need to look at every single reference to user in both the .py files and the templates. Depending on the size of the inherited project this can be a huge PITA, particularly when it's missing tests. Is there any chance you can hunt down, I mean locate the person responsible and get some more insight? It may be the shorter route to getting things working. Good luck!