Django loaddata ValidationError

Question

This thread got too confusing (for me) so I am asking the question again. I could not make the csv2json.py script mentioned in the original question work. I am just trying to find a way to import data to sqlite3 database.

Here's the model I am working with:

from django.db import models

class School(models.Model):    
    school = models.CharField(max_length=300)
    def __unicode__(self):
        return self.school

class Lawyer(models.Model):
    firm_url = models.URLField('Bio', max_length=200)
    firm_name = models.CharField('Firm', max_length=100)
    first = models.CharField('First Name', max_length=50)
    last = models.CharField('Last Name', max_length=50)
    year_graduated = models.IntegerField('Year graduated')
    school = models.CharField(max_length=300)
    school = models.ForeignKey(School)
    class Meta:
        ordering = ('?',)
    def __unicode__(self):
        return self.first

(I'll fix the duplicate "school" later.)

I have this data1.csv file:

pk,firm_url,firm_name,first,last,school,year_graduated
1,http://www.graychase.com/babbas, Gray & Chase, Amr A, Babbas, The George Washington University Law School, 2005

I have to put in "pk" and 1 manually as required by the script.

Then, I run the script and I get the data1.csv.json file:

[
    {
        "pk": 1, 
        "model": "wkw2.Lawyer", 
        "fields": {
            "school": "The George Washington University Law School", 
            "last": "Babbas", 
            "firm_url": "http://www.graychase.com/babbas", 
            "year_graduated": "2005", 
            "firm_name": "Gray & Chase", 
            "first": "Amr A"
        }
    }
]

I put this file in the fixtures folder in the app directory and run manage.py loaddata data1.csv.json

and I get the error

Installing json fixture 'data1.csv' from 'C:\~\Django\sw2\wkw2\fixtures'.

Problem installing fixture 'C:\~\Django\sw2\wkw2\fixtures\data1.csv.json': Traceback (most recent call last):

File "C:\Python26\Lib\site-packages\django\core\management\commands\loaddata.py", line 150, in handle for obj in objects:
File "C:\Python26\lib\site-packages\django\core\serializers\json.py", line 41, in Deserializer for obj in PythonDeserializer(simplejson.load(stream)):
File "C:\Python26\lib\site-packages\django\core\serializers\python.py", line 95, in Deserializer data[field.attname] = field.rel.to._meta.get_fie(field.rel.field_name).to_python(field_value)
File "C:\Python26\lib\site-packages\django\db\models\fields\__init__.py", line 356, in to_python_("This value must be an integer."))
ValidationError: This value must be an integer.

When I comment out the two lines in the script with pk in them I get this error message:

Installing json fixture 'data1.csv' from 'C:\~\Django\sw2\wkw2\fixtures'.
Problem installing fixture 'C:\Users\A\Documents\Projects\Django\sw2\wkw2\fixtures\data1.csv.json': Traceback (most recent call last):
File "C:\Python26\Lib\site-packages\django\core\management\commands\loaddata.py", line 150, in handle for obj in objects:
File "C:\Python26\lib\site-packages\django\core\serializers\json.py", line 41, in Deserializer for obj in PythonDeserializer(simplejson.load(stream)):
File "C:\Python26\lib\site-packages\django\core\serializers\python.py", line 77, in Deserializer data = {Model._meta.pk.attname : Model._meta.pk.to_pytho(["pk"])} 

KeyError: 'pk'

What am I doing wrong?

Edit:

I took out the quotes around the year to make it an integer but I still got the same ValidationError:

...
            "year_graduated": 2005, 
...

Answer 1

Exploring new solutions to a problem is more than OK.

But I think it is tremendous bad form to keep asking the same question in multiple threads like this.

You started with this question.

Then, even after you were given (by me) almost the complete script you needed to write, you posted an alternate question here. That is OK too, I guess. It is a different approach to solve the problem. But, lots of really good responses later, you claim that it confuses you (which is fine, not a problem asking for clarification) - and asks the same question again.

This, in my opinion, is not the right thing to do. It pollutes SO with multiple versions of the same question. You should have stuck with either your original question and asked for more help, or edited question 2 with the proper info so people there could help you.

I posted a revised answer to question 1 with a complete tested script that clearly imports the little snippet of CSV data you provided. See if that helps you.

I posted it there not to irk you, but because in the future someone else may need an example of importing a CSV to Django. That is the right question to hold a solution like the one I'm giving.

I wish had the editing power to close this question. Not only it is a dupe, but it is mildly insulting to the people that tried to help you on question 2.

Answer 2

(I'll fix the duplicate "school" later.)

Actually, that is your problem. The second definition of school as a foreign key will require it to be an integer, thus the error.

You can confirm this by dumping the schema of your table with

sqlite3 <database-file> '.schema wkw2_Lawyer'