Django - dumpdata truncate to last n lines.

Question

Hi all,

Does anyone have a simple solution to using (or modifying) dumpdata to trucate a simple table down to the last n lines. I like to use dump data for test fixtures but the data size has gotten so large this doesn't make sense. BTW - I didn't design the table I'm just the sucker which has to deal with it.

For those who may ask how the structure looks here it is.

From Django Side

class GridResourceUsage(models.Model):
    """Sampled point in time of license usage for individual grid resource. Includes who and quanity."""
    timestamp = models.DateTimeField(db_index=True)
    grid_license_resource = models.ForeignKey(GridLicResource)
    total    = models.IntegerField(default=None, null=True)
    limit    = models.IntegerField(default=None, null=True)
    free     = models.IntegerField(default=None, null=True)
    intern   = models.IntegerField(default=None, null=True)
    extern   = models.IntegerField(default=None, null=True)
    waiting  = models.IntegerField(default=None, null=True)
    def __unicode__(self):
        return str("GRU-" + self.grid_license_resource.name) 
    class Meta:
        ordering = ['-timestamp']
    @models.permalink
    def get_absolute_url(self):
        return('hist_res_id', (), {'resource': str(self.grid_license_resource.name), 'id':str(self.id)})

From MySQL side

CREATE TABLE `gridresource_gridresourceusage` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `timestamp` datetime NOT NULL,
  `grid_license_resource_id` int(11) NOT NULL,
  `total` int(11) DEFAULT NULL,
  `limit` int(11) DEFAULT NULL,
  `free` int(11) DEFAULT NULL,
  `intern` int(11) DEFAULT NULL,
  `extern` int(11) DEFAULT NULL,
  `waiting` int(11) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `gridresource_gridresourceusage_timestamp` (`timestamp`),
  KEY `gridresource_gridresourceusage_grid_license_resource_id` (`grid_license_resource_id`)
) ENGINE=MyISAM AUTO_INCREMENT=2891167 DEFAULT CHARSET=latin1;

Answer 1

I'm not sure if dumpdata can do what you ask for.

Can't you just create a queryset and serialize it? A bit silly example but it should work.

# Create queryset you want
n = SomeModel.objects.count()-1 # Row offset counting from the end
queryset = SomeModel.objects.all()[n:]

# Serialize that queryset to json in this case
from django.core import serializers
data = serializers.serialize("json", queryset)

# And write it into the file
f = open('data.json', 'w')
f.write(data)
f.close()

You can wrap this up in management command and use it more or less the same way you are using the dumpdata command. (You could also look at source of dumpdata for ideas)