django sitemap efficiency

Question

I saw on this page that it's possible to lower the limit on your sitemap so that it is paginated differently: http://stackoverflow.com/questions/2079786/caching-sitemaps-in-django

But when I try to generate my sitemap, it hangs and hangs, and never comes up.

Eventually, if I wait long enough, I get this error in Firefox: XML Parsing Error: no element found Location: http://sitename.com/sitemap.xml Line Number 1, Column 1:

My site has about 70K pages at present, so I'm using the index generator in urls.py. For some reason though it's not working. I'm guessing it is because my server lacks the power to generate sitemaps containing 70K links, but I'm not at all sure.

Does anybody have any insight?

Answer 1

One thing you could do it to split your huge sitemap into several files. Probably each file will contain a different content area of your site and you can use caching as probably not all sections update in the same frequency?

http://docs.djangoproject.com/en/dev/ref/contrib/sitemaps/#creating-a-sitemap-index

Answer 2

I finally figured this out. Turns out I had a misconfiguration in my urls.py. Ugh.