tags:

views:

48

answers:

1

I'm working on a simple CMS (based on Django, not that it matters) similar to Jekyll and Hyde, but dynamic instead of static. The idea is that the server has a copy of the repository, I can push stuff into that, and the CMS will automatically pick up the new content.

Let's say that Markdown-formatted blog posts in my repository follow this file naming scheme:

/blog/2010/08/14/my-blog-post.md

Internally, the processed files will be cached in a SQLite database under a unique ID, for easy searching and fast serving.

The problem is in constructing the URLs in such a way that they can be mapped to files in the repository. I see several options:

  1. /blog/2010/08/14/my-blog-post
    If I simply map (part of) the URL to a filename, renaming a file will break all links pointing to that file. The content admin can leave a symlink in place of the old file, which the CMS can map into a HTTP redirect, but this requires work that is easy to forget.

  2. /blog/2010/08/14/271-my-blog-post
    If I include a database ID in every URL, clearing or rebuilding the cache will invalidate all IDs, which is even worse. I would like the git repository to be the only thing representing the site's contents; everything else should be reconstructible from that.

  3. /blog/2010/08/14/528dc05-my-blog-post
    The only thing uniquely identifying a file in the repo over time, as far as I can tell, is a pair (filename, SHA1). That file is guaranteed to exist in that commit, and we can trace it to the current HEAD through the git log.
    (I won't include the full SHA1, but just enough to make collisions sufficiently unlikely. Will do the math later.)

My question is twofold:

  • Is there an easy and fast way in git to track a (filename, SHA1) pair through renames to the corresponding filename in the current HEAD?

  • Is there a better way to accomplish my goals: not breaking existing URLs, but still allowing for renames and cache rebuilds?

A: 

Easy/fast? Not sure, but I don't think so. Git tracks the contents of files as blobs. The filenames of those blobs are then stored in tree objects. Then, commits point to tree objects, and add some metadata like committer, datetime, and a parent commit.

I don't think Git actually stores renames as such, it's merely a difference between trees pointing to the same blobs.

I think the best you can do is to have /path/to/file as URLs, and when you don't find that file in HEAD, iteratively scan backwards in the history to find the commit where there was one.

If you're going to be doing this kind of repository level stuff, I recommend you pick up a copy of Peepcode's Git Internals, which quite clearly explains the inner workings of a git repository.

Marten Veldthuis
You can use `git log --follow <file>` to trace history for a file through renames, but it does go present->past, so you'll have to have committed the new version of the file, and track it back to the original name. And of course, rename detection will fail if the file gets mostly rewritten at the same time it's renamed.
Jefromi
@Jefromi: Thanks, that's almost half the answer. Failure of rename detection is something I'm willing to live with.
Thomas
@Thomas: Another thought - when you're just about to commit, you could use `git diff --cached --name-status` to see what renames are about to be committed.
Jefromi
Based on another idea of Marten's, I'm now doing things slightly differently. I'm storing the SHA1 of the latest commit in the database, and compare HEAD's SHA1 to that. If it doesn't match, I process the commits in between, one by one. Git's rename detection allows me to put "symlinks" in the database from the old to the new location of the file. Remaining issue: Git 1.7.0.4 on Ubuntu Lucid has a change blindness bug. It does not detect moves of binary files, but doesn't flag them as an add/delete either... Fresh build works fine, though.
Thomas