tags:

views:

38

answers:

2

When using git fetch to fetch refs from one (very large) repository to another one on the local machine, git upload-pack takes a very long time to create pack files. In the local case there's not such a need to minimize the amount of data transported, and I don't care about disk space lost by losing delta compression, so ideally I'd prefer for the missing objects to be copied rather than packed and then imported. Is there any way to tell git fetch to just copy the missing objects when using the local transport?

Or, more generally, is there a way to suppress the generation of pack files globally? Really I just want to use git as a versioned filesystem that doesn't use up extra space for identical files - packing and repacking seems to be the time-consuming step that makes this awkward.

Incidentally, I've spent some time trying to optimize config options so that repacking doesn't take so long (nor start thrashing) so I don't think the answer is "use these config options and packing will happen much faster" - however, perhaps I've got that all wrong, so just to be clear, the config options that I'm typically using (on a maching with 2 GiB of RAM) are:

core.deltacachesize=1
core.packedgitwindowsize=16m
core.packedgitlimit=128m
pack.packsizelimit=512m
pack.windowmemory=200m
pack.deltacachesize=200m
pack.window=4
pack.compression=3
pack.threads=0
gc.auto=0
gc.pruneexpire=never
receive.autogc=false
A: 

Maybe (not tested) setting up a http-backend for your first repo (the one from which you are fetching).

This kind of server has a setting which could be of interest in your case:

http.uploadpack

This serves git fetch-pack and git ls-remote clients.
It is enabled by default, but a repository can disable it by setting this configuration item to false.

VonC
Thanks, VonC, but doesn't setting http.uploadpack=false just disable fetching from the smart HTTP transport entirely? (That's what the test in t556x_common suggests to me, anyway.) That said, I think the actions of the dumb HTTP transport may be what I want to mimic but as an option for the local transport. I should at least try to set up a lightweight HTTP server to see whether using the dumb tranport would be a speed improvement...
Mark Longair
@Mark: that may very well disable the smart HTTP transport. But even with a simple local transport, that is worth a try.
VonC
+1  A: 

Perhaps you could use alternates (alternate object storage) mechanism; this would allow sharing object database with other local repository, and then not having to pack them.

To set up this, use 'git clone' using either --shared option if cloning from local repository, or --reference <repository> if cloning from remote repository but when you have similar repository locally, or just edit .git/objects/info/alternates file.

Jakub Narębski
Jakub: thanks for the answer - I've used alternates before for this, but in the current case I do want to merge the object databases so that I have a standalone repository. As I understand it, making a repository with alternates into a standalone repository would still involve `git repack` and so the length packing procedure.
Mark Longair