huge svn checkout made apache (dav_svn) consume all memory on server - any tips?

Question

Are there any settings in apache2/dav_svn/ssl/subversion on the server that would make the whole perform better (or at least not consume all memory) on big checkouts?

I'm especially interested in suggestions on how to keep memory consumption down.

Reason for this question:

Last week we had an issue with our svn server - it stopped responding to all requests, and the admin informed us that all memory had been taken by apache. We access our subversion repositories over ssl using mod_dav on apache.

The cause of the the memory starvation was one of my colleagues doing a big checkout of ~2.5GB that might have contained some files that were more than 500MB in size. I heard he tried multiple times, so maybe a couple of processes were left hanging on the server.

The server admin said that you should not store such big files in subversion, but I don't agree with that -- if they are essential to the project I would always want them in the repository.

Answer 1

"Perform better" and "consume less memory" are, in many cases, incompatible.

Usually performance is gained when more memory is used (e.g. caching). So while I understand your question is on how to keep memory consumption low, be aware that SVN will be slower as a result.

The server admin guy was right about keeping huge (are they binary?) files in SVN. SVN is mostly for source code - small text files. If you need somewhere to maintain version of related data, input, test-data, resources, 3rd party dependencies, etc. I suggest you create a binary repository on some shared location and use the directory structure + maybe symbolic links to reflect the versioning that exists in your svn repo.

There are real advantages to keeping your SVN as lean as possible (but not leaner!). SVN is notoriously slow doing checkout, due to its design (it's a very heavy operation in comparison to branching, for example, which is very cheap). Faster checkouts and blames, etc., translate into faster development cycles. So I find it worth while to maintain a non-code repository "manually" on some backed-up network share instead of keeping everything in svn.

Remember also that you can't really put everything you want in svn. You don't put the platform sdk there, right? So using an external repository can offer you a more complete solution by allowing you to put even more stuff there.

On an unrelated note: the cheapest solution probably is just to add more RAM. I've actually experimented with "RAM-disk" based SVN repository solutions and they're blazing fast.

Answer 2

This is a well known issue with SVN/Apache/SSL

There a some workarounds, but it seems there is no 100% fix at the moment. Read this issue for more information.

Some proposed workarounds:

• Use VisualSVN Server 1.6.3 (win32, non-SSL and with SSL)
• do not use SSL
• use SSLSessionCache shm:key directive
• use matching svnclient and server
• [..]