Subversion performance tweaks.

Question

We have a 3.5 Gb SVN repository recently migrated from a 6 Gb one. We keep all our in-house made software and there a lot of small shared components. We also do a lot of tagging. Projects keep a copy of it's binary dependencies, mostly libs. We cannot move to GIT right now.

The first impression on our developers was that Subversion is very slow, I keep telling them that history-related operations are, but there are advantages also.

The access is via mod_dav_svn with custom authentication. Authorization will be implemented with post-commit hooks as we outsource some proyects a year and require detailed security rules.

We want to optimize the access so:

• Did 'svn pack'.
• Implemented authentication caching.
• Are ready to evaluate svnserve.

The layout of our repository is as follows:

\root
 \proyect1
  \trunk
   <files>
  \docs
  \branches
  \tags
   \proyect1-1.2.3-beta
    <files>
   \proyect1-1.3.0
     <files>
   etc...
 .
 .
 .
 \proyectn

Are there other optimizations not hardware-related that showed success before? Can the layout of our files make any difference?

Answer 1

I will only comment on the parts I have experienced.

1) Firstly, it's big! Normally it is best to keep binaries and libraries outside of the repository even it is sometimes more convenient to have everything together. We had such projects (where I work) with files that had not their place in a repository and which were taking a lot of place. It was killing the server at each checkout or for important commits, the other projects were just fine.

Separating the projects is also a good idea in general, with svn:externals you have enough flexibility to link them if necessary, and you avoid the single point of failure issue, gigantic repository sizes, it is easier to make backups, and so on.

But you can't always choose those parameters.

2) Is it a Linux Apache server, or does it run on Windows? I've seen quite a difference between the two, the Windows version tends to be slower and is very sensitive to the configuration.

3) I would do a test with svnserve instead of an Apache mod_dav_svn. The load on the network is lighter, which could also be a chocking point when many people share the same server (depending also on the network configuration).

4) Are you using 1.5, or even better, 1.6? The filesystem has been improved, and if you have migrated from a previous version, performing dump/load pays off, ususally (check this link also). We did some tests with big repositories and gained a few percent, but there was a big variance in what we observed.

5) Also, you may consider this (from Version Control with Subversion) - although I've never tried this particular possibility:

As of Subversion 1.6, FSFS filesystems have several configurable parameters which an administrator can use to fine-tune the performance or disk usage of their repositories. You can find these options—and the documentation for them—in the db/fsfs.conf file in the repository.

6) The hook scripts may have an impact if they are badly implemented, adding extra latency to server interactions.

7) TortoiseSVN uses log caching, it is sometimes annoying (it tends to lock files that you are then unable to delete - can be surprising if you don't know about it) but provides users with faster responses when browsing the logs (besides being a well-integrated client on Windows).