Sometimes Apache with mod_dav_svn starts to consume a lot of memory.
Today Apache(httpd.exe) consumed about 600M.
And there was a lot of java.lang.OutOfMemoryError in Hudson when it tried to update working copies for some jobs.
What can I do in order to fix this problem?
It's probably not the answer you're looking for, but you could isolate subversion and Hudson on separate machines or virtual machines. It's much more complex to dive into an exercise in capturing subversion usage and keeping track of apache memory use.
A couple of other thoughts:
If you have large files in the repository, rethink whether you really need to keep that under source control. (I seem to recall that we had issues with large files in the early 1.5 subversion releases. Sorry, but I don't remember more details.)
Google ("subversion apache memory use") has some results about possible memory leaks, but that may or may not apply to whatever version you're running.