Not understanding what you want to do so much, but what about chmoding it to 777 or 775. :-/
Edit:
Noticed your on windows. You'd have to change the permissions. Don't know how windows does that :-/
First of all: "Crash" means an exception, right? Which one? Can you catch it and show it?
Second thing: You are copying subversion repositories, although you don't care about the subversion metadata? That's what svn export is about (no .svn directory in the target).
The answer to the first question is what you really need to provide. Maybe something grabs the .svn and locks some files. TortoiseSVN maybe (to give you nice overlay icons..)?
The only problem I see would be in this part:
// ... do some stuff to the cache to verify what I need.
If you do open the file and forget to close it, you still have exclusive access to it, and thus can't delete it later on.
Sounds like you don't have access to delete the file...
The above link says you get UnauthorizedAccessException when:
The caller does not have the required permission.
-or-
path is a directory.
-or-
path specified a read-only file.
It's one of those.
Your asp process clearly has write access to the directory, have you checked that it also has delete access?
Sounds like a permissions issue. Tricky one though as you obviously have write access if the File.Copy already works....
Only thing I could think of is the file still has a handle opened somewhere (as others have suggested perhaps in your do some stuff to the cache part).
As far as I recall, Subversion marks the files in its .svn subdirectories as read-only.
Try resetting the read-only attribute before deleting the file. I don't really know any C#, but a quick Google suggests this might do the trick:
File.SetAttributes(file, FileAttributes.Normal);