I'm assuming you refer to significant code blocks that seem to be of some value, or good algorithms in their own right, not just the odd line here or there.
In these cases, there's a natural tendency to keep the code, if you've made large changes keeping the old as comments acts as a form of code review. Anyone getting the latest version will be able to instantly see the large changes that were made, and if there is a problem, its much more easily seen what used to be there. If the problem is with the new code, then there's a very simple way to instantly check it.
So, given that, I tend to comment out old code for the first revision, and then only delete it when the code is subsequently changed again, by this time the change will have been 'bedded in' and not likely to be a cause of bugs.
These comments are a form of documentation, there's no need to remove them for any purist ideal of 'clean' coding. Remove them when they are no longer needed, keep them while they may be of value.