When coding and reviewing code, it's easy to spot places where a design pattern could be used. A chain of command here, a strategy there... It's tempting to dive in and apply patterns even though a better solution might be a switch or some simple if's.
Are there some rules or tips you've found valuable to estimate when to do the actual refactoring?
Wait until it becomes too difficult to add features? Wait until the third time you have to change the code? First time you need a hack?
If the code is readable/understandable and isn't likely to be changed or extended in the future, you may want to leave it the way it is.
But if the code will change, or you need to build a workaround for it at some point, you're better off refactoring it right away. And it's a good practice to check-in code cleaner than it was before.
Keep in mind though, it depends on the return on investment. Simple refactoring may cascade into other parts of the code, that then have to be refactored as well. If the code base won't be under constant development in the future, it may not be worth spending too much time refactoring it.
My thinking is most useful refactorings happen when working on a new feature or fixing a bug. When you are writing new code or fixing a bug, you need to check in an improved version of the code (if it could be improved). So while doing this if a design pattern improves the code ( readability, more extensiblity, testablity) you refactor to the pattern.
Refactoring to patterns when there is no other motivation for it has little value IMO. There really needs to be a problem out there to solve for the patterns to be helpful.