I know that being open source does not necessarily makes a program more/less secure than closed source (let's assume this neutrality, to keep flames out of this post). Fact is: since the source code is open, everybody knows your defaults urls, default administrator logins, etc.
I'm using Wordpress and Joomla in some projects of my clients, and I always try to create some kind of additional security. Excluding always updating your files to latest version, what do you usually do to add more security in this scenario? Some of my thoughts:
I always change the "admin" name when applicable;
I would like to don't explicity say which technologies I'm using, but since I want to promote the cms (I think is the minimal I should do), I just don't say the exact version so attackers don't know which exact vulnerabilities they can attack (wordpress automatically creates a meta tag in html saying "Wordpress 2.8.4" for example);
Set correct permissions in directories, and bash scripts in my server that run everyday at 0h setting 755 to directories I may have changed to 775 during the day and forgot to turn back;
When applicable, I set apache configuration to limit ips.
What else should I try to do? What "out of the box" solutions do you usually do to your installations?