I mainly use CocoaMySQL and YourSQL on the Mac, but I am looking for suggestions for Windows, in which I can connect to the database and manage it using the interface.
I used SQuirreL for a bit - it worked quite well, and had drivers for several databases, so it is not purely limited to MySQL. According to the webpage, it works with any JDBC complaint database, and it also has a specific MySQL plugin that is bundled with the distribution to add MySQL specific functionality. Also, it's written in Java, so you're not limited to Windows.
It is hosted at Sourceforge, and the CVS can be browsed here.
I use Sun's MySQL GUI Tools. It has all the functions I've need so far.
Well, I'm not sure what kinds of things you're looking for, but for DB management, MySQL Administrator works OK. It's part of the MySQL GUI Tools bundle for Windows, OSX, and Linux.
Toad® for MySQL is my favorite: http://www.toadsoft.com/toadmysql/
phpMyAdmin is the one that I've used the most and is installed on most hosting solutions. It runs in the browser and therefore is completely platform independent and doesn't require that you open any ports to the outside world. They also keep releasing new versions which keeps it up to date. It's skinnable so you can allow other users into if needed. And it's FREE!