Situation 1 - Connecting the server to the database
There isn't an easy answer here. In order to connect, the server needs the password (or symmetric key, or private key or whatever). It must get it either from the disk or some external means (like an administrator typing it at startup). Adding some indirection, such as encrypting all the sensitive stuff under a master password, can add some convenience but otherwise doesn't change the situation.
Typically, it is fine to put the password or key in a file on a server. If you do this, make sure to set the permissions on the file so that only the users that need it have access to it. This is an excellent reason to have different processes on your system run as different users, and to set up separate roles/accounts and passwords for each.
Situation 2 - Users logging into the server from a remote computer
You are headed in the right direction here, I think. What it sounds like you're asking for is a secure authentication protocol. You want one that provides mutual authentication and prevents a man-in-the-middle attack by failing if such an attack is attempted. There are many to choose from of course.
It is also worth mulling whether your authentication should operate based on "something you know" (passwords) or "something you have" (public/private keys). Assuming based on your question that what we're looking for is passwords, two that I like are SRP and Kerberos.
SRP was mentioned earlier, and that doesn't get nearly the attention it deserves. SRP has the advantage that it doesn't require the server to know the password, or key, or anything that an attacker could use to gain access. If you broke into a correctly configured server using SRP and stole all the data, you'd still need to do something like a dictionary attack on each key individually before you had anything you could use to impersonate a user.
I also like Kerberos because it is supported by tons of software (I know Postgres supports it, I've only found mentions of mysql not supporting any good authentication technology) and has a system of 'tickets' that provides a single sign on capability. Kerberos needs some other technology to help strengthen its initial authentication exchange and SRP would be great for that but I'm not sure they've done that yet. Something about it making the KDC (key server) stateful I think.
Kerberos' weakness is that you have to be more wary of the server storing the keys. While it doesn't store the passwords in plaintext, it does store the keys, which are essentially hashed versions of the passwords. And while the client doesn't exactly send either the password or the key straight over when authenticating (this is a Real auth protocol after all), it does use the hashed password as the key, and so anyone else who knows the algorithm and knows the key could do the same. We say that the server stores a "password equivalent". As a result, all the manuals tell administrators to put the kerberos services on their own separate, locked-down boxes to minimize the chance of compromising their contents.
The nice thing is, once you settle on a strong authentication exchange, other good things typically fall out of it for free. You end up with both parties sharing a mutual 'secret' that can be used once for the duration of the session, never sent over the wire, and can't be known by a third party. Want encryption? There's the key, all ready to go. This is exactly how SRP-secured SSL is defined in RFC 5054.