What we do is to put it in a separate directory on the server (you could use something like /config, /opt/config, /root/config, /home/username/config, or anything you want). When our servlets start up, they read the XML file, get a few things out of it (most importantly DB connection information), and that's it.
I asked about why we did this once.
It would be nice to store everything in the DB, but obviously you can't store DB connection information in the DB.
You could hardcode things in the code, but that's ugly for many reasons. If the info ever has to change you have to rebuild the code and redeploy. If someone gets a copy of your code or your WAR file they would then get that information.
Putting things in the WAR file seems nice, but if you want to change things much it could be a bad idea. The problem is that if you have to change the information, then next time you redeploy it will overwrite the file so anything you didn't remember to change in the version getting built into the WAR gets forgotten.
The file in a special place on the file system thing works quite well for us. It doesn't have any big downsides. You know where it is, it's stored seperatly, makes deploying to multiple machines easy if they all need different config values (since it's not part of the WAR).
The only other solution I can think of that would work well would be keeping everything in the DB except the DB login info. That would come from Java system properties that are retrieved through the JVM. This the Preferences API thing mentioned by Hans Doggen above. I don't think it was around when our application was first developed, if it was it wasn't used.
As for the path for accessing the configuration file, it's just a file on the filesystem. You don't need to worry about the web path. So when your servlet starts up it just opens the file at "/config/myapp/config.xml" (or whatever) and it will find the right thing. Just hardcodeing the path in for this one seems pretty harmless to me.