If I have multiple web server written in Erlang running (load balanced) and Mnesia is used for the backend database, what is the best way to upgrade the whole system to a newer version?
Even though Erlang supports hot code-loading, it is not something one must use. In your case, it seems easier to remove one node a time from the load-balancer, restart it running the new code and put it back in the load-balancer.
Taking down nodes is also something you must be prepared to do if you will want to hot-upgrade to new Erlang/OTP releases in your live system.
But the real issue that can bubble up for you are those concerns that come from mnesia. I think you should ask a new question with the specifics in what want mnesia to do. If there are no schema/table-changes, and you just want to take one node down and add it later. Or if you actually are introducing new tables or new columns in tables. Mnesia does provide the ability to add and remove nodes with table replicas, and it also rather uniquely supports multiple table definions through mnesia:transform_table/3,4
.
If you're just doing code upgrades, I wrote a an article about erlang release handling with fab. With this setup, you can do real-time code loading without having to restart nodes. Transforming the database, though, should be done from a single node, triggered separately from the release upgrade.