Browsers are like people - they get old, they get bloated, and they get ditched for younger and leaner models.
Firefox is not just a browser, it's an ecosystem.
While I feel that recent versions are quite bloated, the core product is generally stable.
However, firefox is an ecosystem/platform for:
1) Badly written plug-ins
2) Badly written JavaScript code that executes within it.
3) Adobe flash as a platform for heavyweight video and for poorly written ad scripts such as 'hit Osama bin Laden with a duck to reduce your mortgage rate and receive a free iPod* (participation required).
4) Quicktime and other media player.
5) Some embedded Java code.
The description of a memory leak suggests a script running amok or a third-party tool requesting more memory. If you ever run Flash on a Mac, that's almost a given along with 90% CPU utilization.
The goal of most programming languages is not to save you but to give you tools to save yourself. You can write bad and bloated code with memory leaks in any language, including ones with garbage collection. Third party tools are usually not as well tested as the platform itself. Web pages that try to do too much are also not uncommon.
If you want to do an experiment to demonstrate this, get a mac with Firefox and go to a well-written site like Stack Overflow and spend an hour. Your memory usage shouldn't grow much. Then spend 5 minutes visiting random pages on Myspace.
Now let me try and answer your questions based on my guesses since I'm not familiar with the source code
- When I close all tabs, why
doesn't the memory usage go all the
way down?
Whereas each browser instance is an independent process with its own memory, the tabs in a single window are all within the same process. Firefox used to have some sort of in-memory caching and merely closing a tab doesn't clear the relevant information immediately from the in-memory cache. If you reopened a tab to the same site, you might get better performance. There was some advanced option to allow you to disable it, something like browser.cache.memory.enable. Or just search for disabling the memory cache.
* Why is there no limits on extensions/themes/plugins memory usage?
For the same reason that Windows or Linux doesn't have a vetting process on applications you can run on them. It's an open environment and you assume the risk. If you want an environment where applications and extensions are 'validated', Apple might be the way to go :)
* Why does the memory usage increase if it's left open for long periods of time?
Not all calculations and actions in a script have visual manifestations. A script could be doing some stuff in the background (like requesting extra materials, pre-fetching stuff, just bugs) even if you don't see it.
* Why are memory leaks so difficult to find and fix?
It's about bookkeeping. Think about every item you ever borrowed (even a pen) or that someone borrowed from you in your entire life. Are they all accounted for? Memory leaks are the same way (you borrow memory from the system), except that you pass items around. Then look at the stuff on your desk, did you leave anything lying around because 'you might need it soon' even though you probably won't? same story.