I would definitely recommend reading Joels essay on this:
"In Defense of Not-Invented-Here Syndrome"
For a dependency, the best metric I can think of would be "would the world grind to a halt if this disappeared". For example if the STL of C++ magically went away, tons of programs would stop working. If .Net or Java disappeared, our economy would probably take a beating due to the number of products that stopped working...
I would think in those terms. Of course many things are a shade of gray between "end of world" and "meh" if they disappeared. The closer the dependency is to potentially causing the end-of-the-world if it dissapeared, the more likely it is to be stable, have an active user base, have its issues well-knows, etc. The more users the better.
Its analogous to being a small consumer of some hardware component. Sometimes hardware goes obsolete. Why? Because no one uses it. If you're a small company and need to get a component for your product, you will pick what is commonly available--what the "big players" order in large quantities, hoping this means that (a) the component won't disappear, (b) the problems with the component are well known, (c) there's a large, informed user base and (d) it might cost less and be more readily available.
Sometimes though, you need that special flux-capacitor part and you take the risk that the company selling it to you might not care to keep producing flux-capacitors if you are only ordering 20 a year, and no one seems to care :). In this case, it might be worth developing your own flux capacitor instead of relying on that unreliable Doc Brown Inc. Just don't buy Plutonium from the Libyans.
If you've dealt in manufacturing something (especially when you're making far fewer than millions of them per year), you've had to deal with with this problem. Software dependencies, I believe, need to be understood in very similar terms.
How to quantify this into a real metric? Roughly count how many people depend on something. If its high, the risk of the dependency hurting you is much lower, and you can decide at what point the risk is too much.