Do you look "under the covers" of libraries that you're using when developing?

Question

While reading this interview with Bjarne Stroustrup on education. He had this to say:

I’m not sure how much of the problem is Java itself and how much is the emphasis on using libraries, though. The trouble is that Java has in many places been used to dumb down the curriculum while at the same time increasing the apparent level of delivered goods. It is good to be able to (quickly) build new things by calling libraries, but often that’s not a skilled, challenging job. If that’s all you have seen, you are completely lost when faced with a job for which a pre-packaged solution does not exist. This is just as bad as trying to build everything from scratch yourself. I think the problem is one of attitude, more than an issue of programming language.

I've found that many libraries and toolkits abstract away and hide the mechanics of what they are doing under the covers which is an excellent way of allowing developers to create a complex solution quickly. Think of what you are able to do with a few simple STL containers and algorithms.

But this simplicity is a bit of a two edged sword. I've noticed that many developers just "look at the tin" to see what a library/toolkit is capable of without really looking deeper to understand what is going on under the covers.

As Bjarne says above:

...you are completely lost when faced with a job for which a pre-packaged solution does not exist.

BTW This question, and especially its relation to Java, is also featured heavily in this interview with Robert Dewar.

So. How much time do you invest to understand what's really happening inside the libraries that you use?

Edit: Not knowing what is happening in the libraries and toolkits that you use is leaving your self open to "programming by coincidence" as The Pragmatic Programmers call it. Everything is fine when it works, but when it breaks, you don't know what's actually going on under the covers and you'll be completely lost trying to find out where it's broken. Never mind working out how you're going to fix it!

Edit Part II (The Revenge) (-: I'm thinking the answer below about markets and food is spurious because surely you don't have to understand what the market is doing to provide you with the food? You wouldn't have to debug your interraction with the market if they couldn't provide you with the result you wanted.

It's a question of complexity of the solution provided by the 3rd party. That is, buying some food versus a forward iterator over a container of objects.

If your 3rd party library stops working in the way you were expecting, and you don't know what's actually going on under the covers, you'll be pushed to debug why the library isn't doing what it's supposed to do when working in conjunction with your code. Any previously good behaviour was totally due to coincidence.

cheers,

Rob

Answer 1

As much time as possible. I have been routinely been bitten on projects by not understanding how something works in a library and trying to do something that is outside of the bounds of its capability. I never assume that something "will just work" anymore.

Answer 2

Sure.

I find myself looking at both the source code for my JDK and the Spring framework that I use. The latter is an especially good teacher, because the quality and attention to detail is so high.

I'm not sure I'm buying the argument, particularly the special citation of Java. The trend in all languages is higher and higher abstraction. We're all further removed from those base abstractions that are lamented here. Are you better off now that you have the TCP/IP stack to keep you from worrying about the networking details when you fire up your browser to log into StackOverflow? Of course you are.

Answer 3

I have a kind-of old jdk source on my system. I rarely look at it - usually when I can't figure out exactly what it's going to do in a certain case that isn't mentioned in the documents, or when I want something that is close but not exactly like the classes provided. For instance, about 8 years ago I used the Observable class to make an RMIObservable class that fired its events over RMI.

Answer 4

As Bjarne says above:

"...you are completely lost when faced with a job for which a pre-packaged solution does not exist."

Sure. I'll be screwed if they stop bringing food to my grocery store, but that isn't inspiring me to start farming and raising animals in my backyard.

Answer 5

First of all, that Stroustrop quote is completely wrong, I think. Abstracting away complexity is exactly what libraries are for. Write my own email client in java because it's too lazy (in some sense) to just write my own from scratch? Not in a production environment, that's for sure.

As for looking under the hood, generally when:

• I don't have a clue how it's doing it's advertised thing
• I don't trust the quality of the code, it's not a well-known/used thing, and I want to get some sense of how well I can trust it.
• To learn something (sometimes the sun java src can be a good reference)

Mostly I don't actually find anything definitive, except for the last reason. Maybe it's just a form of procrastination.

(Response to your edit on "programming by coincidence")

Yes, on some level, not understanding a component is programming by coincidence, but you can clearly take this to an absurd degree. Do you understand the JVM completely (that is, well enough to rewrite it)? How about the operating system?

Most of the libraries I use day-to-day are widely used. My trust in them is somewhat proportional to their use - the more other people using the same library to do the same thing, the less likely I'm going to find something broken using it.

Answer 6

Joel's Law of Leaky Abtractions rings true in this discussion.

Answer 7

The only time I would look at the implementation of a library would be:

• If it lacks documentation (although in this case, it's likely I wouldn't have chosen the library in the first place)
• If it's not behaving as the documentation says it should (I would be searching for a bug)
• If it doesn't do what I need it to, and I want to crib ideas to create my own code that does.

Otherwise it's right and proper that a library should be a black box.

To those who think otherwise, I ask where you draw the line? When you're writing a web application, do you read the source code for the TCP stack? The firmware for the Ethernet card?

Answer 8

I've poked around in library code when things aren't working, and I can usually figure out what's wrong.

Answer 9

frameworks are literally everywhere .i liked to do a lot of read on frameworks like jquery and mootools but then i felt it was making me lazy - and stopped more studies on them .

Answer 10

One point nobody else brought up:

If you go to the source code for answers, there's a risk that you develop against the implementation rather than the specification.

For example, what if I look in the source for a library, and note that the way it gets a set of results means it will always be sorted. So I write my code to use the ready-sorted results. But that detail might be due to an arbitrary choice the developer made.

Later, maybe I swap out to a different implementation of that library (think moving from MySQL jdbc to Oracle jdbc), or just a new release of the same library. Now the source is different, and the results are no longer sorted. My code breaks as a result.

By working to the specs, not the implementation, you avoid this problem.

(OK, you're more likely to notice that results are sorted by empirically observing the library's behaviour. A more realistic example might be choices you make as an API user that are based on the way a list is implemented in the library.)

Answer 11

I've fixed bugs in libraries before when they didn't work as expected. I reported the bugs to the author and low and behold, I never got a reply. I think it comes down to "reinventing the wheel" as well. A lot of developers I know promote the idea of "if it works, and does what you need, use it as it will save you time and money!" which is all well and good, but again, I wouldn't know how to write half the libraries I use. I don't think this should just be applied to Java though, there are a whole hoard of languages out there which would suffer in exactly the same way.

Answer 12

I spotted this phrase in the quote: "...This is just as bad as trying to build everything from scratch yourself..."

Now Bjarne is having it both ways. He provides no guidance whatsoever as to where to draw the line. Is anybody whose line differs from his "bad"?

I would argue that knowledge is better than ignorance, and knowing what's going on under the covers is a terrific thing, but the blanket criticism feels off-base.

Maybe Bjarne is still bitter because Sun's marketing for Java trumped the momentum he was building for C++ in the middle 90s. He's left Bell Labs for the academic life, too. It's natural to reflect on one's legacy at this time in his life.

Answer 13

How deep to you want me to go? Should I study JVM source code? All supported operating systems? Microprocessor design? Yeah, I probably should, but I can't.

Bjarne is talking about education. The point is that you have to learn solving programming problems (if this skill can be learned, and I doubt).

If you have it, looking at libraries source has no additional value.

I see their source only when debugging.

Answer 14

I find myself digging into the library code all the time. Not matter how good the documentation is I sometimes find it easier to learn what is going on by reading the code.

Ironically I find myself resorting to reading the code more often in situations where there is too much documentation (e.g. Spring).

I generally use Jadclipse to decompile the libraries and only bother to download the actual source code if I think I need to step through the code with the debugger.

Answer 15

Of course I look under the covers. How else will I improve my programming skills if not by reading and understanding code that seems beyond my grasp?

People who enjoy programming are primarily the ones who will be looking under the covers of libraries. The people who are in it for the money and nothing else are just content to get the job done.

Answer 16

More times than I can count I've found the "bug" in my code had to do with something a Microsoft dev screwed up.

And on several occasions I've heard people say something really stupid like "There just aren't any bugs in SQL Server"

My Point is, you can't trust anything you haven't written and personally debugged yourself. So, yes, I do look under the covers. Quite often as a matter of fact.