Conquering Complexity, Eckel on Java and Python and Chunk Theory

Question

In the introduction to Bruce Eckel's Thinking In Java, he says, in 1998:

Programming is about managing complexity: the complexity of the problem you want to solve, laid upon the complexity of the machine in which it is solved. Because of this complexity, most of our programming projects fail. And yet, of all the programming languages of which I am aware, none of them have gone all-out and decided that their main design goal would be to conquer the complexity of developing and maintaining programs.

In the second and later editions he adds this footnote (circa 2003):

I take this back on the 2nd edition: I believe that the Python language comes closest to doing exactly that. See www.Python.org.

I am a dabbler with java, with a background in Delphi (Pascal), C, C++, and Python. Here is what I want to know:

1. What exactly did Eckel consider when he called Python 'better' at conquering complexity, and are his thoughts on track with others who have used both?

2. What do you think about conquering complexity? Is the shorter and more terse syntax of Python a key way to conquer complexity (and thus, for instance, Jython might be a nice bridge of Java's great libraries, and Python's terse syntax), or is the strong-typing-mentality of Java, which inherits this idea from C++, which inherited that idea from simula, I think it was, a key to conquering complexity? Or is it the Rapid Application Designer (think Delphi, or for Java, the excellent free NetBeans window/form designer tools) or components, or beans, or J2EE? What conquers all, for you?

This is already tagged subjective. [edit]

Note: More on Bruce's thoughts, on why he loves Python are found here. A key quote from the article:

Bruce Eckel: They say you can hold seven plus or minus two pieces of information in your mind. I can't remember how to open files in Java. I've written chapters on it. I've done it a bunch of times, but it's too many steps. And when I actually analyze it, I realize these are just silly design decisions that they made. Even if they insisted on using the Decorator pattern in java.io, they should have had a convenience constructor for opening files simply. Because we open files all the time, but nobody can remember how. It is too much information to hold in your mind.

So, chunk theory. By the chunk theory metric, Python kills everybody else dead. I'll grant him that. But what is the metric you use? I would like to particularly invite people to stand up for Java, and oppose Bruce, if you care to.

[Please do not vote to re-open, this subject is inherently incendiary, and my gaffes have made it more-so. I agree with the moderators.]

Answer 1

For me, switching from Java to Python was a big win. I can write code faster, with the same or fewer bugs, and modify it more easily. The code also remains much more readable, so when I come back to it after a couple months I can figure out what it's doing faster (and rewrite it without too much trouble when I can't).

Java, being strongly typed, requires a lot of work up front to design and maintain correct type definitions. If you declare a variable as an int, and then decide it should have been a float, you'll need to change that type throughout your program. Are you storing that value in an array? You'll need to change that type declaration, too. Decide to refactor several classes to share a common interface? You'll have to change function definitions throughout your codebase to handle it. If you have a particularly complicated design, you'll find yourself having to deal with a lot of those issues.

Python also has a lot of support in the language for changing how certain things work. Python decorators, for example, can abstract away a lot of complicated code (dealing with caching, or registering functions) keeping code maintenance down. Sophisticated IDEs can help maintain your code, but you're better off starting with a less complicated language.

Answer 2

And yet, of all the programming languages of which I am aware, none of them have gone all-out and decided that their main design goal would be to conquer the complexity of developing and maintaining programs.

Almost every single language is based on conquering complexity. There is no other aim worth pursuing. Assembly, C++, Java, Python, and virtually every other language in existence is based on making programming easier.

How is python good at conquering complexity?

Python definitely has some of the most intuitive syntax of any language IMHO. Its use of block indenting solves a lot of problems, and most of it is as close to plain language as you should get. M = [x for x in S if x % 2 == 0] is a good example, see any python book for countless more.

Is the shorter and more terse syntax of Python a key way to conquer complexity?

I believe the simple syntax of python is a good way of conquering complexity. However, that's the only definitive answer I can give to your other query:

What do you think about conquering complexity?

You are asking the question that is the entire core of language theory, which encompasses battles that will probably rage until the end of time. static vs dynamic typing is one such debate. There are mounds of other developments in language theory like Procedural vs OO vs Functional languages and Aspect Oriented Programming that try to simplify programming. Look at the latest (or any) release of a language to see some examples of what's being done to 'conquer complexity'. There is never going be one definitive answer, and a full discussion of each approach would take a few months to read, and would probably completely change by the time your were done. :D

Answer 3

Bruce Eckel: They say you can hold seven plus or minus two pieces of information in your mind. I can't remember how to open files in Java.

I can:

new FileInputStream(filename);

I've written chapters on it. I've done it a bunch of times, but it's too many steps. And when I actually analyze it, I realize these are just silly design decisions that they made. Even if they insisted on using the Decorator pattern in java.io, they should have had a convenience constructor for opening files simply.

This is solved in a matter of minutes by writing that utility method with a simple api. And it has been. If that is the strongest criticism than can be directed at Java, I remain distinctly unimpressed.

Answer 4

I think Bruce was taking his cue from Fred Brooks, who talks about complexity in his essay "No Silver Bullet", and describes two types. The first type is inherent in the problem you are trying to solve, which he calls essential complexity and is the same regardless of what language you use. The second is the complexity added by the tools and languages we use - all the stuff you have to think about that does not directly add to solving the problem. By this measure Java has a LOT more complexity than Python. The simplest example is the canonical Hello World program. In Python (and several other languages) it is one line:

print "hello World!"

In Java it is

class HelloWorld {
  static public void main( String args[] ) {
    System.out.println( "Hello World!" );
  }
}

most of which has nothing to do with the task of printing "Hello World" and is basically noise.

There are several factors that IMHO add to the complexity of Java compared to other languages like Python.

1) everything must be in a class. This forces you to use the OO paradigm even when it is not appropriate, like the example above, and add lots of unnecessary boilplate class definitions.

2) Even though it forces you to use classes it is not fully object oriented. By this I mean that not everything is an object, such as primitive types and methods. In python you can subclass all the builtin types and pass functions & methods around like any other object.

3) Java is not functional - in fact it goes out of its way to stop you using a functional paradigm. Being able to pass functions around and create closures and lambdas can simplify a lot of code. The closest you can get in Java is to use anonymous inner classes for things like callbacks.

3) Java forces you to put in type declarations everywhere, which adds a lot of clutter without adding useful information. This is not just a static vs dynamic issue - there are statically typed languages like Scala that can infer the types 90% of the time and cut out all the noise.

4) Although Java forces you to use static typing, many (perhaps most) real world Java programs use dynamic type checking part of the time. Every time you do a cast from an Object to a specific type you are doing dynamic type checking - before generics were added in Java 5 this meant every time you used a container class for example. Even when using generic containers some of their type checking is done at runtime. Also every time you have an XML file with class or method names in it them somewhere in the code it has to do a dynamic type check to make sure it matches up with a real class. So many Java programs still have the alleged "dangers" of dynamic typing but with all the verbosity that Java's static typing forces you to add.

I could go on (and often do), but I will stop here with the observation that I have seen a lot of code that is simpler, cleaner and less complex in Python than in Java but none the other way around. If anyone can point me to some then I would love to see it.

Answer 5

Having earned an A in my "OOP With Java" course last semester, and self taught in Python for a few years now, plus being interested in programming in general, and having written several K lines in both languages (not counting the comments), including a Jack lexer/parser in Python (and seeing the equivalent performed -- or not -- in Java by my compatriots), I think I have enough experience to be worth at least a few cents anyway. Or not, but now that I'm done tooting my own horn you can decide for yourself ;)

First of all, I agree wholeheartedly that programming is about reducing complexity. I also agree that Java does not do a great job at reducing complexity.

For instance, to get user input from the command line?

Scanner input = new Scanner(new BufferedReader(System.in)); (I think? And it's only been 2 or 3 months since I've used that)

Python is raw_input() and in 3.0 it's just plain input()

How do you read from a file in Java? Well, honestly I don't know. We didn't do much of it, and it was worse than Scanner. As has been mentioned, the complexity of Java is above and beyond that of any other real language I know, but not quite so bad as "that" language.

I think the main problem with the complexity of Java is found in the example meriton uses in defense of Java. Rather than importing awesome community APIs and implementing them as part of the language, Java has a "core" language and APIs are just an extension of that. What I find interesting is the wealth of tools that Java has to build APIs and documentation for said APIs. I mean, you write your code, put in some certain comments, and (at least in Eclipse) select an item inside a menu and you have generated some beautiful javadocs. Even though it's wonderful to have such great documentation, I wonder if it didn't actually evolve out of necessity because looking at a program gives you no clue what Java is doing. Scanner? Buffered Reader? WTF?? There are layers and layers of added complexity that could (and some argue should) be abstracted away. Heck, I can open and read a file in assembly with less fuss than Java. Of course comparing a few lines is really rather trivial and fairly worthless, but the point is that Java often introduces complexity, rather than solves it.

Here is the biggest reason that I feel Python has the advantage: useful types/functionality is built-in to the language, and they tend to be very well named (and at least if they're not super well named, they at least have some hint).

For instance, let's say I want to open a comma separated file (without using extra APIs or imports), and store each element in a generic collection.

In Python (2.6+, or 2.5 with a import from future) it's literally two lines:

with open('myfile.csv') as f:
    print f.read().split(',')

done.

In Java I don't think you can do it without importing external classes. Of course in reality I think either language just goes to preference, and possibly either genetics or training.

Some people feel that static or dynamic types introduce the least complexity. I fall into the latter camp, but I understand the argument. After all, your compiler will complain if you try to pass a different type, and you always know what something is supposed to be, because it's either explicitly declared or cast as such. Of course that adds the complexity of the casting operation (minor as it may be), but it benefits from not having to wonder "How in the world did the integer I passed in turn into a float|string??"

But when it comes down to it, most people can take a look at a Python program and understand what's going on. Novice programmers to extremely advanced programmers will be able to comprehend the program and purpose by looking at it. Heck, I just wrote a plugin for Bazaar by a combination of reading the (poor) documentation and reading the code for the Bazaar built-in functions. It took relatively little effort, and they even have a few custom definitions. The same thing for some golly scripts. When coding stuff in my Java class, I was also able to understand some other classes. However, I think I was at a severe advantage in that class because the Java concepts are very similar to Python concepts. Or vice versa, whichever way you prefer. (Or they're both similar to Lisp concepts ;)

I guess honestly I think the complexity is just the learning curve. Python has a very shallow learning curve, with the power being an inverse function to the learning curve. Java has a steeper learning curve, with its power curve having a linear (2x? 3x?) relationship. Once you've learned the language and the underlying concepts, the complexity is reduced to near-zero for both languages, I think.