Big picture questions regarding Django, Java, Python, HTML and web-site development in general

Question

I am trying to get a handle on the state of the art regarding web site development and have several questions. Maybe I'll end up finding most of the answers on my own. I come from a background of C++ and Windows development, and generally I am befuddled by what seems to be the ad-hoc nature of web development.

I focussed in on Django, after online research of it and Ruby (on Rails). From what I read, ROR tries to do everything for you behind the scenes and so therefore is slow and unscalable (and overhyped and not ready for prime time). So I have gotten into Django - downloaded Python and Django, the source from a complete Django site, got it running, and so forth.

And the first thing that surprises me about Django, is that there does not seem to be any innovation to speak of regarding actual presentation. All the innovation would concern database issues, business logic, reusability of code, etc - but not actually anything new regarding innovative visual controls or graphics for a web-site. When you build a Django view or template, it will still be making extensive reference to html from what I can see (And presumably also Javascript - but I haven't actually even seen any Javascript in Django templates yet.)

And I would have thought previously that html might be analagous to assembler, so a conventional application developer from years past might know and occasionally even use a little assembler, but generally would hardly ever use it, whereas from what I can see, html (and also CSS and javascript) still have to be mastered and written continually by every web developer, whether they're using Django or anything else. Is that a true statement?

There is one site in particular that would to me as an end user represent the state of the art for web sites and I would be curious as to what the foundations of such a site might be. That site is chess.com, and there are all sorts of facilities for playing chess online with other users, user customization of their account with various graphical effects and so on. Is it most likely Java applets they're using for a site like that? How relevant would Django be for a such a site. Would Django be used in conjunction with something like Flash or even Java applets? Also when a site like chess.com is ported to a mobile device, what is used to write it - the same development tools as for the desktop or something completely different (Yes, I have a lot of catching up to do.)

Are there in fact complete websites written solely in Java, perhaps using very high level Java API's? Why would someone say (as I read somewhere) that they despised Java so that is why they had gotten into Ruby on Rails and Django.

And regarding Python (and also PHP) what is the justification for their existence? First of all, Python is much, much slower than say C++, being interpreted. Why are websites written in Python or PHP - is platform independence the sole issue here. I am incredulous that application development is much faster in Python than C++ (aside from the garbage collection issue - is that what the primary reason for Python is - garbage collection.)

So anyway, a bunch of newbie questions - will probably end up answering most of them myself if they're not answered here. Maybe they're relevant to someone else though.

Answer 1

1. Django wasn't trying to innovate in how web sites are displayed in the browser. Their goal was to simplify the process of building a web site. They could have taken on new ways of creating widgets in the browser as part of that goal, but they didn't. There was plenty of pain to relieve in the classic construction of websites.

2. If you are building web sites, you will be dealing with HTML. Your analogy to assembler is interesting, but in that analogy, no popular higher-level languages have emerged. This is likely because every higher-level language would of necessity impose some constraints on what could be expressed, and the web is not at the point of wanting conformity like that.

3. Python is an easier-to-use language than C++, and you really can develop faster in it. You should try it. Automatic memory management is one reason, but others are easy-to-use data structures, not having to ask permission from the compiler to do what you want, extensive third-party libraries to build on, and a clutter-free language in which to express yourself. About the speed: web sites are not compute-bound, they are I/O-bound, so the speed of the language rarely makes a difference.

4. About the ad-hoc nature of web development. You're coming from a Windows development background, where one vendor defined the entire environment, and did a good job of it. Web development is ad-hoc because the web itself is ad-hoc. No one group defined it, it's grown organically with contributions from many.

Answer 2

Hmm, you've asked a laundry list of questions here. I'll pick a couple of the important ones and answer.

As for the rationale for languages like Python... the truth is that many web applications are either I/O bound or database bound. When that's the case it doesn't matter much if the language you're using is not as fast as C++- the bottleneck is elsewhere. Beyond that an awful lot of the core Python routines and data structures are written in C. Python is used to drive the highest level of logic, but most of the work happens in fast native code. It might surprise you to learn that in my current job I write 3D animation software in Python. Of course a lot of what is going on behind the scenes was written in C++. There's a name for this: "Alternate Hard and Soft Layers." The reason we use Python is pretty simple- our choices are Python or C++ because of the APIs we write to, and we're several times as productive in Python. I would actually ask what the rationale for the existence of C++ is, but that's another subject (and flamebait ;).)

As for the visual presentation issues... traditionally there has been a pretty distinct separation between the server-side logic of an application and the in-browser logic, partially because the only language you can count on being in the browser is Javascript (and even then you can't actually count on it being on, leaving aside the existence of browsers like Lynx.) So you wrote your server logic in some sort of framework like Django, and then you used some Javascript framework to do your front-end stuff, and (in the last few years) you used something like AJAX to let them interoperate a bit more smoothly.

This is still pretty much the dominant paradigm, but things have started to change. A lot of server-side frameworks have started including facilities for generating Javascript in one way or another. And people have started writing compilers that translate from other languages to Javascript. One prominent example is Google's GWT, which translates from Java to Javascript: http://code.google.com/webtoolkit/ There are other examples of this sort of approach though. I did a lot of programming in a common lisp library called parenscript ( http://common-lisp.net/project/parenscript/) a few years ago, and there is the beginning of a Clojure ( http://clojure.org/) library for doing something similar. Also, one of the most interesting set of frameworks around is Seaside/Magritte ( http://www.seaside.st/ and http://www.lukas-renggli.ch/smalltalk/magritte) which use continuations to manage the state of widgets. There are similar frameworks for scheme and common lisp.

As for html being like assembler, I'm inclined to agree in cases where I am writing the html. I tend to use some sort of abstraction layer to generate html in those cases. But an awful lot of the html in the world is made by designers. Some of them use GUI applications to generate html, and the better ones use text editors. But most of them don't want to deal with anything more complicated than simple templating in html, which is what they know.

One thing you have to understand about the evolution of the web is that http and html were not initially designed to do the kinds of things they are being used for today. And on top of that the major browsers have very often been really broken. And on top of that you have no control over what facilities the user has available to them- they could be using any browser, with or without Java, Javascript, Flash, etc, and with any of a number of permutations of bugsets, depending on the browser. So it's only in the last several years that things have stabilized enough for people to be a little less conservative about the facilities they use on the client side. It's still a good idea to make sure that pages degrade gracefully, when possible. A general purpose web page should be usable in a text browser, though of course many types of web applications can't be made to work in that limited an environment.

HTML 5 is going to shake a lot of this up. But it is going to be a long time before it is safe to assume that everyone is using a browser capable of doing anything from HTML 5, and longer before it is safe to assume that everyone is using a browser that implements all of HTML 5. Anyway, I'd suggest you look around at some of the less visible projects, like Seaside for instance. There is a lot of experimentation going on. But the web has always been a tough environment for this kind of thing.

Answer 3

All the innovation would concern database issues, business logic, reusability of code, etc - but not actually anything new regarding innovative visual controls or graphics for a web-site

Correct. Good assessment. Is that a problem?

html might be analagous to assembler, so a conventional application developer from years past might know and occasionally even use a little assembler, but generally would hardly ever use it

False. Indeed, not even close. All browsers uses HTML. That cannot be changed easily.

chess.com... Is it most likely Java applets they're using for a site like that?

Use view source in your browser to answer this question for yourself. In general, you should do this for every web site you visit. You'll learn a great deal about the web and web development.

Would Django be used in conjunction with something like Flash or even Java applets

Yes. We use FLEX and Django.

Also when a site like chess.com is ported to a mobile device, what is used to write it - the same development tools as for the desktop or something completely different (Yes, I have a lot of catching up to do.)

Yes.

Are there in fact complete websites written solely in Java, perhaps using very high level Java API's?

Yes.

Why would someone say (as I read somewhere) that they despised Java so that is why they had gotten into Ruby on Rails and Django.

Some people like to despise Java. There's little technical merit to their argument.

After you've used Java and Python, you'll find that Python's less wordy. You get more done with less typing.

what is the justification for their [Python PHP] existence?

They're better than the alternatives. For specific things people need to do, Python (or PHP) are better than the alternatives. For "everything" or even a broad class of things, it may not be perfectly clear.

We do a lot of ad-hoc data crunching. Python's flexibility is absolutely superior to the alternatives.

First of all, Python is much, much slower than say C++, being interpreted.

That's hardly relevant, it turns out. Web sites are not governed by raw speed of one element of the architecture.

Why are websites written in Python or PHP

It's easier than the alternatives.

is platform independence the sole issue here.

No.

I am incredulous that application development is much faster in Python than C++

Have you done much with Python? You should give it a try for a year or so. It makes C++ quite tedious and error-prone by comparison.

is that what the primary reason for Python is - garbage collection.

No.

Answer 4

And I would have thought previously that html might be analagous to assembler, so a conventional application developer from years past might know and occasionally even use a little assembler, but generally would hardly ever use it, whereas from what I can see, html (and also CSS and javascript) still have to be mastered and written continually by every web developer, whether they're using Django or anything else. Is that a true statement?

Yup — if you want a website, someone’s going to have to write some HTML.

HTML is unlike assembler in that you can’t write parsers for new languages in HTML. HTML is just a declarative language for adding meaning to text. As such, the main thing is that everyone in the world agrees how how to render it, and what the tags mean. Something new might replace it eventually, but HTML has proven pretty serviceable and resilient so far. It’s also pretty easy to learn, and free.

Is it most likely Java applets they're using for a site like that? How relevant would Django be for a such a site. Would Django be used in conjunction with something like Flash or even Java applets?

Django really just concerns itself with the server side of websites. It leaves the client side of things (i.e. whatever runs in the browser) up to you. (Aside from the built-in admin site.)

Are there in fact complete websites written solely in Java, perhaps using very high level Java API's?

I don’t think it’s common. Java applets are hardly used any more, and some people (cough Steve Jobs cough sorry about the cough there, I said “Steve Jobs”) think Flash will go the same way.

Answer 5

In addition to everything everyone else have said so far, just making a very basic web site with HTML, some CSS and maybe even some JavaScript will probably give you a decent understanding of how those three work. A place like HTMLDog is a good place to start.

In addition, read up a little on the HTTP-protocol as this is still what normal web pages use and therefore defines the basics of how servers and clients communicate on the web.

HTTP, HTML, CSS and JavaScript is (and will probably be for quite a while) the same wheather you use Java, Django, ASP.NET or PHP for your application logic. If you are getting more into web development, these are relevant no matter what server-side technology you would ever choose. Also, a general understanding of browsers is nice to have. Both how they handle the visual rendering of HTML and CSS, but also how sessions, cookies and requests are handled.

Answer 6

The javascript stuff might look daunting, but nowadays there are pretty decent libraries that are Free/OSS, most notably jquery. I consider jquery to be partly a replacement for Flash.

You don't need that much extra javascript to use it productively. I'd definitely use jquery if I had to make an interface like chess.com uses.

see www.jquery.com and www.jqueryui.com

Answer 7

From what I read, ROR tries to do everything for you behind the scenes and so therefore is slow and unscalable (and overhyped and not ready for prime time).

Well, first of all, you shouldn't believe everything you read on the Internet:

1. I wouldn't say RoR isn't ready for primetime. RoR, like any tool, has its uses. If you're building a site like Twitter, maybe Rails isn't the best tool (as Twitter found out). While everyone thinks they're building a high-performance site, most developers aren't, and Rails will probably be suitable. Furthermore, Rails can scale, and a lot of work has been done to improve that situation even more.
2. The reason Rails sometimes has performance issues is not because it tries to do everything for you -- it's more because of the nature of the Ruby interpreter itself, which (until Ruby 1.9) was rather slow, and still isn't even as fast as other interpreted languages.

And the first thing that surprises me about Django, is that there does not seem to be any innovation to speak of regarding actual presentation. All the innovation would concern database issues, business logic, reusability of code, etc - but not actually anything new regarding innovative visual controls or graphics for a web-site.

Yes, but that is the innovation. Prior to frameworks like Django and Rails, a lot of that backend work was done by hand. Django frees up a developer's time to work on more application-level features.

And I would have thought previously that html might be analagous to assembler, so a conventional application developer from years past might know and occasionally even use a little assembler, but generally would hardly ever use it, whereas from what I can see, html (and also CSS and javascript) still have to be mastered and written continually by every web developer, whether they're using Django or anything else. Is that a true statement?

Yeah, pretty much. Nothing better than HTML, CSS, and JS has presented itself. While I agree that in some ways, HTML seems "low-level" in the same sense that assembly language is low-level, I think most would agree that, relative to problem domain, HTML is much nicer to work with.

Would Django be used in conjunction with something like Flash or even Java applets?

You can. Flash and Java are just embedded in HTML pages, and Django spits out HTML, so that's certainly possible.

Are there in fact complete websites written solely in Java, perhaps using very high level Java API's? Why would someone say (as I read somewhere) that they despised Java so that is why they had gotten into Ruby on Rails and Django.

In addition to applets, you can write a backend in Java (Java Server Pages, for example). I think most web developers who have worked with both would agree than Ruby and Python are much nicer to use than Java. Java web frameworks are kind of a pain, Java lacks a REPL, Java has a separate compilation step... Java is also statically typed; you can argue all day about the merits of dynamic typing vs. static typing, but Rails and Django both take advantage of Ruby's and Python's typing and introspection capabilities to make a lot of code less verbose than that of Java. (Whether that makes Ruby and Python better than Java is subjective.)

And regarding Python (and also PHP) what is the justification for their existence? First of all, Python is much, much slower than say C++, being interpreted. Why are websites written in Python or PHP - is platform independence the sole issue here. I am incredulous that application development is much faster in Python than C++ (aside from the garbage collection issue - is that what the primary reason for Python is - garbage collection.)

Performance isn't everything. Almost everyone wants to think their code is performance critical, but that's often not the case. As noted in a few other answers, most web apps are I/O bound anyway -- they're either waiting for database access, or waiting on the network, and both of those types of operations are orders of magnitude slower than CPU-intensive tasks, even with slow(er) interpreted languages. Furthermore, a lot of processing in web apps takes place on strings, and string processing is much nicer in Python or Ruby than it is in, e.g., C or C++. Python and Ruby are also more concise languages, and both offer a REPL which can shorten development time. Plus it's easy to write C extension modules in both Python and Ruby, so if you really, really find a code path that calls out for optimization, you can always drop down into C if you want.

Garbage collection is a plus, though.

Answer 8

And I would have thought previously that html might be analagous to assembler, so a conventional application developer from years past might know and occasionally even use a little assembler, but generally would hardly ever use it, whereas from what I can see, html (and also CSS and javascript) still have to be mastered and written continually by every web developer, whether they're using Django or anything else. Is that a true statement?

That's why I use Seaside. You still have to understand html and css and javascript, but at least there is a programming language abstraction over html. I'll never go back to a template based system.