How limiting are web frameworks

Question

Hi,

This is a general question about how limiting are web development frameworks such as Django and ruby-on-rails.

I am planning on building a RESTful web service which will have a purely JSON/XML interface, no GUI. The service will rely on a database however for a few of the more important operations there is no clear way of persisting a "model" object directly into a database table. In addition I require full control over when and how the data is being written to the database. I will need to maintain multiple database connections in order to use some connections only for reads and others only for writes.

I've looked at the "full" MVC frameworks such as Django and more basic ones such web.py and pylons. The impression I currently have is that if I go with the full framework initially things will go faster but eventually I will get stuck because I will be limited by the framework in what I can do. If I go with a more basic framework it will take much longer to get everything running but I will be free to do what I need.

This is what it seems like but I suspect that it might be an incorrect impression given how many sites are written in Django and Rails. Could you please provide your opinion. Am I totally wrong and there is a way to easily do anything with a framework like Django or Rails or given my requirements I should go with something like web.py?

Thank you!

Answer 1

You'll be much more limited by the abilities of yourself versus a diverse community of developers working on a large project to share all those common parts.

Answer 2

Rails is as helpful or not as you need it to be, overall. If you need to load a collection with straight SQL, it's straightforward. If in the same line you want to use all the built-in ActiveRecord Fu, you can. RESTful routing is extremely simple, but again if the particular Rails flavor of REST doesn't meet your needs, the routing is completely configurable. In a Rails app you can use as much or as little of the defaults as you need to, and reconfiguration is available at all levels.

Answer 3

You can still use the full potential of the language in question, even if you also use a framework. A framework isn't a limiting factor, it's basically a tool to ease development of certain parts of your application. Django and rails, for instance, abstract away some database functionality, so you'll only have to worry about your model objects. That doesn't mean you can't do stuff on your own as well...

Answer 4

If you're not using the presentation layer of Rails, you're missing out on a huge part of it. The functionality required to dump objects to json/xml is so small the only real remaining advantages you could gain from it would be ActiveRecord and routing, and if you can't imagine your data cleanly fitting a model, then that doesn't leave much.

I think you really just need a minimalist framework to take care of some of the basics. Something that provides you with some niceties on request/response handling and routing and gets the hell out of the way. The Python equivalent of something like Sinatra might be up your alley. I use a similar framework in Scala called Step for xml/json based webservices where I care about performance (and there's no presentation going on).

I'm glancing over web.py, it seems to cover a similar level of functionality to Sinatra/Step. I feel that's a more appropriate direction than some of the more full features frameworks. I didn't regret my choice of Step, the codebase is so comically small it's impossible not to understand it, and that makes it easy to extend slightly should you have the need.

Answer 5

Web frameworks tend to optimize around building websites, making most normal use cases simpler to accomplish. Once you start to do more "out of the box" stuff with a framework, you might find that you spend more time working around it then you save using it in the first place.

It's hard to generalize here (especially since I've really only worked in-depth with Django), so I'll offer some advice based on my own experiences developing a JSON API using Django:

Simple put, I don't recommend using Django to write a REST API. In my own experience, I really didn't find anything worth writing home about. I didn't need Django's templating system, so all that I really made use of was the URL dispatching and ORM. Even then, I had to do some hacks to get the URL dispatcher to do what I wanted -- had I not used other features it would've been faster to use a different URL system, in fact. In your case, Django's ORM wouldn't even be suitable since it doesn't support multiple databases (unless you're using 1.2 alphas...). Compound that with Django's lack of a good startup signal, and Django starts to look pretty bad for the job.

If I were in your shoes, I'd dig around for specific libraries that did what I needed (ORM, WSGI, etc) and just use them, rather than trying to bend and hammer Django into something that suits my needs.

On a totally different note, you might want to take a look at Tornado as a possible HTTP frontend. It's both simple and fast.

Answer 6

I have used Ruby/Rails for years now, and unlike just about every other language/framework I have used (across nearly 15 years of Java, PHP, ColdFusion, ASP, etc etc) it gets out of the way when you need it to.

It sounds like you might benefit from a "lighter-weight" framework like Sinatra, but with the upcoming Rails 3 release the benefits are becoming less pronounced. Rails 3 makes everything configurable ... in fact, Rails will now just be a particular set of plugins and extensions sitting onto of an infinitely flexible core.

I am interested in this statement:

"The service will rely on a database however for a few of the more important operations there is no clear way of persisting a "model" object directly into a database table."

Not sure what you mean by this statement ... at some point you have something going into the database, right?

In most non-trivial applications you rarely have a single model tied to the end of a request ... you might actually have a quite complex network of models that are returned or updated.

If you are working with JSON, I would definitely suggest looking at a database like MongoDB. MongoDB is based entirely on storing JSON data, and may therefore fit really neatly with your application.

Answer 7

Most web sites will be well suited to rich frameworks like Rails or Django -- but you're building a web service , and that has very different tradeoffs.

Personally, I prefer to use very light frameworks for web services: in Python, that means relying primarily on WSGI (which is a standard, not a framework), with a minimum of highly modular components to provide some little extras that I may need depending on the services' details (WSGI's very good at allowing modular composition of "middleware" bits and pieces).

My personal favorite collection of modular WSGI components is Werkzeug, with WebOb for request and response objects; if I need templates, these days, I tend to go with Django Templates, and if I need a relational DB I prefer to write SQL directly (though SQLAlchemy has its strong points!-).

But the cool thing about using modular components rather than integrated frameworks is that you can change each and every one of these choices (and mix-and-match at will depending on your exact needs, preferences, and tastes!-).

Answer 8

You have written down no requirements, you have written down technology decisions. That's something totally different. What do you want to achieve? Then we might be able to help you with how to achieve them.

Answer 9

Give a try to Spring 3.0 : See this post

Answer 10

In average, the more complete and helpful the web framework is, the more limiting it is when you try to do things another way than the way the web framework thinks is The Right Way. Some web frameworks try to be very helpful and still not restrictive, and some do that better than others.

And the general recommendation there is: Don't fight the framework. You will lose. So it's important to chose a framework that helps you with the things you want to do, but doesn't enforce anything else. For your webservice case, this should not be a problem. There are tons of minimalistic web frameworks out there, at least in the Python world (which is all I care about). Bobo, BFG, Pylons, Werkzeug, etc, etc. None of these will get in your way one bit.

Also don't forget that you often can use several frameworks together by having them run side by side. Especially using techniques such as Dexterity/XDV. Plone.org for example is mostly Plon (duh) an excellent content management system, but extremely restrictive if you want to do something else. So part of the site is Trac, the excellent Python bug tracker. It's all integrated to look the same.

Answer 11

If you know you're not going to use an ORM, or create a user interface, then you've just eliminated about 90% of what you'd use a web application framework for in the first place. If you look at the feature set of Django, for instance: what parts of that would you use to implement a web service that you couldn't get from using something much simpler, like Werkzeug or CherryPy?

The principal differences between building a web service and building any old black box that takes input and produces output are the various technical limitations imposed by the API being HTTP-based, the problem of statelessness, and the problem of idempotence. A web application framework is going to give you a little help with those issues, but not much.