Is it bad practice to "go deep" with your application of callbacks?

Question

Weird question, but I'm not sure if it's anti-pattern or not.

Say I have a web app that will be rendering 1000 records to an html table.

The typical approach I've seen is to send a query down to the database, translate the records in some way to some abstract state (be it an array, or a object, etc) and place the translated records into a collection that is then iterated over in the view.

As the number of records grows, this approach uses up more and more memory.

Why not send along with the query a callback that performs an operation on each of the translated rows as they are read from the database? This would mean that you don't need to collect the data for further iteration in the view so the memory footprint shrinks, and you're not iterating over the data twice.

There must be something implicitly wrong with this approach, because I rarely see it used anywhere. What's wrong with this approach?

Thanks.

Answer 1

Actually, this is exactly how a well-developed application should behave.

There is nothing wrong with this approach, except that not all database interfaces allow you to do this easily.

If we talk about tabularizing 10 records for a yet another social network, there is no need to mess with callbacks if you can get an array of hashes or whatever with a single call that is already implemented for you.

Answer 2

This is roughly what the iterator pattern allows you to do. In many cases this breaks down on the interface between your application and the database. Technologies like LINQ even have solutions that can send back code to the database.

Answer 3

There must be something implicitly wrong with this approach, because I rarely see it used anywhere.

I use it. Frequently. Even when i wouldn't use too much memory repeatedly copying the data, using a callback just seems cleaner. In languages with closures, it also lets you keep relevant code together while factoring out the messy DB stuff.

Answer 4

This is a "limited by your tools" class of problem: Most programming languages don't allow to say "Do something around this code". This was solved in recent years with the advent of closures. Think of a closure as a way to pass code into another method which is then executed in a context. For example, in GSQL, you can write:

def l = []
sql.execute ("select id from table where time > ?", time) { row ->
    l << row[0]
}

This will open a connection to the database, create a statement and a result set and then run the l << it[0] for each row the DB returns. Note that the code runs inside of sql.execute() but it can access local variables (l) and variables defined in sql.execute() (row).

With this kind of code, you can even generate the result of a HTTP request on the fly without keeping much of the page in RAM at any time. In my case, I'd stream a 2MB document to the browser using only a few KB of RAM and the browser would then chew 83s to parse this.

Answer 5

I've found it easier to use an interface resolver than deep callback where its hooked up through several classes. MS has a much fancier version than mine called Unity. This provides a much cleaner way of accessing classes that should not be tightly coupled

http://www.codeplex.com/unity