Web scripting language with parallel non blocking database access?

Question

My webapp will need to use multiple database shards, and occasionally need to query these shards in parallel. Are there any web scripting languages that have mature, stable support for parallel non blocking database access? If so, can you point me in the right direction? Free open source is preferred, but I mostly want something that will work.

Threads are fine with me, but I don't require real multi-threading support. All I want is for five 10-second database queries to five different database servers to take 10 seconds and not 50. It doesn't matter to me how many CPUs it actually used.

Answer 1

Actually, it does not appear that you need parallel non-blocking I/O--you need parallel non-blocking database access, which is typically handled through a different API.

For example, in Java, general I/O is handled through components in java.io and java.net, but database access is handled through java.sql and vendor-specific drivers. Similar characteristics are typical of other platforms/languages.

It would seem that your main issue would then be to implement some kind of multiprocessing/multithreading from which you could make the separate, parallel database access requests. I suggest that is the main area that you should research.

There is a lot of material available on that topic, but you will need to focus on your particular platform/language for much of the detail. Please note that this is traditionally a VERY difficult area of software development.

Best wishes to you...

Answer 2

A possible work-around is to divide the web page so that each part only hits one shard. The browser should launch a separate AJAX request for each part and construct the page client-side.

This solution wouldn't depend on the chosen language/framework, but requires enabling JavaScript on the client side.

The performance might vary depending on each visitor's OS, browser and settings (more info). However, the page would render incrementally as soon as data from each part of the page comes available.

As drawbacks compared to the server-side solution, this method moves complexity from the server to client-side JavaScript and increases the load on the web server.