Parallelized resource loading vs DNS lookup speed

Question

A common technique for reducing page loading times is to parallelize multiple static resource downloads by retrieving them from different hostnames (even if they all resolve to the same server).
However, the browser needs to issue a DNS lookup request for each of these hostnames, which could take a significant time.
Can you propose a method using which the JavaScript code could determine on the fly the optimal number of hostnames to parallelize resource downloads over?

Answer 1

After the first time you resolve a given DNS name, the name should be cached by a resolver very close to the user. So if you want to get more parallel connections, just don't use totally-random, off-the-wall servers; use a consistent set of five or so different hosts. The DNS load will only happen the first time a user hits your site (or, not even then, if someone sharing the same caching resolver has already been there) and will happen in parallel with the download of your page's body.

JavaScript is ill-equipped to determine the "optimal" number, as it has no idea how long the DNS queries will take before issuing them.

Answer 2

There are a number of ways of reducing DNS load time with different host names. One trick is to have the other host names in the AR section of the DNS response; for example, if someone asks for a.example.com, we can helpfully tell them the DNS hostnames for b.example.com, c.example.com, d.example.com, and so on as additional records in the DNS reply, when giving them the IP for a.example.com in the answer.

This will pre-cache the answers for all of those other host names with some, but not all, recursive DNS servers (for security reasons, this trick doesn't work for either of the recursive DNS servers I have implemented, since, for security reasons, they do not blindly cache records in the AR section of a DNS reply).

Another trick to minimize DNS lookups is to have all of the names in the same domain (have all names end in, for example, example.com: a.example.com, b.example.com, etc.), make sure no glueless records need to be followed to resolve the names (make your NS records have names like ns1.example.com and ns2.example.com, and don't use any CNAME entries), and, if your servers do not change IPs very often, have a large TTL for the DNS entries for your servers.