I changed the name servers for one of my domains two hours ago. I kept checking in a browser and just now it stopped taking me to my old site. However, it just displays a "could not connect" error.
I don't know if the transition took place and there's a problem with the new server or it just didn't happen yet.
I know a DNS change takes effect after 24 hours.
So my question is - is there a period where neither the old location nor the new one are working? Or is the transition done seamlessly?
DNS lookups are cached on multiple levels - by nameservers, backup nameservers, providers' nameservers.... That means that it can take up to 24 hours until every computer in the world resolves your host name to the correct IP.
During that period, some requests will end up on server A, some on server B.
If both servers are set up to deal with your host name, a seamless transition is possible. However, some E-Mails may end up in the old mail box; also, if you have a dynamic web site with lots of user interaction, you'll have an awful mess because some interaction has taken place on the old, and some on the new server. In that case, it is indeed better to make a clean cut, change DNS entries and switch off the old server.
Networking is not my specialty area so there is a chance tis may be somewhat incorrect, but I believe it to be at least somewhat accurate.
Most nameservers have authority over a very small section of the internet (like a single domain). For areas where they are not the authority they ask other nameservers until they get an authoritative response. Some nameservers also cache non-authoritative responses so they aren't continually asking for the same info from other nameservers.
A record in DNS is updated when the cached response expires, however there is a chance that, for a time, a DNS somewhere in the chain could return a cached response that could be stale which would result in any client that used that response being routed to the wrong host which explains why your web site may appear down - the dns server your browser is using may not be aware of the change to your site's DNS records yet
Short answer: No, DNS servers have no direct effect on connecting, it's a misconfiguration on your new server (or a typo in IP).
Long answer: No, DNS servers will either return the old DNS record or the new DNS record (old server, new server respectively), not both or either, just the one. I would check new DNS record for typos and the server config (namely the VirtualHost section).
You can check which IP is the one cached by router:
nslookup should be querying your router. The non-authoritative answer is the one you will get until the cached entry expires.