A HTTP request requires a TCP/IP connection to be made (Think, 3-way handshaking handshaking) before it can handle the HTTP request it self
This involves at least a delay of sending the SYN message to the server and getting the SYN/ACK back (It then sends the ACK to OPEN the socket).
So, say the delay between the client and server is uniform both ways and 50ms, that results in a 100ms delay before it can send the HTTP request. It is then another 100ms before it starts getting the actual request back (Sends the request, then server replies).
Of course, you need to also take into consideration that a standard web browser limits the number of concurrent HTTP requests it is processing at the same time. If your requests have to wait, you don't get that handshake time for free (so to say), since you need to wait for another connection to finish as well. Servers play a role as well, depending on how they server the requests.