What is the length of time to send a list of 200,000 integers from a client's browser to an internet sever?

Question

Over the connections that most people in the USA have in their homes, what is the approximate length of time to send a list of 200,000 integers from a client's browser to an internet sever (say Google app engine)? Does it change much if the data is sent from an iPhone?

How does the length of time increase as the size of the integer list increases (say with a list of a million integers) ?

Context: I wasn't sure if I should write code to do some simple computations and sorting of such lists for the browser in javascript or for the server in python, so I wanted to explore this issue of how long it takes to send the output data from a browser to a server over the web in order to help me decide where (client's browser or app engine server) is the best place for such computations to be processed.

More Context:

Type of Integers: I am dealing with 2 lists of integers. One is a list of ids for the 200,000 objects whose integers look like {0,1,2,3,...,99,999}. The second list of 100,000 is just single digits {...,4,5,6,7,8,9,0,1,...} .

Type of Computations: From the browser a person will create her own custom index (or rankings) based changing the weights associated to about 10 variables referenced to the 100,000 objects. INDEX = w1*Var1 + w2*Var2 + ... wNVarN. So the computations refer to vector (array) multiplication to a scalar and addition of 2 vectors, as well as sorting the final INDEX variable vector of 100,000 values.

Answer 1

By default, typically integers are stored in a 32-bit value, or 4 bytes. 200,000 integers would then be 800,000 bytes, or 781.25 kilobytes. It would depend on the client's upload speed, but at 640Kbps upload, that's about 10 seconds.

Answer 2

well that is 800000 bytes or 781.3 kb, or you could say the size of a normal jpeg photo. for broadband, that would be within seconds, and you could always consider compression (there are libraries for this)

the time increases linearly for data.

Answer 3

Since you're sending the data from JavaScript to the server, you'll be using a text representation. The size will depend a lot on the number of digits in each integer. Are talking about 200,000 two to three digit integers or six to eight integers? It also depends on if HTTP compression is enabled and if Safari on the iPhone supports it (I'm not sure).

The amount of time will be linear depending on the size. Typical upload speeds on an iPhone will vary a lot depending on if the user is on a business wifi, public wifi, home wifi, 3G, or Edge network.

If you're so dependent on performance perhaps this is more appropriate for a native app than an HTML app. Even if you don't do the calculations on the client, you can send/receive binary data and compress it which will reduce time.

Answer 4

In a nutshell...

This is probably a bad idea,

in particular with/for mobile devices where, aside from the delay associated with transfer(s), limits and/or extra fees associated with monthly volumes exceeding various plans limits make this a lousy economical option...

A rough estimate (more info below) is that the one-way transmission takes between 0.7 and and 5 seconds.
There is a lot of variability in this estimate, due mainly to two factors

• Network technology and plan
• compression ratio which can be obtained for a 200k integers.

Since the network characteristics are more or less a given, the most significant improvement would come from the compression ratio. This in turn depends greatly on the statistic distribution of the 200,000 integers. For example, if most of them are smaller than say 65,000, it would be quite likely that the list would compress to about 25% of its original size (75% size reduction). The time estimates provided assumed only a 25 to 50% size reduction.

Another network consideration is the availability of binary mime extension (8 bits mime) which would avoid the 33% overhead of B64 for example.

Other considerations / idea:

• This type of network usage for iPhone / mobile devices plans will not fare very well!!!
  ATT will love you (maybe), your end-users will hate you at least the ones with plan limits, which many (most?) have.
• Rather than sending one big list, you could split the list over 3 or 4 chunks, allowing the server-side sorting to take place [mostly] in parallel to the data transfer.
• One gets better compression ratio for integers when they are [roughly] sorted, maybe you can have a first pass sorting of some kind client-side.

How do I figure? ...

1) Amount of data to transfer (one-way)
  200,000  integers 
    = 800,000 bytes  (assumes 4 bytes integers)
    = 400,000 to 600,000 bytes compressed  (you'll want to compress!)
    = 533,000 to 800,000 bytes in B64 format for MIME encoding

2) Time to upload   (varies greatly...)
    Low-end home setup (ADSL)  = 3 to 5 seconds
    broadband (eg DOCSIS)      = 0.7 to 1 second
    iPhone                     = 0.7 to 5 seconds possibly worse;
                                        possibly a bit better with high-end plan

3) Time to download (back from server, once list is sorted)
   Assume same or slightly less than upload time.
   With portable devices, the differential is more notable.
   The question is unclear of what would have to be done with the resulting 
   (sorted) array; so I didn't worry to much about the "return trip".
   ==> Multiply by 2 (or 1.8) for a safe estimate of a round trip, or inquire
       about specific network/technlogy.