Calculate total batch upload transfer percent with limited information

Question

Hi there,

I have a system which uploads to a server file by file and displays a progress bar on file upload progress, then underneath a second progress bar which I want to indicate percentage of batch complete across all files queued to upload.

Information and algorithms I can work out are:

Bytes Sent / Total Bytes To Send = First progress bar (eg. 512KB of 1024KB (50%))

That works fine. However supposing I have two other files left to upload, but both file sizes are unknown (as this is only known once the file is about to commence upload, at which point it is compressed and file size is determined) how would I go about making my third progress bar?

I didn't think this would be possible as I would need "Total Bytes Sent" / "Total Bytes To Send", to replicate the logic of my first progress bar on a larger scale, however I did get a version working: "Current file number we are on" / "total number of files to send" returning the percentage through the batch, however obviously will not incrementally update and it's pretty crude.

So on further thinking I thought if I could incorporate the current file % with this algorithm I could perhaps get the correct progress percentage of my batch's current point.

I tried this algorithm, but alas to no such avail (sorry to any math heads, it's probably quite apparent why it won't work)

("Current file number we are on" / "total number of files to send") * ("Bytes Sent" / "Total Bytes To Send")

For example I thought I was on the right track when testing with this example: 2/3 (2nd of 3rd file) = 66% (this is right so far) but then when I added * 0.20 (for indicating only 20% of 2nd file has uploaded) we went back to 13%. What I need is only a little over 33%! I did try the inverse at 0.80 and a (2/3 * (2/3 * 0.2))

Can this be done without knowing entire bytes in batch to upload?

Please help! Thank you!

Answer 1

If you don't know how big the other queued files are, then there's no way to accurately display a percentage value that's relevant and proportional to the required time.

A number of workarounds come to mind:

• One approach that looks like you know but are in fact deceiving the user would be to assume all the queued files are the same size as the file in progress, or the average of files processed so far. Based on this, the progress bar would be showing "the truth" if all files are indeed the same size, and would be very off if sizes differ significantly.

• Another approach would be for your second progress bar to show not the percentage of bytes transferred but the percentage of files. Thus, if you have 4 files, that bar would hop from 0 to 25% to 50% to 75% to 100%. It wouldn't accurately reflect time taken but at least you wouldn't be lying.

• You can do even worse with an approach like Microsoft's: Make your progress bar's growth asymptotically slower as it approaches 100%, so that it never actually reaches the end. All the user sees is constantly closer values of "almost finished." Such a display looks cool but is in effect giving the user the least possible information.

Answer 2

As @Carl has observed, without knowing how much there is still to send you can't produce a true estimate of the proportion already sent. Putting aside scientific accuracy and integrity for a moment, your calculation:

("Current file number we are on" / "total number of files to send") * 
("Bytes Sent" / "Total Bytes To Send")

should be extended to incorporate the idea of fractions of files. If, say, you've sent 6 files out of 11 and 30% of the 7th file, you'd want to calculate

(6.3 / 11) * ("Bytes Sent" / "Total Bytes To Send")

Somewhere along the line you multiplied the number of files sent by the proportion of the current file already sent, ie you calculated 0.66*0.2=0.13, rather than adding the proportion of then current file.

Picking up scientific accuracy and integrity again, why not use the rate of transmission on the second progress bar ? If you integrate that over a sensible period it should give the watcher the satisfying sense that something is happening and progress is being made.

Answer 3

Oh my gosh, I didn't consider the simplest approach:

(double)((ProcessedFileCount + DecimalBytesTransferred) / TotalFileCount)

(where processedFileCount always indicates fully transferred and complete files)

and proof test case on 3 file batch:

Eg. ((2 + 1.0) / 3) (File 1+2+3: 100%) == batch 100% complete.
Eg. ((2 + 0.9) / 3) (File 1+2: 100%, File 3: 90%) == batch 96% complete.
Eg. ((1 + 0.9) / 3) (File 1: 100%, File 2: 90%) == only 63% complete.

Works a treat! Thanks for your input though guys I will consider all external advice.