Rebol Request-Download doesn't support Big File: how to correct ?

Question

download-dir: request-dir
Print ["downloading " "VStudio2008Express.iso" "..." ]
url: http://go.microsoft.com/fwlink/?LinkId=104679 
file-save: to-rebol-file rejoin [download-dir "VStudio2008Express.iso"]
request-download/to url file-save

In the end whereas the progress bar has shown the download has finished:

** Script Error: Not enough memory
** Where: append
** Near: insert tail series :value
>>

So how to correct request-download as it is a mezzanine function:

func [
    {Request a file download from the net. Show progress. Return none on error.} 
    url [url!] 
    /to "Specify local file target." local-file [file! none!] 
    /local prog lo stop data stat event-port event
][
    view/new center-face lo: layout [
        backeffect [gradient 1x1 water gray] 
        space 10x8 
        vh2 300 gold "Downloading File:" 
        vtext bold center 300 to-string url 
        prog: progress 300 
        across 
        btn 90 "Cancel" [stop: true] 
        stat: text 160x24 middle
    ] 
    stop: false 
    data: read-thru/to/progress/update url local-file func [total bytes] [
        prog/data: bytes / (max 1 total) 
        stat/text: reform [bytes "bytes"] 
        show [prog stat] 
        not stop
    ] 
    unview/only lo 
    if not stop [data]
]

Answer 1

Rebol's read functions read all the input data into memory at once and cannot be used on large datasets. You have to open a port and copy the data from it in chunks to handle large datasets.

I would think that the request-download function could be modified to use ports for both the input and output data. This thread from the Rebol Mailing List may help you:

http://www.rebol.org/ml-display-thread.r?m=rmlFQXC

You can find a fuller example on Carl's blog at http://www.rebol.com/cgi-bin/blog.r?view=0281#comments

Even using this technique there is a limit of approximately 2Gb to the size of files that can be handled in Rebol 2.

Answer 2

Try this

http://anton.wildit.net.au/rebol/util/batch-download.r