tags:

views:

1061

answers:

6

I know it's possible, but I can't seem to figure it out.

I have a mySQL query that has a couple hundred thousand results. I want to be able to send the results, but it seems the response requires content-length header to start downloading.

In phpMyAdmin, if you go to export a database, it starts the download right away, FF just says unknown file size, but it works. I looked at that code, but couldn't figure it out.

Help?

Thanks.

A: 

You don’t need to set a Content-Length header field. This header field is just to tell the client amount of data it has to expected. In fact, a “wrong” value can cause that the client discards some data.

Gumbo
+7  A: 

this document in section 14.14 states that In HTTP, it SHOULD be sent whenever the message's length can be determined prior to being transferred, unless this is prohibited by the rules in section 4.4. This means it DOESN'T have to be sent if you can't say the size of it.

Just don't send it.

If you want to send parts of data to the browser before all data is available, do you flush your output buffer? Maybe that's what is the problem, not lack of a header?

The way you use flush is like that:

  • generate some output, which should add it to the buffer
  • flush() it, which should send current buffer to the client
  • goto 1

So, if your query returns a lot of results, you could just generate the output for, lets say, 100 or 1000 of them, then flush, and so on.

Also, to tell the client browser to attempt to save a file instead of displaying it in window, you can try using the Content-disposition: attachment header as well. See the specification here, 19.5.1 section.

kender
I have the headers, then I have flush, then I have the sql query. It doesn't pop up any save dialog after the flush though. When a small file, it only pops up when the file is done downloading. The sql query I'm attempting takes 45seconds, and keeps timing out after the headers are sent.
phazei
specified a bit more in the anwser
kender
My headers:header('Content-Type: "'.$mime.'"');header('Content-Disposition: attachment; filename="'.$filename.'"');header("Content-Transfer-Encoding: binary");header('Expires: 0');header('Pragma: no-cache');
phazei
Unfortunately I'm not even getting to the buffering point, it's timing out on the query. Usually, I send a 'nbsp;' so the browser won't time out, but since it's a file, I can't have that written to it. The headers don't seem to be stopping it from timing out...
phazei
What's your PHP's timeout setting? http://www.php.net/set_time_limit
kender
It's at 1800. Its the browser that times out. If I echo a single period, it prevents the timeout. So i rearranged everything and have it flush the csv's header line first, and it starts the download right away. Then it says 'starting download' while it waits for the query to finish.
phazei
Got it all working, I flushed every %1000, and used 'pagination' with the sql calls to only pull 50,000 at a time so it didn't kill the memory with a couple 100K in a single call. I can see it pause every 10megs for 30sec or so while doing the next query.
phazei
A: 

If you use the LiveHTTPHeaders plugin in FireFox, you can see the difference between the headers being sent by phpMyAdmin and the headers being sent by your application. I don't know specifically what you're missing, but this should give you a hint. One header that I see running a quick test is "Transfer-Encoding: chunked", and I also see that they're not sending content-length. Here's a link to the FireFox plugin if you don't already have it: LiveHTTPHeaders

Rich
A: 

http content length is a SHOULD field, so you can drop it... but you have to set transfer encoding then have a look at http://stackoverflow.com/questions/328281/why-content-length-0-in-post-requests/328652#328652

Tobiask
+1  A: 

It’s possible that you are using gzip which waits for all content to be generated. Check your .htaccess for directives regarding this.

Ilya Birman
+2  A: 

You can use chunked transfer-encoding. Basically, you send a "Transfer-encoding: chunked" header, and then the data is sent in chunked mode, meaning that you send the length of a chunk followed by the chunk. You keep repeating this until the end of data, at which point you send a zero-length chunk.

Details are in RFC 2616.

Graeme Perrow