Accept-Encoding headers being sent by browser but not received by server

Check your antivirus software. It's probably intercepting your outbound traffic and modifying headers on the fly in order to get uncompressed content. Lazy programmers don't like to include decompression methods themselves, or deal with chunked encoding.

Norton Internet Security will overwrite accept encoding with this line:

---------------: ----- -------

McAfee overwrites with this:

X-McProxyFilter: *****

something I haven't identified yet overwrites with this:

Accept-Xncoding: gzip, deflate

You are probably in the same boat. I read Zone Alarm wipes out the encoding header entirely (which means recalculating the size of the packet, but why should they care how much load they introduce on your system?). If you're running Zone Alarm, turn off the 'internet privacy option' or whatever it is, and try again.

Everytime I've seen this problem, it has been the result of shitty antivirus. Completely disabling someone's ability to receive compressed content without letting them know is dirty.

Ok, I changed the logs on my Apache server to log Accept-Encoding headers. And what shows up in the logs is "-" for that field. So somewhere between my router and the server (any server, not just mine) the Accept-Encoding header is being stripped. Now I'm really confused!

Daniel Jacobs 2010-04-15 15:19:04

If your Apache server is responsible for SSL and has a certificate, if you try this with SSL, you will be able to see whether or not there is a "Man in the Middle" changing the request on the fly. With SSL, your browser can detect and warn you about any "Man in the Middle" fiddling with the request.

Justice 2010-04-15 15:28:09

So if it's being generated by my browser, it's not being changed on my network, it's not being changed mid-stream, and it's not being seen on any remote server, where could the Accept-Encoding header be going?

Daniel Jacobs 2010-04-15 15:55:36

Odd.... What about e.g. using Firefox with Firebug (Net panel) and browsing to any other site over SSL, such as `https://twitter.com/`? What does the response's `Encoding:` header say?

Justice 2010-04-15 17:28:23

Accept-Encoding gzip,deflate was indeed in Firebug's results for https://twitter.com. But Firebug is just showing the headers I'm sending, not necessarily what the remote site receives, right? Because if I go to whatismyip.org/http_compression, for example, it tells me I am not requesting compression. Somewhere the Accept-Encoding header is disappearing still.

Daniel Jacobs 2010-04-16 00:49:39

Firebug will show both the headers the browser sends to the server as well as the headers received by the browser from the server. If the server compresses the response body, then it must also add a header `Encoding: gzip`.

Justice 2010-04-16 02:45:40

Right. And the response headers do not have Encoding: gzip. That's what I'm saying :-) I'm sending the Accept-Encoding header from each of the browsers on my network. None of the servers are receiving that header if I use http, but they are if i use https.

Daniel Jacobs 2010-04-16 13:00:39

Nice. The implication is that you have a Man in the Middle modifying your HTTP requests in-transit and stripping out the `Accept-Encoding` header. The question now is to narrow down who that Man in the Middle is. I doubt it's the router doing that, especially if you checked with two different router models. Perhaps you can double-check with the ISP, or the ISP's upstream?

Justice 2010-04-16 13:42:41

I'll need to try escalating it to get anywhere with the ISP - the first response I got was less than helpful :-) This is really strange. I appreciate you trying to help.

Daniel Jacobs 2010-04-16 14:14:30

ansaurus

tags:

views:

answers:

Accept-Encoding headers being sent by browser but not received by server

related questions