Nodejs proxy script, not working with mod_deflate

Question

Hi Folks,

I created a little proxy node script, which looksup the request.url and either passes a request to my apache server or uses node to process/response to this request. I have been successful so far, everything works fine, but when I enable mod_deflate for the apache, "strange things will happen".

It looks like node just "cancels" or "stops" a response way to early. I'm listening on the "data" event from my request, and at some point node just decides that the response has ended (which is wrong) and fires the "end" event.

Code snippet:

var apache = http.createClient(82, 'localhost');

function pass_to_apache(req, res){
    var request = apache.request(req.method, req.url, req.headers);

    req.addListener('end', function() {
        request.end();
    });

    req.addListener('data', function(chunk) {
        request.write(chunk);
        sys.puts('writting chunk\n');
    });

    request.addListener('response', function(response) {
        res.writeHead(response.statusCode, response.headers);
        response.addListener('data', function(chunk) {
            sys.puts('writting data..\n');
            res.write(chunk);
        });
        response.addListener('end', function() {
            sys.puts('end of request');
            res.end();
        });
    });
}

var MainServer = http.createServer(function(request, response) {
    sys.puts('received '+request.method+' '+request.url + "\n"+JSON.stringify(request.headers));
    if(/^\/node/.test(request.url)) {
        response.writeHead(200, {'Content-Type': 'text/plain'});
        response.end("Hi, it's node =)\n");
    }
    else if(/^\/exit/.test(request.url)) {
        sys.puts('closing..\n');
        MainServer.close();
        throw new Error('forced');
    }
    else {
        pass_to_apache(request, response);
    }
});

MainServer.listen(80, 'typeofnan.com');

You can "see" this in action at www.typeofnan.com && www.typeofnan.com/node/anything
edit: disabled nodejs for now.

Remember, this works like a charm if no gzip/deflate is used by the apache. I tried to set the encoding to "binary" in my reponse, but no success either.

Am I missing something here ? Can someony confirm this behavior? I'm using the latest relase (0.2.0).

Is there maybe another (better) solution to use a proxyscript like this?

Answer 1

I'm intrigued. I fired up your code and pointed it at www.typeofnan.com. It worked fine, but I noticed that the server was not returning compressed responses. So then I set it to proxy apache.org and my browser also rendered it fine with gzipped content! For a GET on "/", I got following response headers:

{"date":"Mon, 13 Sep 2010 11:03:45 GMT","server":"Apache/2.3.8 (Unix) mod_ssl/2.3.8 OpenSSL/1.0.0a","last-modified":"Sat, 11 Sep 2010 19:38:09 GMT","etag":"\"c9489a-4ada-4900100c32a40-gzip\"","accept-ranges":"bytes","vary":"Accept-Encoding","content-encoding":"gzip","cache-control":"max-age=86400","expires":"Tue, 14 Sep 2010 11:03:45 GMT","content-length":"5359","keep-alive":"timeout=5, max=100","connection":"Keep-Alive","content-type":"text/html"}

Hmmm... did I just get lucky and not get a gzipped response that caused your issues? Do you have a page that reliably causes "strange things to happen" that I could test against? Actually, you might need to define "strane things will happen" :)

As a hack, you could get your proxy to change the accept-encoding header so that apache will never return a compressed response. Adding the following to your apache request will force apache to return uncompressed responses:

req.headers['accept-encoding'] = '*;q=1,gzip=0';