"Proxying" HTTP requests

Question

Hi all,

I have some software which runs as a black box, I have no access to it. This software makes HTTP requests. What I want to do is intercept these requests, forward them on, catch the response, do something with it, before passing the response back to the software.

Can this be done? What's the best method?

Thanks

Edit: Requests are to the public internet from a local intranet via a gateway/router. I have root access to my machine. Another machine could be used as intermediate gateway.

Edit 2: Requests are not encrypted. What I am actually trying to do is save down any images that are requested.

Answer 1

Try yellosoft-alchemy.

Answer 2

If the communication isn't encrypted, use Ethereal (or any other similar program) to sniff the communication on the wire.

edit: since the communication isn't encrypted, you can do that easily with Ethereal. You can save each TCP stream independently from there.

Edit2: Ok, you want to do this automatically. In this case, I would suggest you look at two tools available on Linux called tcpflow and tcpreen.

tcpreen creates a proxy similar to what you want between a local port and a remote one. It's a TCP proxy, not an HTTP proxy so this means you'll have to write some parsing tool to isolate the HTTP streams that contain the images you want (probably based on the MIME type of the response). it's not too complex a task, though, if you understand how HTTP works.

tcpflow is similar to tcpreen except that it's a sniffer instead of a proxy. Use whatever tool you think its more adapted to your environment.