I am on Mac OSX. I wrote a program to parse a few websites. The websites are being accessed using curl. How can I view which url my computer is currently hitting?
+1
A:
If it's being called from your program, I'd add something to that program to print each website before running curl. You can tell curl to print the URL it's fetched with something like this:
curl -w "%{url_effective}\n" -o "$outputfile" "$url"
... but that won't print the URL until after it finishes downloading it. Also, note that this prints the effective URL to stdout; if the content of the web page is also going to stdout, they'll just get mixed together; you should only use this with -o or -O to redirect the web content.
Gordon Davisson
2010-10-06 20:24:14
Thank you for this but I need something I can view as the page hits are happening. I'm surprised there isn't a simple way to view outgoing network traffic from my computer.
iljkj
2010-10-07 00:08:53
You can view the network connections with `netstat -a | grep ESTABLISHED`, but it can't really tell you what website is being fetched, just the server's IP address. For example, if I hit www.apple.com, I get directed to the server at 92.123.93.15; `netstat` can tell me that IP (with the `-n` flag), or try to reverse-translate that to a name (without `-n`) but that comes back to a92-123-93-15.deploy.akamaitechnologies.com. The hostname and page curl are fetching are part of its private data; you'd have to use `tcpdump` or Wireshark to spy on that.
Gordon Davisson
2010-10-07 05:14:29