Monitoring outgoing internet traffic

Question

Is there a way to monitoring internet traffic programatically? I would like to log the pages users are visiting on the Internet. Can this be achieved with .NET code, is there a 3rd party .NET component that could be used to retrieved data.

Information about Internet traffic must be stored to a database so I cannot use a plugin or something for IE. We are also looking to include this code into our existing product so we cannot use a 3rd party product that cannot be redistributed.

It would be cool if this thing could monitor traffic for all browsers but monitoring IE traffic might also be sufficient.

Answer 1

Setting up a sniffer is doable via the WinPCap library, which has several projects wrapping it to .NET:

• http://geekswithblogs.net/dotnetnomad/archive/2008/01/31/119140.aspx
• http://www.codeproject.com/KB/IP/dotnetwinpcap.aspx
• http://coolthingoftheday.blogspot.com/2006/02/sharppcap-net-winpcap-wrapper.html

and probably some others as well, just a matter of Googling.

Answer 2

You would need to build some software that acts as a proxy. You should start by looking at programs like "Fiddler" to understand the concepts and what you need to implement.

If you want my profesional opionon you should go to Server Fault and ask for opionons for low cost internet proxy solution. Writing this thing yourself while challenging and fun will not make good business sense.

Answer 3

If you want to monitor all traffic from everyone on your network consider a product like Microsoft's Internet Security and Acceleration (ISA) Server

While this is probably overkill for what you want, the point is that you need a way to have all traffic go through a single point (a proxy server) where the traffic can be logged. Since all traffic goes through this one point, users can't avoid detection by using an alternate browser etc.

Answer 4

You can also try Pcap.Net to easily use WinPcap in .NET. It is a wrapper for WinPcap written in C++/CLI and C# and includes a packet interpretation and creation framework.

Answer 5

Like JD already mentioned you should take a look into Fiddler. With this proxy you are able to monitor all the web traffic (if your browser is configured to use fiddler as proxy).

If you don't want fiddler as a standalone application and need its functionality within your own app, you should take a look into FiddlerCore. It is a .Net assembly which encapsulates the core functionality of Fiddler.

If you need a more raw way to sniff into the data (maybe you don't want to [or can't] configure the proxy of the client) the answer from Vinko will help you further.