PCAP Web Service Usage Logging for Dummies

Question

I've been assigned the task (for work) of working with PCAP for the first time in my life. I've read through the tutorials and have hacked together a real simple capture program which, it turns out, isn't that hard. However, making use of the data is more difficult. My goal is to log incomming and outgoing web service requests. Are there libraries (C or C++) that stitch together the packets from PCAP that would make reporting on this simple (that is Layer 7 reporting)? Baring that is there something short of reading all of the RFC's from soup to nuts that will allow me to have an "ah-ha!" moment (all of the tutorials seem to stop at the raw packet level which isn't useful for me)?

It looks like PERL has a library that may do this and I may eventually attempt a reverse engineer from PERL.

NOTE BENE: Web Server logs aren't acceptable here as I will be intercepting on a routing device. If I had access to those I'd be done and happy...I don't.

NOTE BENE: I do not have access to anything but what my application will install on this black box so script languages are out

Answer 1

You should look at Tshark - the commandline version of Wireshark. It has access to all of the protocol dissectors that Wireshark has, including of course HTTP.

Eg to capture everything involving TCP port 80 and decode it as HTTP:

tshark -d tcp.port==80,http tcp and port 80

You might be able to use the Wireshark dissector code in your app - or it might just end up being easier to call out to tshark to do the work for you.

Answer 2

As this point I'm looking into OpenDPI. I'm not sure if its a tight fit and will respond back here once I know but it does seem that it will cover my needs and won't require reverse engineering: http://code.google.com/p/opendpi/

EDIT: Yep. OpenDPI works for my needs.