How (i.e. what tool to use) to monitior headers sent by Curl (Cookie problem)

Question

I am using Curl (libcurl) in a C++ aplication, and am unable to send cookies (I think).

I have Fiddler, TamperData and LiveHTTP Headers installed, but they are only useful for viewing browser traffic, and are (it would seem) unable of monitoring general network traffic on a machine, so when I run my machine, I cant see the header information being sent. However, when I view the page in a browser, when succesfully logged on, I can see that cookie information is being sent.

When running my app, I succesfully log onto the page, when I subsequently, try to fetch another page, the (page) data suggests that I am not logged on - i.e. "state" has somehow being lost.

My C++ code looks alright, so I dont know what is going wrong - this is why I need to:

1. First be able to view my machines network traffic (not just browser traffic) - which (free) tool?

2. Assuming I am using Curl incorrectly, whats wrong with my code? (the cookies are being retrieved and stored ok, it seems they are just not being sent with requests for some reason.

Here is the section of my class that deals with the cookie side of Http requests:

curl_easy_setopt(curl, CURLOPT_TIMEOUT, long(m_timeout));
curl_easy_setopt(curl, CURLOPT_USERAGENT,
    "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; WOW64; SV1; .NET CLR 2.0.50727)");
curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "cookies.txt");
curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "cookies.txt");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_AUTOREFERER, 1L);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, CurlCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, this);

Is there anything wrong with the above code?

Answer 1

You can use Wireshark (the former Ethereal) to view all the network traffic a machine is sending and receiving.

Answer 2

1. As Sean Carpenter said, Wireshark is the right tool to view network traffic. Start a capture and use http as a filter to see only HTTP traffic. If you just want to see HTTP requests/responses sent/received by Curl, set the CURL_VERBOSE option and look at stderr: curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L).
2. I believe you are using Curl correctly. Compile and run the following (complete) example; you will see that, the second time you run it (when cookies.txt exists) cookies are sent to the server.

Example code:

#include <stdio.h>
#include <curl/curl.h>

int main()
{
    CURL *curl;
    CURLcode success;
    char errbuf[CURL_ERROR_SIZE];
    int m_timeout = 15;

    if ((curl = curl_easy_init()) == NULL) {
        perror("curl_easy_init");
        return 1;
    }

    curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errbuf);
    curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);

    curl_easy_setopt(curl, CURLOPT_TIMEOUT, long(m_timeout));
    curl_easy_setopt(curl, CURLOPT_URL, "http://www.google.com/");
    curl_easy_setopt(curl, CURLOPT_USERAGENT,
        "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; WOW64; SV1; .NET CLR 2.0.50727)");
    curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "cookies.txt");
    curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "cookies.txt");
    curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
    curl_easy_setopt(curl, CURLOPT_AUTOREFERER, 1L);

    if ((success = curl_easy_perform(curl)) != 0) {
        fprintf(stderr, "%s: %s\n", "curl_easy_perform", errbuf);
        return 1;
    }

    curl_easy_cleanup(curl);
    return 0;
}