C++ 'GET' request or how do you download files to work with in C++?

Question

Alright I've spent a good three days trying this, here's the scenario:

I want to download a '.csv' file from Google and then do stuff with the data from the file. It's for a Win32 Console Application. I have the latter down, I just cannot for the life of me figure out how to download the file. I've heard of libcurl, curlpp, ptypes, rolling my own, just using the .NET api, and been told a bunch of times:

...it's just a GET request

Well that's all well and good, but I must be missing something because it seems like everyone was just born knowing how to do this. I have been combing through books looking to figure this out and even had a huge problem with LNKerrors after traveling down the road with "The Art of C++" for a while.

All that being said, I have learned a LOT from this, but at this point I just want to know how to do it. The API for C++ is seriously lacking, no example code to be found. Tutorials online are almost non-existent. And no book out there seems to think this is important.

Can someone please throw me a life raft? I'm a man on the edge here.

edit

By "from Google" I mean that I want to download a .csv file that they host. An example can be found here.

Answer 1

C++ doesn't really have an API per se. To do a GET request, your program needs to open a TCP socket connection to the site in question, and send the information down the socket to conform with the HTTP specification, see here. There are likely a number of libraries you could find to help with this, for example, you could look at what the open source program wget uses. If you're coding it yourself, it'd be helpful to use a packet sniffing tool. You could see exactly what your browser sends to the web server to get the file back, and just send exactly that message. It's not too hard, but it is pretty hard. TCP is a pretty big learning curve initially.

By the way, what do you mean by "from google?"

Answer 2

You should be able to bend this to your will.

Now that I have kinda answered your question. Why C++? Nothing against the language, but pick the best language for the job. Perl, PHP, and Python(and I am sure more) all have great documentation and support on this kind of operation.

In perl(the one I am familiar with) it's just about 3-5 lines of code.

Answer 3

Since you're on the Win32 platform, there is one built in libary you can use to implement a GET request in a relatively straightforward way: WinInet, which is part of the Win32 SDK. The basic reference for WinInet can be found on MSDN.

Be warned there will be some rough sledding ahead if you're not familiar with the Win32 API. There is a fairly useful block of example code here

You will get linker errors if you fail to add the appropriate library reference to your project. It sounds like you've learned some lessons there already so I'll keep it breif, but be assured that you will find references to both the libraries and header file references you will need in the Win32 documentation (you just have to learn where on the page to look for it).

Answer 4

Why not just using what's already there?

UrlDownloadToFile()

Answer 5

You can't do that with the standard library. Since it's win32, you can follow Paul Keister's advice.

But have a look at libcurl, and since you are in C++, I'd recommend POCO it's very usefull too. They are very handy tools!

Answer 6

Another option is WinHTTP, which also comes with Windows.

Answer 7

> no example code to be found

???

Dozens of samples have been posted for 15 years.. See on api ng news://comp.os.ms-windows.programmer.win32 (samples in C)

(COM, INET, etc)

Answer 8

I too have been looking for a good http lib for C++ but haven't found the 'perfect' lib. Boost.Asio is fantastic but quite low level for a simple http library (though the http client examples are a good start). The C++ Networking Library* (cpp-netlib) looks like it's on track, building a higher-level api on top of asio, but is not very mature. Check out the http_client example.

I'm either going to write my own wrappers around asio or extend (and contribute) to cpp-netlib unless I can find a better option.

(BTW libcurl is not a better option for me. It's a much larger api than I want and very unwieldy for something like an http client, especially in C++. Don't get me wrong, it's a great library but not appropriate for a simple http client.)