C++, Get text from a website

Question

I was told I have to use winsock, but I dont know where to start. For example, I am trying to access, lets say http://www.newegg.com/, I am trying to get the text title of just the three front page products. Any help is greatly appreciated. :D

Answer 1

you need to open a socket.

then you need to do an http get

somewhat like :- http://www.esqsoft.com/examples/troubleshooting-http-using-telnet.htm

Answer 2

You could use the QNetworkAccessmanager class from Qt framework.

Answer 3

W3.org themselves provide sample C / C++ librarys for Http requests.

Find them here

Specifically, look for HTTPReq.c

Answer 4

Use c++ if you must, but it might be a lot less painful to use python.

Look at the Python httplib module for how to set the host you want to pull from etc. Python's available for free for most platforms and is enough like C++ that you can probably learn python a heck of a lot faster than you can learn to write a program controlled browser in c++. Well, maybe that's not true for everyone on this site, but I'll bet it's true for "most" of us. I used to get stock quotes updated in near real time from CNN Money years ago and IIRC it was around 100 lines of python code.

Hotei

Answer 5

I'm assuming you need to use c++ for a reason, such as integration with existing software, otherwise, as per some of the other suggestions, choosing a language with a more convenient framework (eg: scripting language) would be better suited for the task.

If you would like to avoid getting your hands dirty with WINSOCK, or have the need to run on a platform other than windows, you could look at the using the boost asio library.

The following page contains links to simple sync and async http clients: http://www.boost.org/doc/libs/1_37_0/doc/html/boost_asio/examples.html

You can find documentation on the library itself at: http://www.boost.org/doc/libs/1_37_0/doc/html/boost_asio.html

Answer 6

I'd also recommend libcurl for this sort of thing.

You can use the cURL command line tool to generate sample code as well, which is helpful for experimentation.

Answer 7

Use boost library and poco. They both provide solutions for network programming. Boost also provide spirit library which you can use for parsing data from websites. Poco libraru also provides NetSSL, crypto solutions.

P.S. boost::spirit is not a library for parsing data from websites, it provides solution for parsing strings ...