Read Webpage

Question

Hello, I was wondering if anyone knew how to read text from a webpage from within C++.

After you first connect to the internet using usual means,

you enter the string e.g. "http://www.bbc.co.uk" and the C++ program reads the information:

"BBC WebsiteThis is the BBC website...."

from the internet.

Would I have to write a program that sends lots of data to and from the modem to do this or is there a simpler way of doing this? Is there some sort of command line program in Windows that does this all for you? Either way if you have any tips of know of any tutorials on the web that would be great.

I would like to know because I think it would be fun to write applications that could read information off the internet, follow links and so forth and I might think of a good game that utilises this.

Thank.

Paul.

Answer 1

You could use wget to fetch pages, but you would still need to parse the output.

You don't need to send data to the modem. All you need to do is connect to the webserver via sockets and retrieve the data from there. Look for literature on socket programming and the http-protocol.

Answer 2

Not directly from C++, but sources are available: there is Lynx text browser.

Answer 3

You can do this pretty easily with boost. Check Asio.

Http Client Example (downloads and displays a .txt file from the internet)

Answer 4

I would recommend lib cURL for http in C/C++ but that still leaves out parsing the html to render the actual text from tags.