Really weird HTTP client using TcpClient in C#

Answer 1

Try using System.Net.WebClient instead of System.Net.Sockets.TcpClient directly:

using System;
using System.Net;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            WebClient wc = new WebClient();
            Console.WriteLine("[requesting...]");
            Console.WriteLine(wc.DownloadString("http://www.google.com"));
            Console.WriteLine("[done!]");
            Console.ReadKey();
        }
    }
}

Answer 2

I found one question in all that:

How i can read the response and be sure i read all of the response in HTTP/1.1 request?

And that is a question I can answer!

All the methods you're using here are synchronous, which is easy to use but not even slightly reliable. You'll see problems as soon as you have a sizable response and only get part of it.

To implement a TcpClient connection most robustly, you should use all asynchronous methods and callbacks. The relevant methods are as follows:

1) Create the connection with TcpClient.BeginConnect(...) with the callback calling TcpClient.EndConnect(...)
2) Send a request with TcpClient.GetStream().BeginWrite(...) with the callback calling TcpClient.GetStream().EndWrite(...)
3) Receive a response with TcpClient.GetStream().BeginRead(...) with the callback calling TcpClient.GetStream().EndRead(...), appending the result to a StringBuilder buffer, and then calling TcpClient.GetStream().BeginRead(...) again (with the same callback) until a response of 0 bytes is received.

It's that final step (repeatedly calling BeginRead until 0 bytes are read) that solves the problem of fetching the response, the whole response, and nothing but the response. So help us TCP.

Hope that helps!

Answer 3

I suggest you try your code against a standard, well tested, largely accepted web server installed on your own local machine, such as Apache HTTPD, or IIS.

Configure your web server to respond without the Host header (e.g. a default web application in IIS) and see if all goes well.

At the bottom line, you can't really tell what goes on behind the scenes, since you don't control web sites / web applications like google, yahoo, etc.
For instance, a web site administrator can configure the site so that there's no default application for incoming TCP connections on port 80, using HTTP protocol.
But he/she may want to configure a default telnet application when connecting via TCP port 23, using TELNET protocol.