Is WebRequest The Right C# Tool For Interacting With Websites?

Question

I'm writing a small tool in C# which will need to send and receive data to/from a website using POST and json formatting. I've never done anything like this before in C# (or any language really) so I'm struggling to find some useful information to get me started.

I've found some information on the WebRequest class in C# (specifically from here) but before I start diving into it, I wondered if this was the right tool for the job.

I've found plenty of tools to convert data into the json format but not much else, so any information would be really helpful here in case I end up down a dead end.

Answer 1

I have used WebRequest for interacting with websites. It is the right 'tool'

I can't comment on the JSON aspect of your question.

Answer 2

in 3.5 there is a built-in jsonserializer. The webrequest is the right class your looking for.

A few examples http://geekswithblogs.net/JuanDoNeblo/archive/2007/10/24/json_in_aspnetajax_part2.aspx http://dev.aol.com/blog/markdeveloper/ShareFileWithNETFramework http://geekswithblogs.net/JuanDoNeblo/archive/2007/10.aspx

Answer 3

WebClient is sometimes easier to use than WebRequest. You may want to take a look at it.

For JSON deserialization you are going to want to look at the JavaScriptSerializer class.

WebClient example:

using (WebClient client = new WebClient ())
{
    //manipulate request headers (optional)
    client.Headers.Add (HttpRequestHeader.UserAgent, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)");

    //execute request and read response as string to console
    using (StreamReader reader = new StreamReader(client.OpenRead(targetUri)))
    {
        string s = reader.ReadToEnd ();
        Console.WriteLine (s);
    }
}

Marked as wiki in case someone wants to update the code

Answer 4

When it comes to POSTing data to a web site, System.Net.HttpWebRequest (the HTTP-specific implementation of WebRequest) is a perfectly decent solution. It supports SSL, async requests and a bunch of other goodies, and is well-documented on MSDN.

The payload can be anything: data in JSON format or whatever -- as long as you set the ContentType property to something the server expects and understands (most likely application/json, text/json or text/x-json), all will be fine.

One potential issue when using HttpWebRequest from a system service: since it uses the IE proxy and credential information, default behavior may be a bit strange when running as the LOCALSYSTEM user (or basically any account that doesn't log on interactively on a regular basis). Setting the Proxy and Authentication properties to Nothing (or, as you C# folks prefer to call it, null, I guess) should avoid that.

Answer 5

WebRequest and more specifically the HttpWebRequest class is a good starting point for what you want to achieve. To create the request you will use the WebRequest.Create and cast the created request to an HttpWebRequest to actually use it. You will then create your post data and send it to the stream like:

HttpWebRequest req = (HttpWebRequest)
WebRequest.Create("http://mysite.com/index.php");
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
string postData = "var=value1&var2=value2";
req.ContentLength = postData.Length;

StreamWriter stOut = new
StreamWriter(req.GetRequestStream(),
System.Text.Encoding.ASCII);
stOut.Write(postData);
stOut.Close();

Similarly you can read the response back by using the GetResponse method which will allow you to read the resultant response stream and do whatever else you need to do. You can find more info on the class at:

http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.aspx