C# HTTP programming

Question

Hello there, i want to build a piece of software that will process some html forms, the software will be a kind of bot that will process some forms on my website automatically.

Is there anyone who can give me some basic steps how to do this job...Any tutorials, samples, books or whatever can help me.

Can some of you post an working code with POST method ?

Answer 1

Check out How to: Send Data Using the WebRequest Class. It gives an example of how create a page that posts to another page using the HttpWebRequest class.

Answer 2

To fill out the form...

1. Find all of the INPUT or TEXTAREA elements that you want to fill out.
2. Build the data string that you are going to send back to the server. The string is formatted like "name1=value1&name2=value2" (just like in the querystring). Each value will need to be URL encoded.
3. If the form's "method" attribute is "GET", then take the URL in the "action" attribute, add a "?" and the data string, then make a "GET" web request to the URL.
4. If the form's "method" is "POST", then the data is submitted in a different area of the web request. Take a look at this page for the C# code.

Answer 3

To expand on David and JP's answers':

Assuming you're working with forms whose contents you're not familiar with, you can probably...

1. pull the page with the form via an HttpWebRequest.
2. load it into an XmlDocument
3. Use XPath to traverse/select the form elements.
4. Build your query string/post data based on the elements.
5. Send the data with HttWebRequest

If the form's structure is known in advance, you can really just start at #4.

(untested) example (my XPath is not great so the syntax is almost certainly not quite right):

HttpWebRequest request;
HttpWebResponse response;
XmlDocument xml = new XmlDocument();
string form_url = "http://...."; // you supply this
string form_submit_url;
XmlNodeList element_nodes;
XmlElement form_element;
StringBuilder query_string = new StringBuilder();

// #1
request = (HttpWebRequest)WebRequest.Create(form_url));
response = (HttpWebResponse)request.GetResponse();

// #2
xml.Load(response.GetResponseStream());

// #3a
form_element = xml.selectSingleNode("form[@name='formname']");
form_submit_url = form_element.GetAttribute("action");

// #3b
element_nodes = form_element.SelectNodes("input,select,textarea", nsmgr)

// #4
foreach (XmlNode input_element in element_nodes) {
  if (query_string.length > 0) { query_string.Append("&"); }
  // MyFormElementValue() is a function/value you need to provide/define.
  query_string.Append(input_element.GetAttribute("name") + "=" + MyFormElementValue(input_element.GetAttribute("name"));
}

// #5
// This is a GET request, you can figure out POST as needed, and deduce the submission type via the <form> element's attribute.
request = (HttpWebRequest)WebRequest.Create(form_submit_url + "?" + query_string.ToString()));

References:

• http://geekswithblogs.net/dtotzke/articles/24571.aspx
• http://www.developerfusion.com/forum/thread/26371/
• http://msdn.microsoft.com/en-us/library/system.xml.xmlelement.getattribute.aspx
• http://msdn.microsoft.com/en-us/library/system.xml.xmlelement.selectnodes.aspx

Answer 4

If you don't want to go the HttpWebRequest route, I would suggest WatiN. Makes it very easy to automate IE or Firefox and not worry about the internals of the HTTP requests.