Problem with retrieving XML via HTTP and writing to file

Answer 1

Try:

XmlDocument doc = new XmlDocument();
doc.Load(url);
doc.Save(filename);

It really is that easy (with some error handling obviously). The .Net framework should do everything for you. I jumped through hoops a month or so ago trying to do the same thing and kicked myself when I read the help file on XmlDocument ;)

Answer 2

It could be as simple as not calling Flush() on your StreamWriter, but why make life hard for yourself? Replace the whole writeXMLtoFile function with this:

public static void writeXMLtoFile()
{
    string url = "http://somevalidurl.com/dataPage.php?lotsofpars=true";
    string xml = ScreenScrape(url);
    File.WriteAllText("xml\\myFile.xml", xml);
}

This way, you can also use the debugger to see what's going on (inspect the xml variable).

Answer 3

Additionally, if really all you want to do is download a page to the file system, investigate the WebClient.DownloadFile method :)