I need to download a text file from the internet using C#. The file size can be quite large and the information I need is always within the first 1000 bytes. Is this possible?
+7
A:
Stolen from here.
string GetWebPageContent(string url)
{
string result = string.Empty;
HttpWebRequest request;
const int bytesToGet = 1000;
request = WebRequest.Create(url) as HttpWebRequest;
//get first 1000 bytes
request.AddRange(0, bytesToGet - 1);
// the following code is alternative, you may implement the function after your needs
using (WebResponse response = request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
byte[] buffer = new byte[1024];
int read = stream.Read(buffer, 0, 1000);
Array.Resize(ref buffer, read);
return Encoding.ASCII.GetString(buffer);
}
}
}
(Edited as requested in the comments... ;) )
Marko
2010-10-04 07:58:26
I would suggest replacing the `ReadToEnd` with a read of (at most) 1000 characters in case the server ignored the range request.
Richard
2010-10-04 08:00:52
@Richard: That wouldn't be necessary since a range header was added to the request. The response will be at most 1000 characters.
Jeff M
2010-10-04 08:06:11
@Jeff: as my comment said "in case the server ignored the range request" as the `Range` header is *optional* for the server and can be ignored.
Richard
2010-10-04 08:09:28
I have to agree with Richard. As per RFC http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35.2 the server *may* ignore the `Range` header, so that you should only read as much as you want and the close the connection.
Lucero
2010-10-04 08:11:31
@Richard: Oh good point. I totally overlooked that.
Jeff M
2010-10-04 08:13:51
so what's the correct answer?
Rana
2010-10-04 18:47:22
@Rana the answer I provided is partially correct. It will work but as per @Richard's comment, the server might ignore the request, in which case you want to limit the StreamReader to only read the first 1000 characters. I'm not 100% sure how to implement that but someone else might have a solution. If anyone does know, please feel free to copy the code above and modify it, it wasn't really mine to begin with :)
Marko
2010-10-04 19:10:03
thanks Marko. does anyone know how to do that? limit streamreader to only read the first 1000 characters?
Rana
2010-10-04 23:22:25
Perhaps start a new question @Rana - and copy my code above as your work in progress.
Marko
2010-10-05 01:38:49
@Timwi, you deserve at least half of the rep points for this answer. So I've just upvoted 4 of your ones. Thank you.
Marko
2010-10-05 23:57:46
+1
A:
I did this as an answer to your newer question. You could put the range header in too if you want, but I excluded it.
string GetWebPageContent(string url)
{
//string result = string.Empty;
HttpWebRequest request;
const int bytesToGet = 1000;
request = WebRequest.Create(url) as HttpWebRequest;
var buffer = new char[bytesToGet];
using (WebResponse response = request.GetResponse())
{
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
sr.Read(buffer, 0, bytesToGet);
}
}
return new string(buffer);
}
Luke Schafer
2010-10-05 22:09:40
Technically this may get up to 2000 bytes if the StreamReader "thinks" that it gets back unicode. My edit (see above) on the other hand forces the encoding, which may be wrong as well. Rama needs to clarify the requirement I guess.
Lucero
2010-10-05 22:12:26