I've an application that create many web request to donwload the news pages of a web site (i've tested for many web sites) after a while I find out that the application slows down in fetching the html source then I found out that HttpWebResonse fails getting the response. I post only the function that do this job. public PageFetchR...
When making a request using HttpWebRequest object, I need to call the method GetResponse() to send the request and get the response back. The problem with this method is that it doesn't return the response object until all data has been received. Say I am downloading a 100 MB file, I won't be able to read it until the response finish and...
Hi I have built a simple threadpool based web crawler within my web application. Its job is to crawl its own application space and build a Lucene index of every valid web page and their meta content. Here's the problem. When I run the crawler from a debug server instance of Visual Studio Express, and provide the starting instance as the ...
I'm trying to read the response stream from an HttpWebResponse object. I know the length of the stream (_response.ContentLength) however I keep getting the following exception: Specified argument was out of the range of valid values. Parameter name: size While debugging, I noticed that at the time of the error, the values were as suc...
Hello all i'm trying to log in a web app from my .net application, but for some reason it doesn't work. Here is the login code: <form action="./process-login.php" method="post"> <table border="0" cellpadding="5" cellspacing="0"> <tr> <td>Username:</td> <td><input type="text" size="20" name="username" v...
I am trying to create a C# Winforms application that will automatically log me into a site and download data. Specifically, I want to have my application automatically log into my online banking site, log me in, and download my transaction history. I can do this manually by logging in through a web browser and downloading it. I am trying...
Looking for the best way to determine if a URI exists in VB.NET without downloading the actual content. System.IO.FileExists/My.Computer.FileSystem.FileExists can be used locally to determine if a file exists, is there an equivalent for the Web? Currently I am using a HttpWebRequest to check URI existance using the ResponseStream. Thi...
I need to check whether the request will return a 500 Server Internal Error or not (so getting the error is expected). I'm doing this: HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest; request.Method = "GET"; HttpWebResponse response = request.GetResponse() as HttpWebResponse; if (response.StatusCode == HttpStatusC...
I have written a web scraping program to go to a list of pages and write all the html to a file. The problem is that when I pull a block of text some of the characters get written as '�'. How do I pull those characters into my text file? Here is my code: string baseUri = String.Format("http://www.rogersmushrooms.com/gallery/loadimage...
I am using the following link as a base for implementing myspace oAuth consumer in my website. http://stevenng.net/2009/03/17/using-myspace-rest-api-with-oauth-and-c/ When i use the basic oAuthBase.cs class i receive the 401 unauthorized error but when i use the updated oAuthBase.cs as mentioned by the blogger in the above post in respo...
Hello, i made some WebRequest's and got Responses to a site and succesfully posted data for 3 pages. Now the last page is a confirmation page and i need to make the user click the confirm button. But when i write response content to the browser like this response = request.GetResponse() as HttpWebResponse; dataStream = response.GetResp...
I wrote this method to test for credentials but I dunno why when it goes to the GetResponse method is actually goes and runs the webservice. I just want it to test if the credentials for that web service are correct or not. private bool TestCredentials(string sURL, ref string sUsername, ref string sPassword) { bool ret...
I'm familiar with returning a CSV attachment during a postback using something like this: Response.Clear() Response.ContentType = "text/csv" Response.AppendHeader("Content-Disposition", "attachment;filename=Test.csv") Response.Write(LoadTextFile(AppPath & "Test.csv")) Response.End() However, is it possible to retur...
I have a Silverlight application hosted in an ASP.NET site, through which I'm starting an HttpWebRequest to a Generic Handler in order to save a CSV file to the user's machine. From the Silverlight app, a Uri is constructed with parameters to make the CSV file server-side. A button is clicked which triggers the following: string http...
Hi Guys, I'm trying to figure out how many times my web request was redirected before I finally landed at the end content. I'm creating my web request as follows: var httpRequest = (HttpWebRequest) WebRequest.Create("some arb path"); httpRequest.AllowAutoRedirect = followRedirects; I've had a look at the following URL http://msdn.m...
I'm just curious about whether this is possible - I know how to download a file, but how can I download only the first 100KB of a file? ...
<?xml version="1.0" encoding="utf-8"?> <rsp stat="ok"> <image_hash>cxmHM</image_hash> <delete_hash>NNy6VNpiAA</delete_hash> <original_image>http://imgur.com/cxmHM.png</original_image> <large_thumbnail>http://imgur.com/cxmHMl.png</large_thumbnail> <small_thumbnail>http://imgur.com/cxmHMl...
Hello all... I have an application written in VB.NET (NOT asp.net, it is a Windows Console app). I am trying to call a url (an html page) and get back the response into a string. The response is straight JSON, no html tags whatsoever. It opens with { and closes with }. I create the HttpWebRequest object fine. Then call req.GetResponse(...
I'm writing a .NET program that downloads web pages using proxy servers from a list. I'm running into the problem that sometimes a proxy server will return its own junk content, instead of returning the content from the intended page. The download will appear to succeed, but when you look at the downloaded HTML, it will have obviously ...
I want to navigate to a page in a web app from a desktop app. "No problem", I hear you say, "just fire up the default browser with the correct URL". However, the web app uses ASP.NET Forms Authentication, and the users don't want to see the login page because they have already authenticated with the same credentials in the desktop app. ...