I need to load an external web (not local) page into my site (some link), but only a part of it. What are the options for doing so?
That depends on whether or not the external page is local, or on a different domain. If it's local, you can use $.load() in the jQuery library. This has an optional parameter to specify which element in the remote-dom to load it:
$("#links").load("/Main_Page #jq-p-Getting-Started li");
If the page is on another domain, you'll need a proxy script. You can do this with PHP and the phpQuery (php port of jQuery) library. You'll just use file_get_contents() to get the actual remote-dom, and then pull out the elements you want based on jQuery-like selectors.
To load a web page in .Net, use the HttpWebRequest class.
Example taken from MSDN, here:
private string StringGetWebPage(String uri)
{
const int bufSizeMax = 65536; // max read buffer size conserves memory
const int bufSizeMin = 8192; // min size prevents numerous small reads
StringBuilder sb;
// A WebException is thrown if HTTP request fails
try
{
// Create an HttpWebRequest using WebRequest.Create (see .NET docs)!
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
// Execute the request and obtain the response stream
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
// Content-Length header is not trustable, but makes a good hint.
// Responses longer than int size will throw an exception here!
int length = (int)response.ContentLength;
// Use Content-Length if between bufSizeMax and bufSizeMin
int bufSize = bufSizeMin;
if (length > bufSize)
bufSize = length > bufSizeMax ? bufSizeMax : length;
// Allocate buffer and StringBuilder for reading response
byte[] buf = new byte[bufSize];
sb = new StringBuilder(bufSize);
// Read response stream until end
while ((length = responseStream.Read(buf, 0, buf.Length)) != 0)
sb.Append(Encoding.UTF8.GetString(buf, 0, length));
}
catch (Exception ex)
{
sb = new StringBuilder(ex.Message);
}
return sb.ToString();
}
Note that this will return the entire page and not just a portion of it. You'll then need to sift through the page to find the information you're looking for.
Once you get the whole page as Michael Todd outlined, you will likely need to either use substring methods for a static means to slice up the content or you can use regex's for a more dynamic way to grab the content. An intro article on Regex's in ASP.Net can be found here. Good luck!