How to get all links avaliable from server on some port?

Question

So I have lots of links like address:port/Bla-bla/bla-bla/file/blabla234times/ created by some server (like VLC) how can I get all links (list of them) avaliable from server (abstract server working on windows) from my C# programm?

So how to get a list or URLs of ALL existing files and folders on server. We have adress and port. (no HTML pages nothing - just server adress)

I want to scan a server to retrive its links

Answer 1

If you are interested in extracting anchor links from HTML document sent by a web server you could use Html Agility Pack to parse this document:

class Program
{
    static void Main()
    {
        var document = new HtmlDocument();
        using (var client = new WebClient())
        using (var reader = new StringReader(client.DownloadString("http://www.google.com")))
        {
            document.Load(reader);
        }

        var anchors = document.DocumentNode.SelectNodes("//a");
        foreach (var anchor in anchors)
        {
            Console.WriteLine(anchor.Attributes["href"].Value);
        }
    }
}