What's the best way to get an array of all the URLs in a web page? and how would I do it?
+1
A:
Using HTML Agility Pack is a good way, maybe not the best as this would be subjective but I can tell you the worst and this is using regular expression to parse html (as you've tagged your question with regex I feel myself ion the obligation to point this out).
Darin Dimitrov
2010-09-04 10:57:11
Why has this been downvoted? Please leave a comment when downvoting an answer to express your opinion on why do you think this answer is wrong.
Darin Dimitrov
2010-09-04 11:57:46
A:
/<a href=\"([^\"]*)\">(.*)<\/a>/iU
or use this previous answer:
http://stackoverflow.com/questions/6173/regular-expression-for-parsing-links-from-a-webpage
Pierre 303
2010-09-04 10:59:11
I can't believe there are answers suggesting using regex to do screen scraping.
Darin Dimitrov
2010-09-04 11:05:31
Darin, I don't suggest, I answer the question, the guy is an adult, not a kid. He has a your answer to think about it.
Pierre 303
2010-09-04 11:10:20
@Pierre 303, SO is not for just answering questions without thinking. It's primary for advocating good practices on how to solve programming related problems. SO is a pretty well referenced site and many people are reading it as a source of information on good practices, and suggesting to use regex to parse HTML in C# is simply [not a good practice](http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454). I am sorry.
Darin Dimitrov
2010-09-04 11:59:05
Darin, except to prove your technical supperiority, I really don't understand your obstination. Maybe being down voted by someone else than me hurted your feelings ? Please put yourself in the shoes of the guy/girl that answer the question. He/she is not asking for an advice in his/her question, he/she is asking for an answer. It's what I provide. If he wants advice, you have provided in your answer. So SO is a great place to have DIFFERENT answers, and not only ONE from the supperior technical guys.
Pierre 303
2010-09-04 12:14:55
Oh, and he/she doesn't want to specifically parse HTML, he/she wants to retrieve ALL URL contained in a text document, that, in her/his case, is an HTML document. Parsing HTML looks like an overhead/overengeneering to me. Will you pay for that instead of having a simple solution that just works?
Pierre 303
2010-09-04 12:20:09
thanks Pierre 303. How would I retrieve the results and convert them into an array in C#?
Rana
2010-09-04 18:34:57
There is a full example at http://dotnetperls.com/scraping-html and a more simple one at http://oreilly.com/windows/archive/csharp-regular-expressions.html
Pierre 303
2010-09-04 18:37:53