tags:

views:

113

answers:

1

Hi. I am trying to access a url, get the html from it and use xpaths to get certain values from it. I am getting the html just fine and Jtidy seems to be cleaning it appropriately. However, when I try to get the desired values using xpaths, I get an empty NodeList back. I know my xpath expression is correct; I have tested it in other ways. Whats wrong with this code. Thanks for the help.

String url_string = base_url + countries[c];
URL url = new URL(url_string);

Tidy tidy = new Tidy();
tidy.setShowWarnings(false);
tidy.setXHTML(true);
tidy.setMakeClean(true);
Document doc = tidy.parseDOM(url.openStream(), null);
//tidy.pprint(doc, System.out);

String xpath_string = "id('catlisting')//a";
XPath xpath = XPathFactory.newInstance().newXPath();
XPathExpression expr = xpath.compile(xpath_string);

NodeList nodes = (NodeList)expr.evaluate(doc, XPathConstants.NODESET);
System.out.println("size="+nodes.getLength());
for (int r=0; r<nodes.getLength(); r++) {
    System.out.println(nodes.item(r).getNodeValue()); 
}
+2  A: 

Try "//div[@id='catlisting']//a"

Maurice Perry
that worked! thanks alot! any idea as to why using id() doesn't work?
Not sure about that. Probably due to the way tidy builds the dom tree.
Maurice Perry