Get Text From All Elements Matching a Pattern in Selenium

Question

I have a site with elements of the form:

<td id="subject_23432423">content I want to read</td>

How do I use Selenium RC (with the Python bindings specifically) to read the content from all these elements? I've gone through all the commands, and while there's a lot of options to find a single element, none of the commands seem to handle lists of multiple matches. For example, I can find the content of a specific element using:

content = sel.get_text("td[@id='subject_23432423']")

but this assumes I already know the id, which I don't because it's generated dynamically.

Answer 1

It is impossible with the Selenium 1 API, however you can call a JavaScript that will locate elements using XPath //td[contains(@id, "subject_")] if the subject_ is always present in the generated id. I am not sure if the Selenium browserbot provides an XPath support for IE, so you may me limited to the browsers that have native support. In Firefox it will be:

var tds = document.evaluate("//td[contains(@id, \"subject_\")]", document, null,
        XPathResult.ORDERED_NODE_SNAPSHOT_TYPE, null); 
for ( var i = 0; i < tds.snapshotLength; i++) {
   var td = tds.snapshotItem(i);
   // get text using td.textContent and add it to array or whatever...
   // and return it
}

return ...

You will need to declare this script as a string in your code and execute via selenium.getEval

However if you are able to switch to Selenium 2 (WebDriver), you can use its API. You will need to use the findElementsBy passing the XPath //td[contains(@id, "subject_")] and then iterate through the returned array of matches and get the text of each element

Answer 2

What I would do is one of the following techniques

count = sel.get_xpath_count("xpath=//td[starts-with(@id,'subject_')]")
someArray = []
for i in count:
  someArray[i] = sel.get_text("xpath=//td[starts-with(@id,'subject_')][" + i + "]")

or for a more effiecent way to use BeautifulSoup or lxml

html = sel.get_html_source()
from BeautifulSoup import BeautifulSoup
soup = BeautifulSoup(html)
#use beautifulsoup to do what you want

Answer 3

It certainly must be possible in Selenium since this type of thing works fine in TestPlan (which uses Selenium and HTMLUnit as the backend). In this case the simple TestPlan script might look like below.

for %Element% in (response //td[starts-with(@id,'subject_')])
  Notice %Element%
end

The conversion to string is called automatically when writing a notice, but to store in an array it is just as easy.