How to use unicode inside an xpath string? (UnicodeEncodeError)

Question

I'm using xpath in Selenium RC via the Python api.

I need to click an a element who's text is "Submit »"

Here's the error that I'm getting:

In [18]: sel.click(u"xpath=//a[text()='Submit \xbb')]")
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)

/Users/me/<ipython console> in <module>()

/Users/me/selenium.py in click(self, locator)
    282         'locator' is an element locator
    283         """
--> 284         self.do_command("click", [locator,])
    285 
    286 

/Users/me/selenium.py in do_command(self, verb, args)
    201         body = u'cmd=' + urllib.quote_plus(unicode(verb).encode('utf-8'))
    202         for i in range(len(args)):
--> 203             body += '&' + unicode(i+1) + '=' + urllib.quote_plus(unicode(args[i]).encode('utf-8'))
    204         if (None != self.sessionId):
    205             body += "&sessionId=" + unicode(self.sessionId)

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 28: ordinal not in range(128)

Answer 1

I think you just need to change

sel.click(u"xpath=//a[text()='Submit \xbb')]")

to

sel.click(u"xpath=//a[text()='Submit \xbb')]".encode('utf8'))

That's because the error indicates Selenium is trying to encode the Unicode object into a byte string (using the default codec for Python, that is, 'ascii') and that's what is failing; by explicitly encoding it yourself first, with what's presumably the right codec ('utf8', the default encoding in XML), you should therefore avoid this problem.

Answer 2

Does sel_click() expect unicode strings or utf-8 (byte) strings? The dreaded UnicodeEncodeError usually happens when you try to pass in the first when the latter (or some other encoding) is expected.

I can't try it right now, but you could try

"xpath=//a[text()='Submit \xc2\xbb')]"

instead of your argument (which you get by using .encode('utf-8')) on it.

Answer 3

sel.click(u"xpath=//a[text()='Submit \xbb')]")

It is possible to write XPath expressions that contain any Unicode characters.

For example:

//a[text()='Submit »')]