Replace newlines in a Unicode string

Question

I am trying to replace newline characters in a unicode string and seem to be missing some magic codes.

My particular example is that I am working on AppEngine and trying to put titles from HTML pages into a db.StringProperty() in my model.

So I do something like:

link.title = unicode(page_title,"utf-8").replace('\n','').replace('\r','')

and I get:

Property title is not multi-line

Are there other codes I should be using for the replace?

Answer 1

It would be useful to print the repr() of the page_title that is seen to be multiline, but the obvious candidate would be '\r'.

Answer 2

Try ''.join(unicode(page_title, 'utf-8').splitlines()). splitlines() should let the standard library take care of all the possible crazy Unicode line breaks, and then you just join them all back together with the empty string to get a single-line version.

Answer 3

Python uses these characters for splitting in unicode.splitlines():

• U+000A LINE FEED (\n)
• U+000D CARRIAGE RETURN (\r)
• U+001C FILE SEPARATOR
• U+001D GROUP SEPARATOR
• U+001E RECORD SEPARATOR
• U+0085 NEXT LINE
• U+2028 LINE SEPARATOR
• U+2029 PARAGRAPH SEPARATOR

As Hank says, using splitlines() will let Python take care of all of the details for you, but if you need to do it manually, then this should be the complete list.