Does MySql full text search works reasonably with none latin languages (Henrew, Arabic, Japanaese...)

Question

Does MySql full text search works reasonably with none Latin languages (Hebrew, Arabic, Japanaese...)

Addition: Did some tests... It has some problems with Hebrew. Example: The name מוסינזון is pronounced the same as מושינזון but searching one won't find the other, as this is a common spelling error in Hebrew, it seems I will have to do some data manipulation for it to work perfectly.

Answer 1

So long as your collation is set properly, it works splendidly.

Unicode will work for most of this, of course. But that doesn't really translate Latin characters to them very well (for example, in a Dutch collation aa will be recognized as å).

Answer 2

Yes, however, check out what stopwords are.

Answer 3

Japanese and Chinese use their own whitespace symbols that MySQL does not understand.

Make sure that the words in the texts you are going to index are separated with ASCII separators (spaces, commas etc). Anything outside the ASCII range will probably not work.

Besides, you'll probably need to fix ft_min_word_len: by default, MySQL won't index words shorter than 4 characters, and most Japanese and Chinese words.

In Cyrillic languages transilteration errors are quite common.

All letters from this sequence: АВЕКМНОРСТуХ / ABEKMHOPCTyX are indistinguishable in most fonts.

The worst of them is Cyrillic С/ Latin C: both these symbols are located on one key on the keyboard and do not differ at all in most fonts, but they have different codes.

MySQL will not catch it either.