views:

403

answers:

1

Hi,

I'm having problems with Full-Text Search on SQL Server 2005. In my setup, I am indexing a single column, let's call it Content. The indexing is done in the neutral culture since the column might contain text in different languages. The fulltext index is created as follows:

CREATE FULLTEXT INDEX
 ON [dbo].[Table1]([Content])
KEY INDEX [UI_Table1_Id] ON [Catalog]
WITH CHANGE_TRACKING AUTO

This table is then filled. The users can then query against the index. The queries look somewhat like this:

SELECT * FROM Table1 AS table1
INNER JOIN CONTAINSTABLE (Table1, Content, @0 , LANGUAGE 1033) AS KEY_TBL 
ON table1.Id = KEY_TBL.[KEY]
WHERE table1.locale = 'en-US'

As I said, the content column contains different languages. thus the LANGUAGE (and table1.locale = 'en-US' in the CONTAINSTABLE may change, to be e.g. Danish, English or Swedish LCID's.

I'm having one problem, though. If the column is filled a text containing the word "koncepttitel" and I query for it, I get no hits if using the Swedish language (LANGUAGE 1053). I will get a hit if I use English (LANGUAGE 1033) for the same word.

Previously I got the "Informational: The full-text search condition contained noise word(s)." error message. I then cleared the Swedish stop word list. Now I get no error message, but still I can't seem to get a hit for my query.

Is there any way for me to configure SQL Server Full-Text Search to output more diagnostic information than this? Is there e.g. a way for me to see which noise word the full-text search condition contained?

The thing is, I don't care so much for my users searching for this specific word. However, I'm worried that this error may be across more relevant search terms, which I won't be able to foresee, which means that my users won't be able to find what they are looking for.

Update: I'm wondering if I may have misinterpreted the set-up for full-text search. Could this issue be due to me indexing the content in the neutral culture and querying in a specific culture? Should I always use the neutral culture when querying?

A: 

In my dealings with FTS and noise files, you need to restart the service (FTS service) then run an update population.

Depending on the size you may just want to consider dropping the index and recreating it.

doug_w
Update population, is that ALTER FULLTEXT CATALOG <catalog> REBUILD?
Thomas Lundström
@Thomas - Yes, you need to repopulate the full text catalog, not update. Sorry for the confusion.
doug_w
Ok, that I've done. As I mentioned in my update of the question above, I'm afraid I've misinterpreted the way I query for data when I've indexed it in the neutral culture.
Thomas Lundström
@Thomas - when you query for the term 'koncepttitel' can you find the records that you are looking for using the like operator? If not you may need to consider indexing views that segment the table by language. The problem with the neutral language word breaker is that it does not know how to conjugate verbs or in the case of German, compound nouns. This is what MSDN has to say about specifying the language in the query ' If language_term is specified, the language it represents will be applied to all elements of the search condition.' It will not however rebreak the text you are searching
doug_w
Yup, I can find the word using like (or, for that matter, when I query using the neutral culture or another language, e.g. english).My hunch right now, and the MSDN text kind of supports my thesis is as follows: The index contains the word broken according to the neutral culture. When I query for 'koncepttitel', which btw is a compound noun in Swedish, using the swedish lcid, there's a mis-match between the index and the query. Thus, the FTS engine doesn't find the row I'm looking for.
Thomas Lundström
@Thomas - The only other option I can come up with for you is to create indexed views which segment your data by language. You can then Full Text Index those views and query them using the appropriate language for each language that you support. Is it possible for you to create these indexed views? There is a summary of indexed view restrictions here: http://technet.microsoft.com/en-us/library/cc917715.aspx . Hope that helps
doug_w