How to compare string data to table data in SQL Server - I need to know if a value in a string doesn't exist in a column

Question

I have two tables, one an import table, the other a FK constraint on the table the import table will eventually be put into. In the import table a user can provide a list of semicolon separated values that correspond to values in the 2nd table.

So we're looking at something like this:

TABLE 1
ID | Column1
1  | A; B; C; D

TABLE 2
ID  | Column2
1   | A
2   | B
3   | D
4   | E

The requirement is:

Rows in TABLE 1 with a value not in TABLE 2 (C in our example) should be marked as invalid for manual cleanup by the user. Rows where all values are valid are handled by another script that already works.

In production we'll be dealing with 6 columns that need to be checked and imports of AT LEAST 100k rows at a time. As a result I'd like to do all the work in the DB, not in another app.

BTW, it's SQL2008.

I'm stuck, anyone have any ideas. Thanks!

Answer 1

Perhaps inserting those composite values into 'TABLE 1' may have seemed like the most convenient solution at one time. However, unless your users are using SQL Server Management Studio or something similar to enter the values directly into the table then I assume there must be a software layer between the UI and the database. If so, you're going to save yourself a lot headaches both now and in the long run by investing a little time in altering your code to split the semi-colon delimited inputs into discrete values before inserting them into the database. This will result in 'TABLE 1' looking something like this

TABLE 1
ID  | Column1
1   | A
1   | B
1   | C
1   | D

It's then trivial to write the SQL to find those IDs which are invalid.

Answer 2

Seems to me you could pass ID & Column1 values from Table1 to a Table-Valued function (or a temp table in-line) which would parse the ;-delimited list, returning individual values per record.

Here are a couple options: http://pietschsoft.com/post/2006/02/T-SQL-Parse-a-delimited-string.aspx http://codebetter.com/blogs/raymond.lewallen/archive/2005/10/26/133774.aspx

The result (ID, value) from the function could be used to compare (unmatched query) against values in Table 2.

SELECT tmp.ID FROM tmp LEFT JOIN Table2 ON Table2.id = tmp.ID WHERE Table2.id is null

The ID results of the comparison would then be used to flag records in Table 1.

Answer 3

If it is possible, try putting the values in separate rows when importing (instead of storing it as ; separated).

This might help - http://www.sommarskog.se/arrays-in-sql-2005.html