Performance implications of sql 'OR' conditions when one alternative is trivial ?

Question

I'm creating a stored procedure for searching some data in my database according to some criteria input by the user.

My sql code looks like this:

Create Procedure mySearchProc
(
@IDCriteria bigint=null,
...
@MaxDateCriteria datetime=null
)
as
select Col1,...,Coln from MyTable 
where (@IDCriteria is null or ID=@IDCriteria)
...
and (@MaxDateCriteria is null or Date<@MaxDateCriteria)

Edit : I've around 20 possible parameters, and each combination of n non-null parameters can happen.

Is it ok performance-wise to write this kind of code? (I'm using MS SQL Server 2008)

Would generating SQL code containing only the needed where clauses be notably faster?

Answer 1

OR clauses are notorious for causing performance issues mainly because they require table scans. If you can write the query without ORs you'll be better off.

Answer 2

where (@IDCriteria is null or ID=@IDCriteria)
  and (@MaxDateCriteria is null or Date<@MaxDateCriteria)

If you write this criteria, then SQL server will not know whether it is better to use the index for IDs or the index for Dates.

For proper optimization, it is far better to write separate queries for each case and use IF to guide you to the correct one.

IF @IDCriteria is not null and @MaxDateCriteria is not null

  --query
  WHERE ID = @IDCriteria and Date < @MaxDateCriteria

ELSE IF @IDCriteria is not null

  --query
  WHERE ID = @IDCriteria

ELSE IF @MaxDateCriteria is not null

  --query
  WHERE Date < @MaxDateCriteria

ELSE

  --query
  WHERE 1 = 1

If you expect to need different plans out of the optimizer, you need to write different queries to get them!!

Would generating SQL code containing only the needed where clauses be notably faster?

Yes - if you expect the optimizer to choose between different plans.

---

Edit:

DECLARE @CustomerNumber int, @CustomerName varchar(30)

SET @CustomerNumber = 123
SET @CustomerName = '123'

SELECT * FROM Customers
WHERE (CustomerNumber = @CustomerNumber OR @CustomerNumber is null)
  AND (CustomerName = @CustomerName OR @CustomerName is null)

CustomerName and CustomerNumber are indexed. Optimizer says : "Clustered Index Scan with parallelization". You can't write a worse single table query.

---

Edit : I've around 20 possible parameters, and each combination of n non-null parameters can happen.

We had a similar "search" functionality in our database. When we looked at the actual queries issued, 99.9% of them used an AccountIdentifier. In your case, I suspect either one column is -always supplied- or one of two columns are always supplied. This would lead to 2 or 3 cases respectively.

It's not important to remove OR's from the whole structure. It is important to remove OR's from the column/s that you expect the optimizer to use to access the indexes.

Answer 3

Regarding "Would generating SQL code containing only the needed where clauses be notably faster?"

I don't think so, because this way you effectively remove the positive effects of query plan caching.

Answer 4

You could perform selective queries, in order of the most common / efficient (indexed etc), parameters, and add PK(s) to a temporary table

That would create a (hopefully small!) subset of data

Then join that Temporary Table with the main table, using a full WHERE clause with

SELECT ...
FROM @TempTable AS T
    JOIN dbo.MyTable AS M
        ON M.ID = T.ID
WHERE (@IDCriteria IS NULL OR M.ID=@IDCriteria)
...
AND (@MaxDateCriteria IS NULL OR M.Date<@MaxDateCriteria)

style to refine the (small) subset.

Answer 5

What if constructs like these were replaced:

WHERE (@IDCriteria IS NULL OR @IDCriteria=ID)
AND (@MaxDateCriteria IS NULL OR Date<@MaxDateCriteria)
AND ...

with ones like these:

WHERE ID = ISNULL(@IDCriteria, ID)
AND Date < ISNULL(@MaxDateCriteria, DATEADD(millisecond, 1, Date))
AND ...

or is this just coating the same unoptimizable query in syntactic sugar?

Answer 6

Choosing the right index is hard for the optimizer. IMO, this is one of few cases where dynamic SQL is the best option.

Answer 7

So, to boil down the above comments:

Create a separate sub-procedure for each of the most popular variations of specific combinations of parameters, and within a dispatcher procedure call the appropriate one from an IF ELSE structure, the penultimate ELSE clause of which builds a query dynamically to cover the remaining cases.

Perhaps only one or two cases may be specifically coded at first, but as time goes by and particular combinations of parameters are identified as being statistically significant, implementation procedures may be written and the master IF ELSE construct extended to identify those cases and call the appropriate sub-procedure.

Answer 8

this is one of the cases i use code building or a sproc for each searchoption. since your search is so complex i'd go with code building. you can do this either in code or with dynamic sql. just be careful of SQL Injection.

Answer 9

I suggest one step further than some of the other suggestions - think about degeneralizing at a much higher abstraction level, preferably the UI structure. Usually this seems to happen when the problem is being pondered in data mode rather than user domain mode.

In practice, I've found that almost every such query has one or more non-null, fairly selective columns that would be reasonably optimizable, if one (or more) were specified. Furthermore, these are usually reasonable assumptions that users can understand.

Example: Find Orders by Customer; or Find Orders by Date Range; or Find Orders By Salesperson.

If this pattern applies, then you can decompose your hypergeneralized query into more purposeful subqueries that also make sense to users, and you can reasonably prompt for required values (or ranges), and not worry too much about crafting efficient expressions for subsidiary columns.

You may still end up with an "All Others" category. But at least then if you provide what is essentially an open-ended Query By Example form, then users will have some idea what they're getting into. Doing what you describe really puts you in the role of trying to out-think the query optimizer, which is folly IMHO.

Answer 10

I'm currently working with SQL 2005, so I don't know if the 2008 optimizer acts differently. That being said, I've found that you need to do a couple of things...

1. Make sure that you are using WITH (RECOMPILE) for your query

2. Use CASE statements to cause short-circuiting of the logic. At least in 2005 this is NOT done with OR statements. For example:

.

SELECT
     ...
FROM
     ...
WHERE
     (1 =
          CASE
               WHEN @my_column IS NULL THEN 1
               WHEN my_column = @my_column THEN 1
               ELSE 0
          END
     )

The CASE statement will cause the SQL Server optimizer to recognize that it doesn't need to continue past the first WHEN. In this example it's not a big deal, but in my search procs a non-null parameter often meant searching in another table through a subquery for existence of a matching row, which got costly. Once I made this change the search procs started running much faster.

Answer 11

My suggestion is to build the sql string. You will gain maximum performance from index and reuse execution plan.

DECLARE @sql nvarchar(4000);
SET @sql = N''

IF @param1 IS NOT NULL
    SET @sql = CASE WHEN @sql = N'' THEN N'' ELSE N' AND ' END + N'param1 = @param1';
IF @param2 IS NOT NULL
    SET @sql = CASE WHEN @sql = N'' THEN N'' ELSE N' AND ' END + N'param2 = @param2';
...
IF @paramN IS NOT NULL
    SET @sql = CASE WHEN @sql = N'' THEN N'' ELSE N' AND ' END + N'paramN = @paramN';

IF @sql <> N''
    SET @sql = N' WHERE ' + @sql;
SET @sql = N'SELECT ... FROM myTable' + @sql;

EXEC sp_executesql @sql, N'@param1 type, @param2 type, ..., @paramN type', @param1, @param2, ..., @paramN;

Answer 12

Each time the procedure is called, passing different parameters, there is a different optimal execution plan for getting the data. The problem being, that SQL has cached an execution plan for your procedure and will use a sub-optimal (read terrible) execution plan.

I would recommend:

1. Create specific SPs for frequently run execution paths (i.e. passed parameter sets) optimised for each scenario.
2. Keep you main generic SP for edge cases (presuming they are rarely run) but use the WITH RECOMPILE clause to cause a new execution plan to be created each time the procedure is run.

We use OR clauses checking against NULLs for optional parameters to great affect. It works very well without the RECOMPILE option so long as the execution path is not drastically altered by passing different parameters.