views:

44

answers:

2

We have TABLE A partitioned by date and does not contain data from today, it only contains data from prior day and going to year to date.

We have TABLE B also partitioned by date which does contain data from today as well as data from prior day going to year to date. On top of TABLE B there is a view, View_B which joins against View_C, View_D and left outer joins Table E. View_C and View_D are each selects from 1 table and do not have any other tables joined in. So View_B looks something like

SELECT b.Foo, c.cItem, d.dItem, E.eItem 
FROM TABLE_B b JOIN View_C c on c.cItem = b.cItem
JOIN View_D d on b.dItem = d.dItem
LEFT OUTER JOIN TABLE_E on b.eItem = e.eItem

View_AB joins TABLE A and View_B on extract date as well as one other constraint. So it looks something like:

SELECT a.Col_1, b.Col_2, ... 
FROM TABLE_A a LEFT OUTER JOIN View_B b 
on a.ExtractDate = b.ExtractDate and a.Foo=b.Foo

-- no where clause

When searching for data from anything other than prior day, the query analyzer does what would be expected and does a hash match join to complete the outer join and reads about 116 pages worth of data from table B. If run for prior day however, the query optimizer freaks out and uses a nested join, scans the table 7000+ times and reads 8,000,000+ pages in the join.

We can fake it/force it to use a different query plan by using join hints, however that causes any constraints in the view that look at table B to cause the optimizer to throw an error that the query can't be completed due to join hints.

Editing to add that the pages/scans = the same number as is hit in one scan when run for a prior day where the optimizer correctly chooses a hash instead of nested join.

As mentioned in the comments, we have severely reduced the impact by creating a covered index on TABLE_B to cover the join in View_B but the IO is still higher than it would be if the optimizer chose the correct plan, especially since the index is essentially redundant for all but prior day searches.

The sqlplan is at http://pastebin.com/m53789da9, sorry that it's not the nicely formatted version.

+1  A: 

If you can post the .sqlplan for each of the queries it would help for sure, but my hunch is that you are getting a parallel plan when querying for dates prior to the current day and the nested loop is possibly a constant loop over the partitions included in the table which would then spawn a worker thread for each partition (for more information on this, see the SQLCAT post on parallel plans with partitioned tables in Sql2005). Can't verify if this is the case or not without seeing the plans however.

chadhoc
SQL 2k8 QO behaves much nicer in regard to partitions
Remus Rusanu
yes it does, though given the title I'm assuming 2005...
chadhoc
I don't think that's what's causing the problem here, though it was a very interesting read. Running with a OPTION(MAXDOP 1) still has the query optimizer chugging through millions of pages of IO.It appears that the optimizer is getting partially strung up on nesteed views.
Rebthor
You mentioned nested views, do you mean you are querying views, or do you mean it's using a nested loop execution operator? Without seeing anything additional, you'll likely want to investigate what @Remus mentions above - i.e. is the optimizer choosing a non-covering non-clustered index to fulfill the query, leading to a nested loop between the non-clustered and the clustered index (basically to perform what used to be a bookmark lookup operation in Sql 2000). If you can post anything additional (indexes and schemas, .sqlplans, etc.), we can probably provide some additional insight.
chadhoc
By nested views, I mean that there is a view on table B which is in the join. I will correct the original question to better reflect the actual SQL.Additionally it is using nested loops to execute the join which is where the issue lies.
Rebthor
A: 

In case anyone ever runs into this, the issue appears to be only tangentially related to the partitioning scheme. Even though we run a statistics update nightly, it appears that SQL Server

  1. Didn't create a statistic on the ExtractDate
  2. Even when the extract date statistic was explicitly created, didn't pick up that the prior day had data.

We resolved it by doing a CREATE STATISTICS TABLE_A_ExtractDate_Stats ON TABLE_A WITH FULLSCAN. Now searching for prior day and a random sampling of days appears to generate the correct plan.

Rebthor