Slow query in SQL Server 2008 using linked server. What can I look at?

Question

This query originally came from a VB6 program accessing MS Access tables which are linked to external databases through ODBC. It takes about 3:30 to run.

Now I've setup a SQL Server 2008 Express box to evaluate how we can migrate to a better database system. So I setup a linked server to the external server (we call it DWPROD) and when I converted the query (changed iif statements to case) and run it, I let it run for 12 minutes and it still doesn't finish. I'm new to SQL Server, so what can I look at to see what is taking so long? Any suggestions for speeding this up? Any resources recommended so I can learn more about this? This is a simple query compared to what we have so I'll hit this problem again.

Query:

SELECT [FISCAL_YEAR] AS FISCAL_YEAR, 
[Budgets].[dbo].[Departments].strdepentity AS ENTITY_CODE,
[Budgets].[dbo].[Departments].depdiv1 AS DIVISION_CODE, 
FINANCIAL_DEPTID AS DEPTID, 
FUND_CODE, 
[CLASS_FLD] AS CLASS_CODE, 
[PROJECT_ID], 
[PROGRAM_CODE], 
[ACCOUNT] AS ACCOUNT_CODE, 
CASE
    WHEN [ACCOUNT] in ('500020','520000','520220','520240') THEN 2
    WHEN LEFT([ACCOUNT],1)='5' THEN 1
    WHEN Left([ACCOUNT],1)='6' THEN 3
    WHEN Left([ACCOUNT],1)='7' THEN 4
    WHEN Left([ACCOUNT],1)='8' THEN 5
    ELSE 0
    END AS ACCT_GRP_CODE,
Sum([BUDGET_AMOUNT]) AS GL_BUD_AMT, 
Sum([ENCUMBRANCE_AMOUNT]) AS GL_ENC_AMT, 
Sum([EXPENDITURE_AMOUNT]) AS GL_EXP_AMT, 
CASE
    WHEN Left([CLASS_FLD],2)='12' THEN 0
    WHEN Left([CLASS_FLD],3)='113' THEN 3
    WHEN Left([CLASS_FLD],3)='112' THEN 14
    WHEN Left([CLASS_FLD],3)='115' THEN 10
    WHEN Left([CLASS_FLD],3)='116' THEN 13
    WHEN Left([CLASS_FLD],3)='117' THEN 12
    WHEN Left([CLASS_FLD],3)='118' THEN 11
    WHEN Left([CLASS_FLD],2)='13' THEN 2
    WHEN Left([CLASS_FLD],2)='14' THEN 3
    WHEN Left([CLASS_FLD],1)='4' THEN 4
    WHEN Left([CLASS_FLD],1)='6' THEN 6
    ELSE 9
    END AS FUND_SOURCE 

FROM [DWPROD]..[DISC].[PS_LEDGER_DETAIL] LEFT JOIN [Budgets].[dbo].[Departments] ON    FINANCIAL_DEPTID=[Budgets].[dbo].[Departments].deporg 
WHERE ((([BUDGET_PERIOD])='2010') And (([ACCOUNTING_PERIOD]) Between 1 And 12)) 


GROUP BY 
[FISCAL_YEAR],
[Budgets].[dbo].[Departments].strdepentity, [Budgets].[dbo].[Departments].depdiv1, 
[FINANCIAL_DEPTID],
FUND_CODE, 
[CLASS_FLD], 
[PROJECT_ID], 
[PROGRAM_CODE],
[ACCOUNT], 
CASE
    WHEN [ACCOUNT] in ('500020','520000','520220','520240') THEN 2
    WHEN LEFT([ACCOUNT],1)='5' THEN 1
    WHEN Left([ACCOUNT],1)='6' THEN 3
    WHEN Left([ACCOUNT],1)='7' THEN 4
    WHEN Left([ACCOUNT],1)='8' THEN 5
    ELSE 0
    END,
    CASE
    WHEN Left([CLASS_FLD],2)='12' THEN 0
    WHEN Left([CLASS_FLD],3)='113' THEN 3
    WHEN Left([CLASS_FLD],3)='112' THEN 14
    WHEN Left([CLASS_FLD],3)='115' THEN 10
    WHEN Left([CLASS_FLD],3)='116' THEN 13
    WHEN Left([CLASS_FLD],3)='117' THEN 12
    WHEN Left([CLASS_FLD],3)='118' THEN 11
    WHEN Left([CLASS_FLD],2)='13' THEN 2
    WHEN Left([CLASS_FLD],2)='14' THEN 3
    WHEN Left([CLASS_FLD],1)='4' THEN 4
    WHEN Left([CLASS_FLD],1)='6' THEN 6
    ELSE 9
    END
HAVING (((FISCAL_YEAR)=2010) 
AND ((FINANCIAL_DEPTID) Between '100' And '999') 
AND ((ACCOUNT) Between '500000' And '899999')) 
ORDER BY [FINANCIAL_DEPTID], [CLASS_FLD], [PROJECT_ID], [ACCOUNT]

I know it's long. Thanks for looking.

Answer 1

I re-wrote your query:

WITH ledger_detail AS (
      SELECT pld.fiscal_year,
             pld.financial_deptid AS DEPTID, 
             pld.fund_code,
             pld.class_fld AS CLASS_CODE, 
             pld.project_id,
             pld.program_code,
             CASE
               WHEN pld.account IN ('500020','520000','520220','520240') THEN 2
               WHEN LEFT(pld.account,1) = '5' THEN 1
               WHEN LEFT(pld.account,1) = '6' THEN 3
               WHEN LEFT(pld.account,1) = '7' THEN 4
               WHEN LEFT(pld.account,1) = '8' THEN 5
               ELSE 0
             END AS ACCT_GRP_CODE,
             pld.budget_amount,
             pld.encumbrance_amount,
             pld.expenditure_amount,
             pld.account AS ACCOUNT_CODE, 
             CASE
               WHEN LEFT(pld.class_fld, 2) ='12' THEN 0
               WHEN LEFT(pld.class_fld, 3)='113' THEN 3
               WHEN LEFT(pld.class_fld, 3)='112' THEN 14
               WHEN LEFT(pld.class_fld, 3)='115' THEN 10
               WHEN LEFT(pld.class_fld, 3)='116' THEN 13
               WHEN LEFT(pld.class_fld, 3)='117' THEN 12
               WHEN LEFT(pld.class_fld, 3)='118' THEN 11
               WHEN LEFT(pld.class_fld, 2)='13' THEN 2
               WHEN LEFT(pld.class_fld, 2)='14' THEN 3
               WHEN LEFT(pld.class_fld, 1)='4' THEN 4
               WHEN LEFT(pld.class_fld, 1)='6' THEN 6
               ELSE 9
             END AS FUND_SOURCE 
        FROM [DWPROD]..[DISC].[PS_LEDGER_DETAIL] pld
       WHERE pld.budget_period = '2010' 
         AND pld.accounting_period BETWEEN 1 AND 12
         AND pld.fiscal_year = 2010
         AND pld.financial_deptid BETWEEN '100' AND '999'
         AND pld.account BETWEEN '500000' AND '899999')
   SELECT x.fiscal_year,
          y.strdepentity AS ENTITY_CODE,
          y.depdiv1 AS DIVISION_CODE,
          x.deptid, 
          x.fund_code,
          x.class_code,
          x.project_id,
          x.program_code,
          x.acct_grp_code,
          SUM(x.budget_amount) AS GL_BUD_AMT, 
          SUM(x.encumbrance_amount) AS GL_ENC_AMT, 
          SUM(x.expenditure_amount) AS GL_EXP_AMT,
          x.account AS ACCOUNT_CODE, 
          x.fund_source
     FROM ledger_detail x
LEFT JOIN [Budgets].[dbo].[Departments] y ON y.deporg = x.financial_deptid    
 GROUP BY x.fiscal_year, y.strdepentity, y.depdiv1, x.deptid, x.fund_code, x.class_code, x.project_id, x.program_code, x.acct_grp_code
 ORDER BY x.financial_deptid, x.class_fld, x.project_id, x.account

Even if you have indexes on [DWPROD]..[DISC].[PS_LEDGER_DETAIL]'s account and class_fld, the use of a function (LEFT) on them renders them un-usable for this query.

Also, you're including filter criteria in the HAVING clause rather than the WHERE clause.

UPDATE: Non-CTE Equivalent

   SELECT x.fiscal_year,
          y.strdepentity AS ENTITY_CODE,
          y.depdiv1 AS DIVISION_CODE,
          x.deptid, 
          x.fund_code,
          x.class_code,
          x.project_id,
          x.program_code,
          x.acct_grp_code,
          SUM(x.budget_amount) AS GL_BUD_AMT, 
          SUM(x.encumbrance_amount) AS GL_ENC_AMT, 
          SUM(x.expenditure_amount) AS GL_EXP_AMT,
          x.account AS ACCOUNT_CODE, 
          x.fund_source
     FROM (SELECT pld.fiscal_year,
                  pld.financial_deptid AS DEPTID, 
                  pld.fund_code,
                  pld.class_fld AS CLASS_CODE, 
                  pld.project_id,
                  pld.program_code,
                  CASE
                    WHEN pld.account IN ('500020','520000','520220','520240') THEN 2
                    WHEN LEFT(pld.account,1) = '5' THEN 1
                    WHEN LEFT(pld.account,1) = '6' THEN 3
                    WHEN LEFT(pld.account,1) = '7' THEN 4
                    WHEN LEFT(pld.account,1) = '8' THEN 5
                    ELSE 0
                  END AS ACCT_GRP_CODE,
                  pld.budget_amount,
                  pld.encumbrance_amount,
                  pld.expenditure_amount,
                  pld.account AS ACCOUNT_CODE, 
                  CASE
                    WHEN LEFT(pld.class_fld, 2) ='12' THEN 0
                    WHEN LEFT(pld.class_fld, 3)='113' THEN 3
                    WHEN LEFT(pld.class_fld, 3)='112' THEN 14
                    WHEN LEFT(pld.class_fld, 3)='115' THEN 10
                    WHEN LEFT(pld.class_fld, 3)='116' THEN 13
                    WHEN LEFT(pld.class_fld, 3)='117' THEN 12
                    WHEN LEFT(pld.class_fld, 3)='118' THEN 11
                    WHEN LEFT(pld.class_fld, 2)='13' THEN 2
                    WHEN LEFT(pld.class_fld, 2)='14' THEN 3
                    WHEN LEFT(pld.class_fld, 1)='4' THEN 4
                    WHEN LEFT(pld.class_fld, 1)='6' THEN 6
                    ELSE 9
                  END AS FUND_SOURCE 
             FROM [DWPROD]..[DISC].[PS_LEDGER_DETAIL] pld
            WHERE pld.budget_period = '2010' 
              AND pld.accounting_period BETWEEN 1 AND 12
              AND pld.fiscal_year = 2010
              AND pld.financial_deptid BETWEEN '100' AND '999'
              AND pld.account BETWEEN '500000' AND '899999') x
LEFT JOIN [Budgets].[dbo].[Departments] y ON y.deporg = x.financial_deptid    
 GROUP BY x.fiscal_year, y.strdepentity, y.depdiv1, x.deptid, x.fund_code, x.class_code, x.project_id, x.program_code, x.acct_grp_code
 ORDER BY x.financial_deptid, x.class_fld, x.project_id, x.account