SQL query producing duplicate rows and I can't see why

Question

My query always produces duplicate results. How best do I go about troubleshooting this query with a database > 1 million rows.

Select segstart
    ,segment
    ,callid
    ,Interval
    ,dialed_num
    ,FiscalMonthYear
    ,SegStart_Date
    ,row_date
    ,Name
    ,Xferto
    ,TransferType
    ,Agent
    ,Sup
    ,Manager
    ,'MyCenter' = Case Center
When 'Livermore Call Center' Then 'LCC'
When 'Natomas Call Center' Then 'NCC'
When 'Concord Call Center' Then 'CCC'
When 'Virtual Call Center' Then 'VCC'
When 'Morgan Hill Call Center' Then 'MHCC'
Else Center
End
    ,Xferfrom
    ,talktime
    ,ANDREWSTABLE.transferred
    ,ANDREWSTABLE.disposition
    ,dispsplit
    ,callid
    ,hsplit.starttime
    ,CASE
    WHEN hsplit.callsoffered > 0 
    THEN (CAST(hsplit.acceptable as DECIMAL)/hsplit.callsoffered)*100
    ELSE '0'
    END AS 'Service Level'
    ,hsplit.callsoffered
    ,hsplit.acceptable
FROM
(
Select segstart,
    100*DATEPART(HOUR, segstart) + 30*(DATEPART(MINUTE, segstart)/30) as Interval,
    FiscalMonthYear,
    SegStart_Date,
    dialed_num,
    callid,
    Name,
    t.Queue AS 'Xferto',
    TransferType,
    RepLName+', '+RepFName AS Agent,
    SupLName+', '+SupFName AS Sup,
    MgrLName+', '+MgrFName AS Manager,
    q.Center,
    q.Queue AS 'Xferfrom',
    e.anslogin,
    e.origlogin,
    t.Extension,
    transferred,
    disposition,
    talktime,
    dispsplit,
    segment
From CMS_ECH.dbo.CaliforniaECH e

INNER JOIN Cal_RemReporting.dbo.TransferVDNs t on e.dialed_num = t.Extension
INNER JOIN InfoQuest.dbo.IQ_Employee_Profiles_v3_AvayaId q on e.origlogin = q.AvayaID
INNER JOIN Cal_RemReporting.dbo.udFiscalMonthTable f on e.SegStart_Date = f.Tdate

Where SegStart_Date between getdate()-90 and getdate()-1
    And q.Center not in ('Collections Center',
                         'Cable Store',
                         'Business Services Center',
                         'Escalations')
    And SegStart_Date between RepToSup_StartDate and RepToSup_EndDate
    And SegStart_Date between SupToMgr_StartDate and SupToMgr_EndDate
    And SegStart_Date between Avaya_StartDate and Avaya_EndDate
    And SegStart_Date between RepQueue_StartDate and RepQueue_EndDate
    AND (e.transferred like '1'
    OR e.disposition like '4') order by segstart
) AS ANDREWSTABLE

--Left Join CMS_ECH.dbo.hsplit hsplit on hsplit.starttime = ANDREWSTABLE.Interval and hsplit.row_date = ANDREWSTABLE.SegStart_Date and ANDREWSTABLE.dispsplit = hsplit.split

Answer 1

I have into this a couple of times myself and it always ends up being one of my join statements. I would try removing your join statements one at a time and seeing if removing one of them reduced the number of duplicates.

You other option is to find a duplicate set of rows and query each table in the join on the join values and see what you get back.

Also, what database are you running and what version?

Answer 2

There are two possibities:

1. There are multiple records in your system which will appear to produce duplicate rows in your resultset because your projection doesn't select sufficent columns to distinguish them or your where clause doesn't filter them out.
2. Your joins are generating spurious duplicates because the ON clauses are not complete.

Both of these can only be solved by somebody with the requisite level of domain knowledge. So we are not going to fix that query for you. Sorry.

What you need to do is comapare some duplicate results with some non-duplicate results and discover what the first group has in common which also distinguishes it from the second group.

I'm not saying it is easy, esecially with millions of rows. But if it was easy it wouldn't be worth doing.