views:

46

answers:

3

I have a table with records which include a datetime column "CreationDate".

I need to get the following information for every of the last 90 days:

  • How many records were there in total in existence
  • How many records were added on that day

I could do this through a loop of counting of course, but this would hit the database 90 times... is there a better way of doing this aggregate without having to riddle the DB with requests?

I'm using C#, LINQ, SQL Server 2008.

+1  A: 

Pull the records (or just the ids and creation dates, if that is all you need), and then perform the logic in code. One SELECT against the DB.

edit

In response to comment:

You can get the number of items for each day with a query like this:

SELECT CreationDate, COUNT(CreationDate) FROM MyTable GROUP BY CreationDate

Note that this assumes no times in CreationDate. If you have different times, the grouping won't work -- you'll have to flatten those out.

You can also add a WHERE clause to only look at the items from the last 90 days.

Jay
Selecting 400,000 records and then performing the logic in code doesn't sound right? Or is this not what you mean?
Alex
Well, that depends largely on the unknowns here. For one, have all 400,000 been created in the last 90 days? If not, then you only need the count of records created more than 90 days ago and then the records created since then. If lots of records have been created in the last 90 days, you could pull distinct dates and the number of records corresponding to each. So at day -90 you have the old record count + number of items for that day. At day -89 you have the previous result + number of items for that day, and so on.
Jay
About 80,000 have been created in the past 90 days. Would your suggestion still result in 90 queries (pulling the addition for each day)?
Alex
@Alex No, you can get it down to at most 2 queries in 1 transaction (1 query to get the total items older than 90 days -- you could roll this all in to 1 query, but you'd have one column with all the same values). See my edit.
Jay
+3  A: 

Are you looking for something like this?

WITH CTE AS
    (SELECT COUNT(*) OVER () AS TotalCount,
        CAST(CONVERT(VARCHAR, CreationDate, 101) as DATETIME) as DateValue, *
    FROM MyTable
    WHERE CreationDate >= DATEADD(DD, -90, GETDATE())
    )

SELECT DateValue, TotalCount, COUNT(*) as RowCount
FROM CTE
group by DateValue, TotalCount
order by DateValue
;
bobs
Would you mind explaining your query a little bit, and can I translate this into a LINQ to SQL call on the C# side or do I need an SP for this?
Alex
The WITH CTE... creates a Common Table Expression. This adds two columns to a SELECT FROM MyTable. The `COUNT(*) OVER ()` shows the total count for the past 90 days. It appears on each row in the results. The other column `DateValue` removes the time element from CreationDate (need to update DATECHANGED to CreationDate). Then WHERE clause returns rows that are within the last 90 days.Then `SELECT ... FROM CTE` returns the results from the CTE common table expression. The COUNT(*) and the GROUP BY produce a count for each day.This can be used as a stored procedure.
bobs
A: 

Bringing back the daily totals for the 90 days then aggregating in your application would probably be the best idea. There is currently no particularly satisfactory way of calculating running totals in SQL Server. An example of how you could do it though is below (using sys.objects as the demo table)

    IF OBJECT_ID('tempdb..#totals') IS NOT NULL
       DROP TABLE #totals

DECLARE @EndDate DATE = CURRENT_TIMESTAMP;
DECLARE @StartDate DATE = DATEADD(DAY,-89,@EndDate);

    WITH DateRange AS
    (
        SELECT
            @StartDate [DATE]
        UNION ALL
        SELECT
            DATEADD(DAY, 1, DATE) [DATE]
        FROM
            DateRange
        WHERE
            DATE < @EndDate
    )

    SELECT DATE,COUNT(t.modify_date) AS DailyTotal
    INTO #totals
    FROM DateRange LEFT JOIN sys.objects t
    ON  modify_date BETWEEN @StartDate AND @EndDate 
           AND CAST(t.modify_date AS DATE) = DateRange.Date
    GROUP BY DATE 
    ORDER BY DATE


DECLARE @BaseNumber INT = (SELECT COUNT(*) FROM sys.objects WHERE 
         modify_date < @StartDate);


    SELECT t1.Date,
           t1.DailyTotal, 
           @BaseNumber + SUM(t2.DailyTotal) AS RunningTotal
    FROM #totals t1 
     JOIN #totals t2 ON t2.date <= t1.date 
              /*Triangular join will yield 91x45 rows that are then grouped*/
    GROUP BY  t1.Date,t1.DailyTotal
    ORDER BY  t1.Date
Martin Smith