PostgreSQL and Sequential Data

Question

I have a dataset that contains:

Table { date itemName }

The date for the most part is sequential. There are no duplicates of the date [as it is the primary key].

The question is split up into multiple parts (all with respect to using SQL):

1. Is it possible to find gaps in the date series listed in the table? For example: Dates 1/2/09-1/3/09 are missing
2. Is it possible to find sections of dates that are missing from the table, that has a range greater than n (this is a number determined at run time)? For example: For n = 2 Dates 1/2/09-1/3/09 are not returned but Dates 5/6/09-6/1/09 are.

Answer 1

Just create a function in plsql or in a client which will be checking all dates. Like this pseudocode:

date checked_date = 2000-01-01;
int unchecked_section = 0;
while ( checked_date <= today() ) {
  if (! sql(select itemName from Table where itemName=checked_date)) {
    unchecked_section++;
  } else {
    if ( unchecked_section>=n ) {
      print checked_date-unchecked_section, checked_date
    }
    unchecked_section = 0;
  }
  checked_date++;
}
if ( unchecked_section ) {
  print checked_date-unchecked_section, checked_date
}

It does not have to be very fast as it is maintenance only. There aren't many dates to check - only 365 a year.

Answer 2

After some testing I came up with the following SQL statement:

SELECT date, itemName
  FROM "Table" as t1
  WHERE NOT EXISTS (
     SELECT date 
     FROM "Table" as t2 
     WHERE t2.date = (t1.date - INTERVAL '1 day')
  )
  ORDER BY date
  OFFSET 1  -- this will skip the first element

This will get you all rows that have no direct successor.

If you modify the statement to:

SELECT date, itemName
  FROM "Table" as t1
  WHERE NOT EXISTS (
    SELECT date 
    FROM "Table" as t2 
    WHERE (t2.date >= (t1.date - INTERVAL '2 day'))
    AND (t2.date < t1.date)
  )
  ORDER BY date
  OFFSET 1

you can use the INTERVAL length in the subselect's WHERE clause to filter by gaps of at least that size.

Hope that helps.

Answer 3

If you can use PostgreSQL 8.4 then window functions will help:

SELECT *
    FROM (SELECT itemName, date, date - lag(date) OVER w AS gap
              FROM someTable WINDOW w AS (ORDER BY date)
         ) AS pairs
    WHERE pairs.gap > '1 day'::interval;