MySQL - Group on similar timestamps

Question

Query:

  SELECT project_id, 
         COUNT(*) AS count,
         MIN(date_added) AS date_start, 
         MAX(date_added) AS date_end
    FROM my_table 
GROUP BY project_id, TIMESTAMPDIFF(MINUTE, date_added) < 5
WHERE user_id = 1 LIMIT 10

How can I accomplish this? I want to group the items so that no two consecutive items in a group are more than 5 minutes apart, but the start and end time can be any distance apart. Is there any way to do this in the database, or will I need to grab all the data and figure it out in the program?

Answer 1

Since you want to chain together the longest possible chain of records where each pair is less than 5 minutes apart, I don't think you can do this with GROUP BY.

Try writing a stored procedure using a WHILE loop. Start with a cursor that selects the records ORDER BY date_added. You can create a TEMPORARY TABLE and INSERT a row into it at the end of each group. By doing it as a stored procedure you can avoid pulling all of the records back into your program (often over the network) which can make it much faster.

Answer 2

Off the top of my head, I didn't try:

  SELECT project_id, 
         COUNT(*) AS count,
         MIN(date_added) AS date_start, 
         MAX(date_added) AS date_end
    FROM my_table 
   WHERE user_id = 1 
GROUP BY project_id, ROUND(date_added / (5 * 60))
   LIMIT 10

Assuming "dated_added" in seconds of course.

In other words, items are grouped according to which 5-minute slice they belong.

Answer 3

Okay, here goes:

SELECT id, project_id, start_time, MAX(end_time) AS end_time FROM (
  SELECT
    @new_group :=
      ((TIME_TO_SEC(date_added) - @prev_second) > (5 * 60)) ||
      (project_id <> @prev_project_id) AS new_group,
    @date_added_group := @date_added_group + @new_group AS date_added_group,
    @start_time := IF(@new_group, date_added, @start_time) AS start_time,
    id,
    project_id,
    date_added AS end_time,
    @prev_second := TIME_TO_SEC(date_added) AS prev_sec,
    @prev_project_id := project_id AS prev_project
  FROM my_table,
  (SELECT
    @new_group :=0,
    @date_added_group := 0,
    @start_time := 0,
    @prev_second := 0,
    @prev_project_id := 0) AS vars
  ORDER BY project_id, date_added
) AS my_table GROUP BY project_id, date_added_group;

Given this data:

+----+------------+---------------------+
| id | project_id | date_added          |
+----+------------+---------------------+
|  1 |          1 | 2010-07-15 19:00:00 | < new project
|  2 |          1 | 2010-07-15 19:01:00 |
|  3 |          1 | 2010-07-15 19:02:00 |
|  4 |          2 | 2010-07-15 19:03:00 | < new project
|  5 |          2 | 2010-07-15 19:04:00 |
|  6 |          2 | 2010-07-15 19:25:00 | < new interval
|  7 |          2 | 2010-07-15 19:26:00 |
|  8 |          2 | 2010-07-15 19:27:00 |
|  9 |          2 | 2010-07-15 19:48:00 | < new interval
| 10 |          2 | 2010-07-15 19:49:00 |
| 11 |          3 | 2010-07-15 19:50:00 |
| 12 |          3 | 2010-07-15 20:11:00 | < new interval
| 13 |          4 | 2010-07-15 20:12:00 | < new project
| 14 |          4 | 2010-07-15 20:13:00 |
| 15 |          4 | 2010-07-15 20:14:00 |
| 16 |          5 | 2010-07-15 20:15:00 | < new project
| 17 |          5 | 2010-07-15 20:16:00 |
| 18 |          5 | 2010-07-15 21:27:00 | < new interval
| 19 |          5 | 2010-07-15 21:28:00 |
| 20 |          6 | 2010-07-15 21:29:00 | < new project
| 21 |          6 | 2010-07-15 21:30:00 |
| 22 |          6 | 2010-07-15 21:31:00 |
+----+------------+---------------------+

The query returns this result set:

+----+------------+---------------------+---------------------+
| id | project_id | start_time          | end_time            |
+----+------------+---------------------+---------------------+
|  1 |          1 | 2010-07-15 19:00:00 | 2010-07-15 19:02:00 | 
|  4 |          2 | 2010-07-15 19:03:00 | 2010-07-15 19:04:00 |
|  6 |          2 | 2010-07-15 19:25:00 | 2010-07-15 19:27:00 |
|  9 |          2 | 2010-07-15 19:48:00 | 2010-07-15 19:49:00 |
| 11 |          3 | 2010-07-15 19:50:00 | 2010-07-15 19:50:00 |
| 12 |          3 | 2010-07-15 20:11:00 | 2010-07-15 20:11:00 |
| 13 |          4 | 2010-07-15 20:12:00 | 2010-07-15 20:14:00 |
| 16 |          5 | 2010-07-15 20:15:00 | 2010-07-15 20:16:00 |
| 18 |          5 | 2010-07-15 21:27:00 | 2010-07-15 21:28:00 |
| 20 |          6 | 2010-07-15 21:29:00 | 2010-07-15 21:31:00 |
+----+------------+---------------------+---------------------+