Ruby Array group and average by hour

Question

We get our data from a sensor which records and stores data like hashes. At any time it measures a few stuff like that:

{:temperature => 30, :pression => 100, :recorded_at => 14:34:23}
{:temperature => 30, :pression => 101, :recorded_at => 14:34:53}
{:temperature => 31, :pression => 102, :recorded_at => 14:34:24}
{:temperature => 30, :pression => 101, :recorded_at => 14:34:55}
{:temperature => 30, :pression => 102, :recorded_at => 14:34:25}
{:temperature => 31, :pression => 101, :recorded_at => 14:34:56}

We need to be able to export that data on a JSON format, but we have way too much data (the sensor records about every 30 seconds) and we need to remove some of the data. Ideally we'd want to export 1 measure per hour in the last 24 hours so we have something like

{0 => {:temperature => 30, :pression => 100}, 1 => {:temperature => 30, :pression => 100}, 2 => {:temperature => 30, :pression => 100}, 3 => {:temperature => 30, :pression => 100}, 4 => {:temperature => 30, :pression => 100}}

For each hour, the temperature is the average of all temperatures measured within that hour. Also, if for any reason some data is missing for 1hour, I'd like to to extrapolate it by being the mean between the previous and next hour. Anybody can help?

Answer 1

something like this

t = [....] - array of measurings
result = {}

(1..24).each do|hour| 
    #  measurings of given hour
    measurings = t.select{|measuring| measuring[:recorded_at][0, 2].to_i == hour}

    #  average temperature of hour
    sum = measurings.inject(0){|sum, measuring| sum + measuring[:temperature].to_i} 
    average_temperature = (measurings.length == 0)? nil: sum/measurings.length.to_f

    result[hour] = average_temperature
end

Answer 2

More functional version (with simple interpolation of missing values)

probs = [{:temperature => .. }] # array of measurings

def average(list, key)
  list.reduce(0){|acc,el| acc+el[key]} / list.length unless list.empty
end

prob_groups = probs.group_by{|prob| prob[:recorded_at][0,2].to_i}
average_groups = prob_groups.map do |hour,prob_group|
  { hour => {
      :temperature => average(prob_group, :temperature),
      :pression    => average(prob_group, :pression)
  }}
end.reduce{|acc,el| acc.merge(el)}

def interpolate(p, n, key)
  (p[key] + n[key])/2 unless p.nil? || n.nil? || p[key].nil? || n[key].nil?
end

resuls = (1..24).map do |hour|
  if average_groups[hour]
    { hour => average_groups[hour] }
  else
    { hour => {
      :temperature => interpolate(average_groups[hour-1], average_groups[hour+1], :temperature),
      :pression => interpolate(average_groups[hour-1], average_groups[hour+1], :pression)
    }}
  end
end.reduce{|acc,el| acc.merge(el)}

Hope it works

Answer 3

If you are not interested on the history but only on an approximation of actual value(s), consider to use a "moving metric" (http://en.wikipedia.org/wiki/Moving%5Faverage).