Aging a dataset

Question

For reasons I'd rather not go into, I need to filter a set of values to reduce jitter. To that end, I need to be able to average a list of numbers, with the most recent having the greatest effect, and the least recent having the smallest effect. I'm using a sample size of 10, but that could easily change at some point.

Are there any reasonably simple aging algorithms that I can apply here?

Answer 1

• Have a look at the exponential smoothing. Fairly simple, and might be sufficient for your needs. Basically recent observations are given relatively more weight than the older ones.
• Also (depending on the application) you may want to look at various reinforcement learning techniques, for example Q-Learning or TD-Learning or generally speaking any method involving the discount.

Answer 2

I ran into something similar in an embedded control application.

The simplest option that I came across was a 3/4 filter. This gets applied continuously over the entire data set:

current_value = (3*current_value + new_value)/4

I eventually decided to go with a 16-tap FIR filter instead:

Overview
FIR FAQ
Wikipedia article

Answer 3

Many weighted averaging algorithms could be used.

For example, for items I(n) for n = 1 to N in sequence (newest to oldest):

(SUM(I(n) * (N + 1 - n)) / SUM(n)

Answer 4

It's not exactly clear from the question whether you're dealing with fixed-length data or if data is continuously coming in. A nice physical model for the latter would be a low pass filter, using a capacitor and a resistor (R and C). Assuming your data is equidistantly spaced in time (is it?), this leads to an update prescription

U_aged[n+1] = U_aged[n] + deltat/Tau (U_raw[n+1] - U_aged[n])

where Tau is the time constant of the filter. In the limit of zero deltat, this gives an exponential decay (old values will be reduced to 1/e of their value after time Tau). In an implementation, you only need to keep a running weighted sum U_aged.

deltat would be 1 and Tau would specify the 'aging constant', the number of steps it takes to reduce a sample's contribution to 1/e.