Problem with iterating over a time-series in clojure.

Question

Hi there,

I have the following problem: I have a time-series with more than 10000 entries and I want to perform some calculations with each of them. This alone wouldn't be a problem, but I need to get the last calculated value in order to get the next one. A very simple form of what I need would look like this:

Val(n) = Val(n-1) + (time-series-entry / 2) (or something like it!)

I don't have any idea how to manage this. Simply doing something like this:

(defn calc-val
  [time-series element]
  (seq (cons (generate-val-element time-series element)
             (calc-val time-series (inc element)))))

wouldn't work because can't (at least I don't know how!) get the last computed value. Then I thought: OK, let's use Loop-Recur. This would give me the value corresponding to the time-series entry BUT for the next one I would have to do all the computations again. Iterate would be the right thing, but it didn't work because the function has side effects.

So I'm stuck here on this one. It would be great if someone could give me a hint.

Answer 1

If you just want a hint; look into using partition.

For a little more than a hint…

(defn calc-val
  [time-series element]
  (let [p (partition 2 1 time-series)]
    (for [t p]
      (let [first-value (first t)
            second-value (second t)]
        (do whatever you need to here)))))

Though this hasn't been tested, it should work or be close to working :)

Explanation

(partition n i seq) separates seq into parts that lists of length n (2 in this case) with overlap i (1 in this case), and then we iterate over those with for, and do what we want with the parts.

Answer 2

If you only care about the final result, use reduce; if you need to get a seq of results of transforming each value in turn (where each transformation depends on the previous ones), use reductions (found in clojure.contrib.seq-utils in 1.1 and in clojure.core in 1.2).

Below, transform-first-entry does whatever you want to do to the first entry (if you don't need to transform it in any way, you can just leave out the first argument to reduce / reductions and use entries rather than (rest entries as the final argument); transform-entry is the function which takes the result of transforming the previous entry and the current entry (in this order) and produces the transformation result for the current entry.

;;; only care about the final result
(reduce transform-entry
        (transform-first-entry (first series))
        (rest entries))

;;; need to get a seq of intermediate results
(reductions ...arguments as above...)

Note that reductions is lazy.

Assuming you wanted to leave the first entry unchanged and apply your example transformation from the question text to the subsequent entries, you could use

(defn transform-entry [prev-transformed current]
  (+ prev-transformed
     (/ current 2)))

as the reduction function in

(reduce transform-entry series) ; ...or reductions