views:

126

answers:

3

I'm trying to write a function that returns a memoized recursive function in Clojure, but I'm having trouble making the recursive function see its own memoized bindings. Is this because there is no var created? Also, why can't I use memoize on the local binding created with let?

This slightly unusual Fibonacci sequence maker that starts at a particular number is an example of what I wish I could do:

(defn make-fibo [y]
  (memoize (fn fib [x] (if (< x 2)
             y
             (+ (fib (- x 1))
                (fib (- x 2)))))))

(let [f (make-fibo 1)]
  (f 35)) ;; SLOW, not actually memoized

Using with-local-vars seems like the right approach, but it doesn't work for me either. I guess I can't close over vars?

(defn make-fibo [y]
  (with-local-vars [fib (fn [x] (if (< x 2)
                                  y
                                  (+ (@fib (- x 1))
                                     (@fib (- x 2)))))]
    (memoize fib)))

(let [f (make-fibo 1)]
  (f 35)) ;; Var null/null is unbound!?! 

I could of course manually write a macro that creates a closed-over atom and manage the memoization myself, but I was hoping to do this without such hackery.

A: 

Your first version actually works, but you're not getting all the benefits of memoization because you're only running through the algorithm once.

Try this:

user>  (time (let [f (make-fibo 1)]
          (f 35)))
"Elapsed time: 1317.64842 msecs"
14930352

user>  (time (let [f (make-fibo 1)]
          [(f 35) (f 35)]))
"Elapsed time: 1345.585041 msecs"
[14930352 14930352]
Joost Diepenmaat
It doesn't work recursively, though, and that far more important than just caching the single end value.
ivar
+1  A: 
(def fib (memoize (fn [x] (if (< x 2)
                              x
                              (+ (fib (- x 1))
                                 (fib (- x 2)))))))
(time (fib 35))
PheliX
That is more typical style if you want the var bound in your namespace, but unfortunately you have incorrectly changed the function! What happened to the y parameter?!
ivar
+4  A: 

This seems to work:

(defn make-fibo [y]
  (with-local-vars
      [fib (memoize
            (fn [x]
              (if (< x 2)
                y
                (+ (fib (- x 2)) (fib (dec x))))))]
    (.bindRoot fib @fib)
    @fib))

with-local-vars only provides thread-local bindings for the newly created Vars, which are popped once execution leaves the with-local-vars form; hence the need for .bindRoot.

Michał Marczyk
Ding ding ding, thank you, we have a winner! But why did we have to jump into javaland to do the bindRoot? More importantly, doesn't this create a concurrency hazard if two threads do a .bindRoot at nearly the same time, before the vars are closed over when they exit the scope of this function? Is this still safe for concurrent creations of the generated Fibonacci functions? Or is the .bindRoot lexically scoped somehow? I'm still a little confused...
ivar
`.bindRoot` is `synchronized`, however this doesn't even matter here, since we call it on a local Var which is not accessible from any other thread at this point. As for the Javaish feel of a method call, I believe it is unavoidable here (`alter-var-root` won't work, since it requires *some* root binding to be already in place), but I don't see this as a problem. If anything, I wonder if I'd maybe prefer doing the same thing in some way not involving local Vars, but on the other hand, this does seem to be a particularly simple approach...
Michał Marczyk
Thanks, I think I get it now. The bindRoot call creates a root binding of the var, however this binding is not shared with other threads because they have their own thread-local bindings of the var, and therefore the dynamic scoping of the vars doesn't bite us in the ass. Also, the bindRoot doesn't imply that the var will be visible from the toplevel.
ivar
The root binding *is* accessible from other threads through the machinery behind `memoize` -- the latter is, however, thread-safe. (But see [this blog post](http://kotka.de/blog/2010/03/memoize_done_right.html) by Meikel Brandmeyer for an in-depth analysis of memoization in Clojure and associated gotchas.) The Var is not, however, directly visible from anywhere except the body of the `with-local-vars` form (it's a Var *local* to that body) and so cannot be got at in any way after `make-fibo` returns except through calls to the returned function.
Michał Marczyk