More explanation on Lexical Binding in Closures?

Question

There are many SO posts related to this, but I am asking this again with a different purpose

I am trying to understand why closures are important and useful. One of things that I've read in other SO posts related to this is that when you pass a variable to closure, the closure starts remembering this value from then onwards. Is this the entire Technical aspect of it or there is more to what happens there.

What I wonder then is what would happen when the variable used inside the closure gets modified from outside. Should they be constants only?

In the language Clojure, I can do the following: But since there are value is immutable, this issue does not arise. What about other languages and what is the proper technical definition of a closure?

(defn make-greeter [greeting-prefix]
    (fn [username] (str greeting-prefix ", " username)))

((make-greeter "Hello") "World")

Answer 1

You can think of a closure as an "environment", in which names are bound to values. Those names are entirely private to the closure, which is why we say that it "closes over" its environment. So your question isn't meaningful, in that the "outside" cannot affect the closed-over environment. Yes, a closure can refer to a name in a global environment (in other words, if it uses a name that is not bound in its private, closed-over environment), but that's a different story.

If you like, you can think of an environment as a dictionary, or hash table. A closure gets its own little dictionary where names are looked up.

Answer 2

A closure is really a data structure used by the compiler to make sure that a function will always have access to the data that it needs to opperate. here is an example of a function that recordes when it was defined.

(defn outer []
    (let [foo (get-time-of-day)]
      (defn inner []
          #(str "then:" foo " now:" (get-time-of-day)))))


(def then-and-now (outer))
(then-and-now)    ==> "then:1:02:03 now:2:30:01"
....
(then-and-now)    ==> "then:1:02:03 now:2:31:02"

when this function is defined a class is created and a small structure (a closure) is allocated on the heap that stores the value of foo. the class has a pointer to that (or it contains it im not sure). if you run this again then a second closure would be allocated to hold that other foo. When we say "this function closes over foo" we mean to say that it has a reference to a stricture/class/whatever that stores the state of foo at the time it was compiled. The reason you need to close over something is because the function that contains it is going away before the data will be used. In this case outer (which contains the value of foo) is going to end and be gone long before foo is used so nobody will be around to modify foo. of course foo could pas a ref to somebody who could then modify it.

Answer 3

A lexical closure is one in which the enclosed variables (e.g. greeting-prefix in your example) are enclosed by reference. The closure created does not simply get the value of greeting-prefix at the time it is created, but gets a reference. If greeting-prefix is modified after the closure is created, then its new value will be used by the closure every time it is called.

In pure functional languages this isn't much of a distinction, because values are never changed. So it doesn't matter if the value of greeting-prefix is copied into the closure: there's no possible difference in behaviour that could arise from referring to the original versus its copy.

In "imperative-languages-with-closures", such as C# and Java (via anonymous classes), some decision has to be made about whether the enclosed variable is enclosed by value or by reference. In Java this decision is pre-empted by only allowing final variables to be enclosed, effectively mimicking a functional language as far as that variable is concerned. In C# I believe it is a different matter.

Enclosing by value simplifies the implementation: the variable to be enclosed will often exist on the stack and hence will be destroyed when the function constructing the closure returns -- that means it can't be enclosed by reference. If you need enclosure by reference, a workaround is to identify such variables and keep them in an object allocated each time that function is called. This object is then kept as part of the closure's environment and must remain live as long as all closures using it are live. (I do not know if any compiled languages directly use this technique.)

Answer 4

You might enjoy reading On lambdas, capture, and mutability, which describes how this works in C# and F#, for comparison.

Answer 5

Have a look at this blog post: ADTs in Clojure. It shows a nice application of closures to the problem of locking up data so that it is accessible exclusively through a particular interface (rendering the data type opaque).

The main idea behind this type of locking is more simply illustrated with the counter example, which huaiyuan posted in Common Lisp while I was composing this answer. Actually, the Clojure version is interesting in that it shows that the issue of a closed-over variable changing its value does arise in Clojure if the variable happens to hold an instance of one of the reference types.

(defn create-counter []
  (let [counter (atom 0)
        inc-counter! #(swap! counter inc)
        get-counter (fn [] @counter)]
    [inc-counter! get-counter]))

As for the original make-greeter example, you could rewrite it thus (note the deref/@):

(defn make-greeter [greeting-prefix]
    (fn [username] (str @greeting-prefix ", " username)))

Then you can use it to render personalised greetings from the different operators of various sections of a website. :-)

((make-greeter "Hello from Gizmos Dept") "John")
((make-greeter "Hello from Gadgets Dept") "Jack").

Answer 6

For more descriptions see for example:

Common Lisp HyperSpec, 3.1.4 Closures and Lexical Binding

and

Common Lisp the Language, 2nd Edition, Chapter 3., Scope and Extent

Answer 7

This is not the sort of answer that appears to get up-votes around here, but I would heartily urge you to discover the answer to your question by reading Shriram Krishnamurthi's (free!) (online!) textbook, Programming Languages: Application and Interpretation.

I will paraphrase the book very, very briefly, by summarizing the development of the teeny tiny interpreters that it leads you through:

• an arithmetic expression language (AE)
• an arithmetic expression language with named expressions (WAE); implementing this involves developing a substitution function that can replace names with values
• a language that adds first-order functions (F1WAE): using a function involves substituting values for each of the parameter names.
• The same language, without substitution: it turns out that "environments" allow you to avoid the overhead of pre-emptive substitution.
• a language that eliminates the separation between functions and expressions by allowing functions to be defined at arbitrary locations (FWAE)

This is the key point: you implement this, and then you discover that with substitution it works fine, but with environments it's broken. In particular, in order to fix it up, you must be sure to associate with an evaluated function definition the environment that was in place when it was evaluated. This pair (fundef + environment-of-definition) is what's called a "closure".

Whew!

Okay, what happens when we add mutable bindings to the picture? If you try this yourself, you'll see that the natural implementation replaces an environment that associates names with values with an environment that associates names with bindings. This is orthogonal to the notion of closures; since closures capture environments, and since environments now map names to bindings, you get the behavior you describe, whereby mutation of a variable captured in an environment is visible and persistent.

Again, I would very much urge you to take a look at PLAI.

Answer 8

You can think of a closure as an "environment", in which names are bound to values. Those names are entirely private to the closure, which is why we say that it "closes over" its environment. So your question isn't meaningful, in that the "outside" cannot affect the closed-over environment. Yes, a closure can refer to a name in a global environment (in other words, if it uses a name that is not bound in its private, closed-over environment), but that's a different story.

I suppose that the question was if things like these are possible in languages which allow mutation of local variables:

CL-USER> (let ((x (list 1 2 3)))
           (prog1
               (let ((y x))
                 (lambda () y))
             (rplaca x 2)))
#<COMPILED-LEXICAL-CLOSURE #x9FEC77E>
CL-USER> (funcall *)
(2 2 3)

And -- since they are obviously possible -- I think the question is legitimate.