Difference between let* and set? in Common Lisp

Question

I am working on a genetic programming hobby project.

I have a function/macro setup that, when evaluated in a setq/setf form, will generate a list that will look something like this.

(setq trees (make-trees 2)) 
==> (+ x (abs x))

Then it will get bound out to a lambda function #<FUNCTION :LAMBDA (X) ... > via strategic use of functions/macros

However, I want to get a bit more effective with this than manually assigning to variables, so I wrote something like this:

(setq sample 
      (let* ((trees (make-trees 2))
         (tree-bindings (bind-trees trees))
         (evaluated-trees (eval-fitness tree-bindings))))
      (list (trees tree-bindings evaluated-trees)))

However, I get EVAL: trees has no value when I place this in a let form. My suspicion is that the macro expansions don't get fully performed in a LET as compared to a SETF, but that doesn't make sense to me.

What is the cause of this issue?

--- edit: yanked my code and put the whole file in a pastebin ---

Supposing that I decide that a setq isn't going to do it for me and I write a simple function to do it:

(defun generate-sample () (let ((twiggs (make-trees 2))) (let ((tree-bindings (bind-trees twiggs))) (let ((evaluated-trees (eval-fitness tree-bindings)))
(list twiggs tree-bindings evaluated-trees)))))

This yields an explosion of ...help file error messages (??!?)... and "eval: variable twiggs has no value", which stems from the bind-trees definition on SLIME inspection.

I am reasonably sure that I've completely hosed my macros. http://pastebin.org/673619

Answer 1

(Setq make-trees 2) sets the value of the variable make-trees to 2, then returns 2.

I do not see a reason for a macro in what you describe. Is it true that your make-trees creates a single random tree, which can be interpreted as a program? Just define this as a function with defun. I am thinking of something like this:

(defun make-tree (node-number)
  (if (= node-number 1)
      (make-leaf)
      (cons (get-random-operator)
            (mapcar #'make-tree
                    (random-partition (- node-number 1)))))) 

Let and setq do totally different things. Setq assigns a value to an existing variable, while let creates a new lexical scope with a number of lexical bindings.

I think that you should present more of your code; currently, your question does not make a lot of sense.

---

Update:

I will fix your snippet's indentation to make things clearer:

(setq sample 
      (let* ((trees (make-trees 2))
             (tree-bindings (bind-trees trees))
             (evaluated-trees (eval-fitness tree-bindings))))
      (list (trees tree-bindings evaluated-trees)))

Now, as written before, let* establishes lexical bindings. These are only in scope within its body:

(setq sample 
      (let* ((trees (make-trees 2))
             (tree-bindings (bind-trees trees))
             (evaluated-trees (eval-fitness tree-bindings)))
        ;; here trees, tree-bindings, and evaluated-trees are bound
        ) ; end of let* body
      ;; here trees, tree-bindings, and evaluated trees are not in scope anymore
      (list (trees tree-bindings evaluated-trees)))

That last line is spurious, too. If those names were bound, it would return a list of one element, which would be the result of evaluating the function trees with tree-bindings and evaluated-trees as arguments.

You might get what you want like this:

(setq sample 
      (let* ((trees (make-trees 2))
             (tree-bindings (bind-trees trees))
             (evaluated-trees (eval-fitness tree-bindings)))
        (list trees tree-bindings evaluated-trees)))

---

Another update:

The purpose of macros is to eliminate repeated code when that elimination is not possible with functions. One frequent application is when dealing with places, and you also need them to define new control constructs. As long as you do not see that something cannot work as a function, do not use a macro for it.

Here is some code that might help you:

(defun make-tree-lambda (depth)
  (list 'lambda '(x)
        (new-tree depth)))

(defun make-tree-function (lambda-tree)
  (eval lambda-tree))

(defun eval-fitness (lambda-form-list input-output-list)
  "Determines how well the lambda forms approach the wanted function
by comparing their output with the wanted output in the supplied test
cases.  Returns a list of mean quadratic error sums."
  (mapcar (lambda (lambda-form)
            (let* ((actual-results (mapcar (make-tree-function lambda-form)
                                           (mapcar #'first input-output-list)))
                   (differences (mapcar #'-
                                        actual-results
                                        (mapcar #'second input-output-list)))
                   (squared-differences (mapcar #'square
                                                differences)))
              (/ (reduce #'+ squared-differences)
                 (length squared-differences))))
          lambda-form-list))

(defun tree-fitness (tree-list input-output-list)
  "Creates a list of lists, each inner list is (tree fitness). Input
is a list of trees, and a list of test cases."
  (mapcar (lambda (tree fitness)
            (list tree fitness))
          tree-list
          (eval-fitness (mapcar #'make-tree-lambda tree-list)
                        input-output-list)))