Common Lisp condition system for transfer of control

Question

I'll admit right up front that the following is a pretty terrible description of what I want to do. Apologies in advance. Please ask questions to help me explain. :-)

I've written ETLs (Extract, Transform, Load) in other languages that consist of individual operations that look something like:

// in class CountOperation
IEnumerable<Row> Execute(IEnumerable<Row> rows) {
    var count = 0;
    foreach (var row in rows) {
        row["record number"] = count++;
        yield return row;
    }
}

Then you string a number of these operations together, and call The Dispatcher, which is responsible for calling Operations and pushing data between them.

I'm trying to do something similar in Common Lisp, and I want to use the same basic structure, i.e., each operation is defined like a normal function that inputs a list and outputs a list, but lazily.

I can define-condition a condition (have-value) to use for yield-like behavior, and I can run it in a single loop, and it works great. I'm defining the operations the same way, looping through the inputs:

(defun count-records (rows)
   (loop for count from 0
         for row in rows
         do (signal 'have-value :value `(:count ,count @,row))))

The trouble is if I want to string together several operations, and run them. My first attempt at writing a dispatcher for these looks something like:

(let ((next-op ...))  ;; pick an op from the set of all ops
  (loop
    (handler-bind
        ((have-value (...)))  ;; records output from operation
    (setq next-op ...)  ;; pick a new next-op
    (call next-op)))

But restarts have only dynamic extent: each operation will have the same restart names. The restart isn't a Lisp object I can store, to store the state of a function: it's something you call by name (symbol) inside the handler block, not a continuation you can store for later use.

Is it possible to do something like I want here? Or am I better off just making each operation function explicitly look at its input queue, and explicitly place values on the output queue?

Answer 1

I don't think that condition system is the right thing to use here. It'll be better with continuations. Also, C# compiler turns the method you presented into a continuation-like object.

In Common Lisp, you can make continuations with cl-cont library.

Answer 2

Plain Common Lisp does not support coroutines or downward continuations. You can't jump out of some computation and then jump back in. There are libraries (for example cl-cont) to provide 'some' support for continuations.

I would use 'stream'-like (see SICP) abstractions (using FORCE and DELAY) or something like SERIES (which implements an efficient lazy computation facility).