Can a language have Lisp's powerful macros without the parentheses?

Question

Can a language have Lisp's powerful macros without the parentheses?

Answer 1

Parentheses are irrelevant to macros. It's just Lisp's way of doing things.

For example, Prolog has a very powerful macros mechanism called "term expansion". Basically, whenever Prolog reads a term T, if tries a special rule term_expansion(T, R). If it is successful, the content of R is interpreted instead of T.

Answer 2

Erlang's parse transforms are similar in power to Lisp macros, though they are much trickier to write and use (they are applied to the entire source file, rather than being invoked on demand).

Lisp itself had a brief dalliance with non-parenthesised syntax in the form of M-expressions. It didn't take with the community, though variants of the idea found their way into modern Lisps, so you get Lisp's powerful macros without the parentheses ... in Lisp!

Answer 3

Not to mention the Dylan language, which has a pretty powerful syntactic macro system, which features (among other things) referential transparency, while being an infix (Algol-style) language.

Answer 4

Yes. Parentheses in Lisp are used in the classic way, as a grouping mechanism. Indentation is an alternative way to express groups. E.g. the following structures are equivalent:

A ((B C) D)

and

A
    B
    C
  D

Answer 5

Code rewriting in Tcl in a manner recognizably similar to Lisp macros is a common technique. For example, this is (trivial) code that makes it easier to write procedures that always import a certain set of global variables:

proc gproc {name arguments body} {
    set realbody "global foo bar boo;$body"
    uplevel 1 [list proc $name $arguments $realbody]
}

With that, all procedures declared with gproc xyz rather than proc xyz will have access to the foo, bar and boo globals. The whole key is that uplevel takes a command and evaluates it in the caller's context, and list is (among other things) an ideal constructor for substitution-safe code fragments.

Answer 6

No, it's not necessary. Anything that gives you some sort of access to a parse tree would be enough to allow you to manipulate the macro body in hte same way as is done in Common Lisp. However, as the manipulation of the AST in lisp is identical to the manipulation of lists (something that is bordering on easy in the lisp family), it's possibly not nearly as natural without having the "parse tree" and "written form" be essentially the same.

Answer 7

sure, the question is whether it is convenient to use.

Let's first look how Lisp is slightly different.

Lisp has a two-stage syntax.

A) there is the data syntax for s-expressions

examples:

(mary called tim to tell him the price of the book)

(sin ( x ) + cos ( x ))

s-expressions are lists of atoms or lists (lists of lists).

B) then there is the Lisp language syntax on top of s-expressions. Not every s-expression is a valid Lisp program.

(3 + 4)

is not a valid Lisp program.

(+ 3 4)

is a valid Lisp program.

The interesting part is now that s-expressions can be read and then Lisp uses the normal data structures to represent them.

Most other programming languages don't have a primitive representation for internalized source - other than strings.

The internalized source code now makes it easy to calculate code.

Let's look at the invalid Lisp code:

(3 + 4)

The program

(defun convert (code)
   (list (second code) (first code) (third code)))

(convert '(3 + 4)) -> (+ 3 4)

has converted an infix expression into the valid prefix expression.

(eval (convert '(3 + 4))) -> 7

EVAL evaluates the converted source code.

Programming languages now have at least three choices to make such calculations possible:

1. base the source code transformations on string transformations

2. use a similar primitive data structure like Lisp. A more complex variant of this is a syntax based on XML. One could then transform XML expressions. There are other possible external formats combined with internalized data.

3. use a real syntax description format and represent the source code internalized as a syntax tree using data structures that represent syntactic categories.

For all these approaches you will find programming languages. Lisp is more or less in camp 2. The consequence: it is theoretically not really satisfying and makes it impossible to statically parse source code (if the code transformations are based on arbitrary Lisp functions). The Lisp community struggles with this for decades (see for example the myriad of approaches that the Scheme community has tried). Fortunately it is relatively easy to use, compared to some of the alternatives and quite powerful. Variant 1 is plain ugly. Variant 3 leads to a lot complexity in simple AND complex transformations.

Another problem is HOW to transform the code. One approach would be based on transformation rules (like in some Scheme macro variants). Another approach would be a special transformation language (like a template language which can do arbitrary computations). The Lisp approach is to use Lisp itself. That makes it possible to write arbitrary transformations using the full Lisp language. In Lisp there is not a separate parsing stage, but at any time expressions can be read, transformed and evaluated.

Lisp is kind of a local maximum of simplicity for code transformations.

Also note that READ reads s-expressions to internal data. In Lisp one could either use a different READer for a different external syntax or reuse the Lisp built-in reader and reprogram it using the read macro mechanism. There are examples for both approaches to provide a different external syntax in Lisp.

The current Lisp syntax is popular among Lisp programmers for two reasons:

1) the data is code is data idea makes it easy to write all kinds of code transformations based on the internalized data. There is also a relatively direct way from reading code, over manipulating code to printing code.

2) the text editor can be programmed in a straight forward way to manipulate s-expressions. That makes code and data transformations in the editor relatively easy.

Answer 8

Yes, it is certainly possible. Especially if it is still a Lisp under the bonnet:

http://www.meta-alternative.net/pfront.pdf

http://www.meta-alternative.net/pfdoc.pdf

Answer 9

Absolutely. It's just a couple orders of magnitude more complex, if you have to deal with a complex grammar. As Peter Norvig noted:

Python does have access to the abstract syntax tree of programs, but this is not for the faint of heart. On the plus side, the modules are easy to understand, and with five minutes and five lines of code I was able to get this:

>>> parse("2 + 2")

['eval_input', ['testlist', ['test', ['and_test', ['not_test', ['comparison', ['expr', ['xor_expr', ['and_expr', ['shift_expr', ['arith_expr', ['term', ['factor', ['power', ['atom', [2, '2']]]]], [14, '+'], ['term', ['factor', ['power', ['atom', [2, '2']]]]]]]]]]]]]]], [4, ''], [0, '']]

This was rather a disapointment to me. The Lisp parse of the equivalent expression is (+ 2 2). It seems that only a real expert would want to manipulate Python parse trees, whereas Lisp parse trees are simple for anyone to use. It is still possible to create something similar to macros in Python by concatenating strings, but it is not integrated with the rest of the language, and so in practice is not done.

Since I'm not a super-genius (or even a Peter Norvig), I'll stick with (+ 2 2).

Answer 10

Boo has a nice "quoted" macro syntax that uses [| |] as delimiters, and has certain substitutions which are actually verified syntactically by the compiler pipeline using $variables. While simple and relatively painless to use, it's much more complicated to implement on the compiler side than s-expressions. Boo's solution may have a few limitations that haven't affected my own code. There's also an alternate syntax that reads more like ordinary OO code, but that falls into the "not for the faint of heart" category like dealing with Ruby or Python parse trees.

Answer 11

Here's a shorter version of Rainer's answer:

In order to have lisp-style macros, you need a way of representing source-code in data structures. In most languages, the only "source code data structure" is a string, which doesn't have nearly enough structure to allow you to do real macros on. Some languages offer a real data structure, but it's too complex, like Python, so that writing real macros is stupidly complicated and not really worth it.

Lisp's lists and parentheses hit the sweet spot in the middle. Just enough structure to make it easy to handle, but not too much so you drown in complexity. As a bonus, when you nest lists you get a tree, which happens to be precisely the structure that programming languages naturally adopt (nearly all programming languages are first parsed into an "abstract syntax tree", or AST, before being actually interpreted/compiled).

Basically, programming Lisp is writing an AST directly, rather than writing some other language that then gets turned into an AST by the computer. You could possibly forgo the parens, but you'd just need some other way to group things into a list/tree. You probably wouldn't gain much from doing so.