Read macros: what do you use them for?

Question

I'm trying to get a feel for the parts of Lisp that I haven't used very much up to now. Read macros have caught my attention at the moment. There isn't a huge amount of info about their use and it would help to see what people have done with them, both to get examples of how they work and also to see what sorts of problems can be approached with them. Following on that, are there any guidelines for knowing what constitutes good and bad use of read macros?

Answer 1

Although i use a lot of macros, i've never found the need to use read-macros, apart from casual experimentation. If it helps, on "Let Over Lambda" you will find an extensive discussion about them: http://letoverlambda.com/index.cl/toc

Answer 2

I actually tend to avoid them for my ordinary Lisp code; recently, I even found myself rejecting a third-party library because of its use of reader-macros. This is mostly due to the fact, that unlike symbols, there is only a single "namespace" for reader-macros. And I often seem to disagree with authors of libraries about their taste when choosing an appropriate dispatch character.

However, I used customized non-standard readtables + read successfully for simple parsing tasks. The most complex parser thing I implemented so far using a custom readtable was a HTML template engine (yet another, sorry) with a syntax similar to JSP/ASP, but using Common Lisp as the actual template language, so you can have things like

<% (for (title . link) in breadcrumb do %><a href="<%= link %>"><%= title %></a><% ) %>

(wasn't done with readtable hacks alone, though, code had to undergo a preprocessing stage).

Answer 3

Reader macros are used when there is a syntax for literal objects that you may want to have. The single problem with them is the flat namespace for possible syntaces (however, there are ways to work around that). There are not so many usages of reader macros. Some examples that come to my mind are:

• http://weitz.de/cl-interpol/ - a syntax for string interpolation and extended string notation
• http://trac.clozure.com/openmcl/wiki/OpenMclFfi - a syntax that imports symbols from C language (an automated form of FFI)
• http://clsql.b9.com/manual/sql.html - a syntax for SQL queries - not terribly useful
• http://www.agentsheets.com/lisp/XMLisp/ - embedding of literal XML fragments into Lisp code

Answer 4

S-expressions are Lisp's syntax for Lisp data. S-expressions are read with the function READ and read macros are Lisp's built-in way to extend the reader. This means that the most direct use of read macros is to implement the pre-defined data syntax and open up possibilities to change or extend the way Lisp reads s-expressions.

Lisp comes with a pre-defined external syntax for a lot of data types: symbols, numbers, strings, arrays, characters, conses, lists, structures and more. It allows data objects to be printed and read back.

1. Lisp lacks syntax for several other data types - prominently hash tables and CLOS objects. So the first use of read macros in user code would be to extend the reader to be able to read data structures like hash tables, parallel vectors, new number types, ... basically every data type the developer wants to have an external syntax that can be read back.

2. Since Lisp uses s-expressions also for code, the second use of read macros is to extend the notation for Lisp programs. A typical example is the use of [ and ] to write embedded SQL code. The usual Lisp syntax looks similar, but the use of [ and] helps the SQL expressions to stand out in the code. Another example is to use read macros to provide identifiers for embedded programming languages, like Objective C constants, messages, etc.. Clozure CL uses this to represent case sensitive / case preserving identifiers and to look up their definition during read time using an index of externally available identifiers.

3. The third use is to embed different syntaxes into Lisp syntax. An old example for that is the infix read macro, which allows embedded infix expressions. Other examples are embedded HTML or XML syntax, or embedded fragments of other programming language syntaxes.

4. Sometimes read macros are used to implement other (related) languages that use s-expression syntaxes that are different from the pre-defined Common Lisp syntax. An example would be a reader for Scheme s-expressions - which are slightly different from Common Lisp.

Answer 5

One particularly common and useful alternative syntax that you can embed with read macros is regex syntax. It's not at all hard to implement it, since it's just reading a string with different escaping rules, but if you use regexes at all frequently, it can really pay off. The CL-INTERPOL library that dmitry-vk mentioned provides this functionality, along with a lot of other features.