Functions as Representation
7.1 How Macros Work
Since macros can be called and return values, they tend to be associated with func-tions. Macro definitions sometimes resemble function definitions, and speaking informally, people call do, which is actually a macro, a “built-in function.” But pushing the analogy too far can be a source of confusion. Macros work differently from normal functions, and knowing how and why macros are different is the key to using them correctly. A function produces results, but a macro produces expressions—which, when evaluated, produce results.
The best way to begin is to move straight into an example. Suppose we want to write a macro nil!, which sets its argument to nil. We want (nil! x) to have the same effect as (setq x nil). We do it by defining nil! as a macro which turns instances of the first form into instances of the second.
> (defmacro nil! (var) (list ’setq var nil)) NIL!
82
7.1 HOW MACROS WORK 83
Paraphrased in English, this definition tells Lisp: “Whenever you see an expression of the form (nil! var), turn it into one of the form (setq var nil) before evaluating it.”
The expression generated by the macro will be evaluated in place of the original macro call. A macro call is a list whose first element is the name of a macro.
What happens when we type the macro call (nil! x) into the toplevel? Lisp notices that nil! is the name of a macro, and
1. builds the expression specified by the definition above, then 2. evaluates that expression in place of the original macro call.
The step of building the new expression is called macroexpansion. Lisp looks up the definition of nil!, which shows how to construct a replacement for the macro call. The definition of nil! is applied like a function to the expressions given as arguments in the macro call. It returns a list of three elements: setq, the expression given as the argument to the macro, and nil. In this case, the argument to nil! is x, and the macroexpansion is (setq x nil).
After macroexpansion comes a second step, evaluation. Lisp evaluates the macroexpansion (setq x nil) as if you had typed that in the first place. Evalu-ation does not always come immediately after expansion, as it does at the toplevel.
A macro call occurring in the definition of a function will be expanded when the function is compiled, but the expansion—or the object code which results from it—won’t be evaluated until the function is called.
Many of the difficulties you might encounter with macros can be avoided by maintaining a sharp distinction between macroexpansion and evaluation. When writing macros, know which computations are performed during macroexpansion, and which during evaluation, for the two steps generally operate on objects of two different sorts. The macroexpansion step deals with expressions, and the evaluation step deals with their values.
Sometimes macroexpansion can be more complicated than it was in the case of nil!. The expansion of nil! was a call to a built-in special form, but sometimes the expansion of a macro will be yet another macro call, like a Russian doll which contains another doll inside it. In such cases, macroexpansion simply continues until it arrives at an expression which is no longer a macro call. The process can take arbitrarily many steps, so long as it terminates eventually.
Many languages offer some form of macro, but Lisp macros are singularly powerful. When a file of Lisp is compiled, a parser reads the source code and sends its output to the compiler. Here’s the stroke of genius: the output of the parser consists of lists of Lisp objects. With macros, we can manipulate the program while it’s in this intermediate form between parser and compiler. If necessary, these manipulations can be very extensive. A macro generating its expansion has
at its disposition the full power of Lisp. Indeed, a macro is really a Lisp function—
one which happens to return expressions. The definition of nil! contains a single call to list, but another macro might invoke a whole subprogram to generate its expansion.
Being able to change what the compiler sees is almost like being able to rewrite it. We can add any construct to the language that we can define by transformation into existing constructs.
7.2 Backquote
Backquote is a special version of quote which can be used to create templates for Lisp expressions. One of the most common uses of backquote is in macro definitions.
The backquote character, ‘, is so named because it resembles a regular quote,
’, reversed. When backquote alone is affixed to an expression, it behaves just like quote:
‘(a b c) is equal to ’(a b c).
Backquote becomes useful only when it appears in combination with comma, ,, and comma-at, ,@. If backquote makes a template, comma makes a slot within a template. A backquoted list is equivalent to a call to list with the elements quoted. That is,
‘(a b c) is equal to (list ’a ’b ’c).
Within the scope of a backquote, a comma tells Lisp: “turn off the quoting.”
When a comma appears before one of the elements of the list, it has the effect of cancelling out the quote that would have been put there. So
‘(a ,b c ,d) is equal to (list ’a b ’c d).
Instead of the symbol b, its value is inserted into the resulting list. Commas work no matter how deeply they appear within a nested list,
> (setq a 1 b 2 c 3) 3
> ‘(a ,b c) (A 2 C)
> ‘(a (,b c)) (A (2 C))
7.2 BACKQUOTE 85
and they may even appear within quotes, or within quoted sublists:
> ‘(a b ,c (’,(+ a b c)) (+ a b) ’c ’((,a ,b))) (A B 3 (’6) (+ A B) ’C ’((1 2)))
One comma counteracts the effect of one backquote, so commas must match backquotes. Say that a comma is surrounded by a particular operator if the operator is prepended to the comma, or prepended to an expression which contains it. In
‘(,a ,(b ‘,c))), for example, the last comma is surrounded by one comma and two backquotes. The general rule is: a comma surrounded by n commas must be surrounded by at least n+1 backquotes. An obvious corollary is that commas may not appear outside of a backquoted expression. Backquotes and commas can be nested, so long as they obey the rule above. Any of the following expressions would generate an error if typed into the toplevel:
,x ‘(a ,,b c) ‘(a ,(b ,c) d) ‘(,,‘a)
Nested backquotes are only likely to be needed in macro-defining macros. Both topics are discussed in Chapter 16.
Backquote is usually used for making lists.1 Any list generated by backquote can also be generated by using list and regular quotes. The advantage of backquote is just that it makes expressions easier to read, because a backquoted expression resembles the expression it will produce. In the previous section we defined nil! as:
(defmacro nil! (var) (list ’setq var nil))
With backquote the same macro can be defined as:
(defmacro nil! (var)
‘(setq ,var nil))
which in this case is not all that different. The longer the macro definition, however, the more important it is to use backquote. Figure 7.1 contains two possible definitions of nif, a macro which does a three-way numeric if.2
The first argument should evaluate to a number. Then the second, third, or fourth argument is evaluated, depending on whether the first was positive, zero, or negative:
> (mapcar #’(lambda (x)
(nif x ’p ’z ’n))
’(0 2.5 -8)) (Z P N)
With backquote:
(defmacro nif (expr pos zero neg)
‘(case (truncate (signum ,expr)) (1 ,pos)
(0 ,zero) (-1 ,neg))) Without backquote:
(defmacro nif (expr pos zero neg) (list ’case
(list ’truncate (list ’signum expr)) (list 1 pos)
(list 0 zero) (list -1 neg)))
Figure 7.1: A macro defined with and without backquote.
The two definitions in Figure 7.1 define the same macro, but the first uses backquote, while the second builds its expansion by explicit calls to list. From the first definition it’s easy to see that (nif x ’p ’z ’n), for example, expands into
(case (truncate (signum x)) (1 ’p)
(0 ’z) (-1 ’n))
because the body of the macro definition looks just like the expansion it generates.
To understand the second version, without backquote, you have to trace in your head the building of the expansion.
Comma-at, ,@, is a variant of comma. It behaves like comma, with one difference: instead of merely inserting the value of the expression to which it is affixed, as comma does, comma-at splices it. Splicing can be thought of as inserting while removing the outermost level of parentheses:
> (setq b ’(1 2 3)) (1 2 3)
1Backquote can also be used to create vectors, but this is rarely done in macro definitions.
2This macro is defined a little oddly to avoid using gensyms. A better definition is given on page 150.
7.2 BACKQUOTE 87
> ‘(a ,b c) (A (1 2 3) C)
> ‘(a ,@b c) (A 1 2 3 C)
The comma causes the list (1 2 3) to be inserted in place of b, while the comma-at causes the elements of the list to be inserted there. There are some additional restrictions on the use of comma-at:
1. In order for its argument to be spliced, comma-at must occur within a sequence. It’s an error to say something like ‘,@b because there is nowhere to splice the value of b.
2. The object to be spliced must be a list, unless it occurs last. The expression
‘(a ,@1) will evaluate to (a . 1), but attempting to splice an atom into the middle of a list, as in ‘(a ,@1 b), will cause an error.
Comma-at tends to be used in macros which take an indeterminate number of arguments and pass them on to functions or macros which also take an indetermi-nate number of arguments. This situation commonly arises when implementing implicit blocks. Common Lisp has several operators for grouping code into blocks, including block, tagbody, and progn. These operators rarely appear directly in source code; they are more often implicit—that is, hidden by macros.
An implicit block occurs in any built-in macro which can have a body of expressions. Both let and cond provide implicit progn, for example. The simplest built-in macro to do so is probably when:
(when (eligible obj) (do-this)
(do-that) obj)
If (eligible obj) returns true, the remaining expressions will be evaluated, and the when expression as a whole will return the value of the last. As an example of the use of comma-at, here is one possible definition for when:
(defmacro our-when (test &body body)
‘(if ,test (progn
,@body)))
This definition uses an &body parameter (identical to &rest except for its effect on pretty-printing) to take in an arbitrary number of arguments, and a comma-at to splice them into a progn expression. In the macroexpansion of the call above, the three expressions in the body will appear within a single progn:
(if (eligible obj) (progn (do-this)
(do-that) obj))
Most macros for iteration splice their arguments in a similar way.
The effect of comma-at can be achieved without using backquote. The pression ‘(a ,@b c) is equal to (cons ’a (append b (list ’c))), for ex-ample. Comma-at exists only to make such expression-generating expressions more readable.
Macro definitions (usually) generate lists. Although macro expansions could be built with the function list, backquote list-templates make the task much easier. A macro defined with defmacro and backquote will superficially resemble a function defined with defun. So long as you are not misled by the similarity, backquote makes macro definitions both easier to write and easier to read.
Backquote is so often used in macro definitions that people sometimes think of backquote as part of defmacro. The last thing to remember about backquote is that it has a life of its own, separate from its role in macros. You can use backquote anywhere sequences need to be built:
(defun greet (name)
‘(hello ,name))