Macros, clojure vs common lisp

Question

A few of my friends and I are working on a new platform and we want to build it in lisp. The main attraction are macros. We all use Common Lisp but I want to explore the option of Clojure.

When I proposed this one of them said that the macro system is 'weaker'. I wanted to know if this is true, and in what areas.

Citric

I am not sure how to respond to the word "weaker" -- it depends on the your perspective. Whether one thing is weaker than another depends upon what you need or what you care about. Perhaps you could start by asking your friend for specifics on why he believes Clojure's macro system is weaker than Common Lisp's and why those things matter to him. After that, you can ask yourself whether your friend's perspective matches your own. Moreover, once you are armed with specifics, you can ask specific questions in StackOverflow, and hopefully receive specific answers. – Falconer

Answer 1

They are both pretty much equivalent in terms of what you can do with them, i.e.:

• They execute at compile time
• They can perform arbitrary transformation and generation of code (exploiting the homoiconicity of Lisp code)
• They are suitable for "extending the language" with new language constructs or DSLs
• You feel very powerful, and can be very productive (in the beating the averages way)

And now for some differences:

Common Lisp also allows reader macros which allow you to change the behaviour of the reader. This allows you to introduce completely new syntax (e.g. for data structure literals). This might be the reason that your friend describes Clojure's macro system as "weaker" since Clojure does not allow reader macros. In Clojure you are basically stuck with the syntax (macro-name ....) but apart from that you can do anything you want. Opinion is divided as to whether reader macros a good thing or not: my personal view is not, as it doesn't give you any extra "power" and has the potential to cause extreme confusion.

Clojure has, in my view, a nicer implementation of namespaces that I think make Clojure's macro system easier to use. Every symbol in Clojure is namespace-qualified, so different libraries can define the same symbol different in their own namespace. So + can be defined separately as clojure.core/+ and my.vector.library/+ without any risk of conflicts. In your own namespace, you can use definitions from another namespace, which will mean that you can choose to take + from either clojure.core or my.vector.library as needed.

Clojure on the other hand has extra literals for maps {} and vectors []. These give you a bit more expressivity (in the sense of concise readable syntax) than traditional Lisp s-expressions. In particular, the use of [] for binding forms is a convention in Clojure that I think works well both for macros and normal code - it makes them stand out clearly from the other parentheses.

Clojure is also a Lisp-1 (like Scheme) so it doesn't have a separate namespace for functions and data. Common Lisp is a Lisp-2 which has separate function and data namesapces (so you can have both a function called foo and a data item called foo). I slightly prefer the Lisp-1 approach, since it is simpler and the division between code and data seems a bit arbitrary when you are writing in a functional langauge. This is probably a personal taste thing though.

Overall, the differences are relatively minor. I think Clojure is a bit simpler and more elegant, whereas Common Lisp has some extra features (use at your own risk!). Both are extremely capable, so you can't go wrong choosing either.

Virga

vectors in CL are written as #(a b c). Others data syntaxes can be implemented via read macros. That's what they are for and CL reserves characters like {} for the user. Namespaces for symbols are called packages in CL. CL is a Lisp-n since it has more than 2 namespaces. – Entertainer

@Rainer: I think Lisp-1 / Lisp-2 typically refers to the fact that functions and data have separate namespaces, not the total number of namespaces. Source: nhplace.com/kent/Papers/Technical-Issues.html – Virga

Thank you for this elaborate response, you have convinced me that in practice the Clojure macro system is just as powerful. – Citric

clojure.core/+ and my.vector.library/+ is equivalent to cl:+ and my.vector.library:+. In CL you can import-from and shadow symbols from any package, including cl – Pufahl

Thought: if "extra literals for maps {} and vectors []" do in fact "give you a bit more expressivity than traditional Lisp s-expressions", and these literals could be created in common lisp with reader macros (which I imagine they could), then I think that means that reader macros do offer more "power" - which seems to me to be in contrast to your statement that reader macros don't "give you any extra "power"". A power to be used sparingly, perhaps, but an important one to be aware of, no? – Trabeated

@Trabeated - my point regarding power was simply that reader macros don't enable you to do anything that a regular macro couldn't. Certainly they let you make cosmetic changes to the syntax (which can be nice and make your code more readable/expressive), but they don't affect your actual ability to do things with the language (which is how I interpret "power" in this context). Regular macros on the other hand do increase your power in the sense that they enable you to do important things that aren't possible in languages without them (introduce new control flow structures, for example....) – Virga

@mikera: I think, though, that you'll find that making some code "more readable/expressive" is exactly what is meant in Beating the Averages by "powerful"... whether it's a flow control structure or a way for representing hashes, it's an abstraction that's somehow useful and makes the language more "powerful". At least that's how I'm reading Mr. Graham's words. – Trabeated

A language where you can declare a vector as [1 2 3 4] is exactly equal in power to one where you declare it as (vector 1 2 3 4) or even #{1,2,3,4}#. You may argue that one is more "expressive" than the others but that is subjective. OTOH, if you can't populate a vector without an explicit loop then your language is lacking power in an objective sense - it simply doesn't offer a construct for declaring a vector literal. These two things are fundamentally different. I remain convinced that Paul Graham talks about power in the latter sense, and not about cosmetic syntactic choices. – Virga

@Trabeated - perhaps the word "expressivity" is the problem here as it is slightly overloaded/ambiguous. Will reword the answer to make it more clear in what sense this is meant. – Virga

@Virga thank you for clearing up the distinction you're making. I now much better understand your point... I'll have to think about it more to know whether I agree or not, but at least I better understand what you're saying, which is good. :) – Trabeated

My 2 cents - a quote from Paul Graham's "On Lisp", page 225: "[...] read-macros are at least as powerful as ordinary macros. Indeed, read-macros are more powerful [emphasis mine] in at least two ways: A read-macro affects everything read by Lisp, while a macro will only be expanded in code. And since read-macros generally call read recursively, an expression like ''a becomes (quote (quote a)), whereas if we had tried to define an abbreviation for quote using a normal macro, it would work in isolation, but not when nested." – Jaggy

It is the [] and {} which makes me not like Clojure. If I am to accept Lisp's disadvantage of having lots of parenthesis, I might as well insist on the symmetry to make it an advantage. – Oxazine

Answer 2

Regarding ordinary macros here are the differences between Lisp's and Clojure's variants:

1. Clojure is Lisp-1, while CL is Lisp-2. The history has shown, that macros in one-namespace language are generally more error-prone, because there's a bigger name collision chance. To mitigate this problem Clojure uses a non-traditional quasi-quote implementation.
2. But it comes at a cost: as in Clojure symbols inside backquote are eagerly resolved in the namespace, where macros are defined, there are a lot of subtle issues, like this one.
3. Clojure macros use auto-gensyms. This is advertised as an advantage, and it indeed removes some boilerplate (surely, you can achieve the same in Lisp - see defmacro!). And the conceptual question of when to use gensyms, obviously, remains.

So, overall, Lisp's approach is more rough, but more flexible. Clojure's macros may be slightly easier to approach, but become harder to use, when you go beyond simple syntax modifications and start to define complete DSLs.

Regarding reader macros, as you know, Clojure doesn't give this option to the user, while CL does. So in CL reader macros find a lot of uses, like string interpolation, internationalization support, SQL DSLs or java interop. Not to mention a plethora of special cases, when reader macros can be used to help create advanced DSLs.

Pufahl

Answer 3

I was a lisp programmer on lisp machines in the 80's. In terms of productivity, I still have to see a comparable development environment. That said, clojure macro system is powerful enough to have been already abused. Macros help when you have to implement a "compiler" of a new language and you should be able to describe it in a formal notation. Not because is absolutely need by you. It's needed by the users of the new language itself. So, it should be a very useful formal language to be deserved a compiler and a formal notation. If this is not the case, stay away from macros and use pure functional programming style to make your abstractions. You'll feel again the joy of coding sequentially with your analog though, instead of being closer to dumb sequences of bits. And the users of your code will be happier too. I hope clojure macro abuse is not going to be the next accidental complexity. An old lisper and clojure's newbie.

Phenothiazine

Answer 4

May be your friend is referring Clojure macro system is "weaker" in the sense that Clojure as of now doesn't support "reader macros", but that doesn't mean that the macro system is weaker as reader macros can make things really really complex as far as my understanding of using them is concerned.

I also wont suggest that you should take macros as the main attraction. They are powerful but when you are attracted to some specific technique you start applying it everywhere even though there is a much simpler technique available (in macro case the simpler technique is functions).

Another important thing you should consider is the "platform" that you want to target as Clojure supports JVM and along with all the tools that are there for JVM platfom.

Annoying

I'm not sure the JVM can be an argument because of ABCL. – Fascicle

I think Clojure JVM interop is much more better – Annoying

Thank you for the warning, I agree (if this is what you're saying) that functions generally are the way to go when they suffice – Citric

Answer 5

Clojure Macros have some advantages:

• accidental symbol capture is hard because it namespace expands symbols
• other peoples reader macros wont bite you (not your's or course, im talking about that other guy ;))
• autogensyms takes a LOT of the clutter out of writing hygenic macros.

Incoherence

Auto gensyms sounds helpful with readability, though you could always make it yourself. If you want to use variable capture, could you still do it? – Citric

@Frawr Yes if you want variable capture you can still use it but there is one gotcha: you have to write ~'it to capture the (unqualified) symbol it. So Paul Graham's aif is (defmacro aif [test true-fm & [false-fm]] `(let [~'it] ~true-fm ~false-fm)) – Novick