In what sense are languages like Elixir and Julia homoiconic?
Asked Answered
R

1

61

Homoiconicity in Lisp is easy to see:

(+ 1 2)

is both the function call to + with 1, 2 as arguments, as well as being a list containing +, 1, and 2. It is simultaneously both code and data.

In a language like Julia, though:

1 + 2

I know we can parse this into an Expr in Julia:

:(1 + 2)

And then we can get the AST and manipulate it:

julia> Meta.show_sexpr(:(1+2)) (:call, :+, 1, 2)

So, we can manipulate a program's AST in Julia (and Elixir). But are they homoiconic in the same sense as Lisp- is any snippet of code really just a data structure in the language itself?

I don't see how code like 1 + 2 in Julia is, immediately, data- like how (+ 1 2) in Lisp is just a list. Is it still homiconic, then?

Ronel answered 30/7, 2015 at 20:50 Comment(3)
You might be interested to know that Julia doesn't claim that it is homoiconic, anymore at least. The main reason being that it is poorly defined, so claiming to be so raised the ire of some purists. You won't find the word in documentation or website.Azal
@Azal Interesting indeed- maybe it's safe to say then that languages like Julia and Elixir support metaprogramming and macros, but not by virtue of homoiconicity.Ronel
Yeah I think that was the logic for removing it. Julia may not be homoiconic, but you can do most of things you'd associate with the idea (I think of it informally as the built-in relatively-easy-to-use capability of writing code to manipulate code).Azal
B
83

In the words of Bill Clinton, "It depends upon what the meaning of the word 'is' is". Well, ok, not really, but it does depend on what the meaning of the word "homoiconic" is. This term is sufficiently controversial that we no longer say that Julia is homoiconic – so you can decide for yourself whether it qualifies. Instead of trying to define homoiconicity, I'll quote what Kent Pitman (who knows a thing or two about Lisp) said in a Slashdot interview back in 2001:

I like Lisp's willingness to represent itself. People often explain this as its ability to represent itself, but I think that's wrong. Most languages are capable of representing themselves, but they simply don't have the will to. Lisp programs are represented by lists and programmers are aware of that. It wouldn't matter if it had been arrays. It does matter that it's program structure that is represented, and not character syntax, but beyond that the choice is pretty arbitrary. It's not important that the representation be the Right® choice. It's just important that it be a common, agreed-upon choice so that there can be a rich community of program-manipulating programs that "do trade" in this common representation.

He doesn't define homoiconicity either – he probably doesn't want to get into a definitional argument any more than I do. But he cuts to the heart of the matter: how willing is a language to represent itself? Lisp is willing in the extreme – you can't even avoid it: the representation of the program as data is just sitting right there, staring you in the face. Julia doesn't use S-expression syntax, so the representation of code as data is less obvious, but it's not hidden very deep:

julia> ex = :(2a + b + 1)
:(2a + b + 1)

julia> dump(ex)
Expr
  head: Symbol call
  args: Array(Any,(4,))
    1: Symbol +
    2: Expr
      head: Symbol call
      args: Array(Any,(3,))
        1: Symbol *
        2: Int64 2
        3: Symbol a
      typ: Any
    3: Symbol b
    4: Int64 1
  typ: Any

julia> Meta.show_sexpr(ex)
(:call, :+, (:call, :*, 2, :a), :b, 1)

julia> ex.args[3]
:b

julia> ex.args[3] = :(3b)
:(3b)

julia> ex
:(2a + 3b + 1)

Julia code is represented by the Expr type (and symbols and atoms), and while the correspondence between the surface syntax and the structure is less immediately obvious, it's still there. And more importantly, people know that code is simply data which can be generated and manipulated, so there is a "rich community of program-manipulating programs", as KMP put it.

This is not just a superficial presentation of Julia code as a data structure – this is how Julia represents its code to itself. When you enter an expression in the REPL, it is parsed into Expr objects. Those Expr objects are then passed to eval, which "lowers" them to somewhat more regular Expr objects, which are then passed to type inference, all implemented in Julia. The key point is that the compiler uses the exact the same representation of code that you see. The situation is not that different in Lisp. When you look at Lisp code, you don't actually see list objects – those only exist in the computer's memory. What you see is a textual representation of list literals, which the Lisp interpreter parses and turns into list objects which it then evals, just like Julia. Julia's syntax can be seen as a textual representation for Expr literals – the Expr just happens to be a somewhat less general data structure than a list.

I don't know the details, but I suspect that Elixir is similar – maybe José will chime in.

Update (2019)

Having thought about this more for the past 4+ years, I think the key difference between Lisp and Julia is this:

  • In Lisp, the syntax for code is the same as the syntax for the data structure that is used to represent that code.
  • In Julia, the syntax for code is quite different from the syntax for the data structure that represents that code.

Why does this matter? On the pro-Julia side, people like special syntax for things and often find S-expression syntax inconvenient or unpleasant. On the pro-Lisp side, it's much easier to figure out how to do metaprogramming correctly when the syntax of the data structure you're trying to generate (to represent code) is the same as the syntax of the code that you would normally write. This is why one of the best pieces of advice when people are trying to write macros in Julia is to do the following:

  1. Write an example of the kind of code you want your macro to generate
  2. Call Meta.@dump on that code to see it as a data structure
  3. Write code to generate that data structure—this is your macro.

In Lisp, you don't have to do step 2 because syntax for the code is already the same as the syntax for the data structure. There are the quasiquoting (in Lisp speak) quote ... end and :(...) constructs in Julia, which allow you to construct the data structures using code syntax, but that's still not as direct as having them use the same syntax in the first place.

See also:

Badger answered 30/7, 2015 at 21:51 Comment(9)
Thus it's more a PEPL, not a REPL. Since it does not read data expressions (like Lisp does), but parses program expressions into specialized data of type expr.Comstockery
I'm not sure that's a difference – S-exprs, despite their minimal syntax, do still need to be parsed, so a Lisp REPL is really a PEPL too. I think the main difference is that the Expr type isn't a general one – it's specific for representing code.Badger
The difference is that the programming language parsing in Lisp is not done in READ on text input, but slightly different in EVAL on Lisp data structures. EVAL might not convert the Lisp data into an expression data structure - evaluators can be implemented without that step.Comstockery
Thanks for the great answer.Ronel
@RainerJoswig, I'm not following you – lists are the Lisp expression data structure.Badger
Lists are just lists. There is no encoding of what an arg is, what data is, what a definition is, what a call is, what a binding is, what a function is, etc. Lisp lists know nothing about Lisp syntax. READ reads any data in s-expression format and it has zero knowledge about Lisp syntax.Comstockery
@RainerJoswig it's just an abstraction, what about this? julia> using LispREPL, typing ) gets you a lisp> prompt for Julia where you can input s-expressions! lisp> (for (i (range 1 10)) (print i)), prints: 12345678910, lisp> (show '(+ 2 3)) prints: Any[:+,2,3] and lisp> (typeof '(+ 2 3)) prints Array{Any,1}. github.com/swadey/LispREPL.jl :DRedoubt
I would say that if Julia is homoiconic, every other programming languages out there are also homoiconic, as long as they provide constructs like dump and Meta.show_sexpr.Tomtom
@WongJiaHau I guess Julia occupies a niche between languages like C where macro is all about string concatenation and regexp, and Lisp by exposing an intermediate (properly homoiconic) representation you can manipulate (which was also the first strategy of LISP if I recall correctly). As a Lisp enthusiast, I don't dig it this much, but I really do like the author's honesty and their attempt at providing something along the metaprogramming lines. Aaaannnd.... only now I notice the comment is 5 years old... :DAntinode

© 2022 - 2024 — McMap. All rights reserved.