Can you get the "code as data" of a loaded function in Clojure?

Question

To put it another, way, "Okay, so code is data..."

That thread addresses how to read from a source file, but I'm wondering how to get the s-expression of an already-loaded function into a data structure that I can read and manipulate.

In other words, if I say,

(defn example [a b] (+ a b))

can't I get that list at runtime? Isn't this the whole point of "code as data"?

This is really a general Lisp question, but I'm looking for an answer in Clojure.

Sigismondo

Answer 1

You can use the clojure.repl/source macro to get the source of a symbol:

user> (source max)
(defn max
  "Returns the greatest of the nums."
  {:added "1.0"
   :inline-arities >1?
   :inline (nary-inline 'max)}
  ([x] x)
  ([x y] (. clojure.lang.Numbers (max x y)))
  ([x y & more]
   (reduce1 max (max x y) more)))
nil

But this is only part of the answer. AFAICT source looks up the source filename and line number that define the given symbol, and then prints the source code from the file. Therefore, source will not work on symbols that you do not have the source for, i.e. AOT-compiled clojure code.

Coming back to your original question, you can think of source as reading the meta data associated with the given symbol and simply printing that. I.e. it's cheating. It's not in any way returning "code as data" to you, where with code I mean a compiled clojure function.

In my mind "code as data" refers to the feature of lisps where source code is effectively a lisp data structure, and therefore it can be read by the lisp reader. That is, I can create a data structure that is valid lisp code, and eval that.

For example:

user=> (eval '(+ 1 1))
2

Here '(+ 1 1) is a literal list which gets read by the clojure reader and then evaluated as clojure code.

Update: Yehonathan Sharvit was asking in one of the comments if it's possible to modify the code for a function. The following snippet reads in the source for a function, modifies the resulting data structure, and finally evaluates the data structure resulting in a new function, my-nth, being defined:

(eval
 (let [src (read-string (str (source-fn 'clojure.core/nth) "\n"))]
   `(~(first src) my-nth ~@(nnext src))))

The syntax-quote line replaces nth with my-nth in the defn form.

Coletta

This snippet got me started down the road I'm interested in, but I believe the real answer is, No, Clojure does not internally retain the "data" representation of your functions. This method won't work, for example, on dynamically-created functions, and as you indicated, isn't even available on pre-compiled functions after the source is gone. To get what I'm after, it appears you'd need to use a special form-recording macro to define the functions, as suggested in this discussion. Thanks for the help. – Sigismondo

If I do: (def x {:add-ten #(+ % 10)}), where is the code stored for (:add-ten x)? Is it stored as byte-code? – Breakneck

Answer 2

You can get the source in recent versions of clojure with the source function.

user=> (source nth)
(defn nth
  "Returns the value at the index. get returns nil if index out of
  bounds, nth throws an exception unless not-found is supplied.  nth
  also works for strings, Java arrays, regex Matchers and Lists, and,
  in O(n) time, for sequences."
  {:inline (fn  [c i & nf] `(. clojure.lang.RT (nth ~c ~i ~@nf)))
   :inline-arities #{2 3}
   :added "1.0"}
  ([coll index] (. clojure.lang.RT (nth coll index)))
  ([coll index not-found] (. clojure.lang.RT (nth coll index not-found))))
nil

to get the string as a value you can wrap this in with-out-str:

user=> (with-out-str (source nth))
"(defn nth\n  \"Returns the value at the index. get returns nil if index out of\n  bounds, nth throws an exception unless not-found is supplied.  nth\n  also works for strings, Java arrays, regex Matchers and Lists, and,\n  in O(n) time, for sequences.\"\n  {:inline (fn  [c i & nf] `(. clojure.lang.RT (nth ~c ~i ~@nf)))\n   :inline-arities #{2 3}\n   :added \"1.0\"}\n  ([coll index] (. clojure.lang.RT (nth coll index)))\n  ([coll index not-found] (. clojure.lang.RT (nth coll index not-found))))\n"
user=> 

Termite

Thanks, this is helpful in its own right. But this just prints the source to stdout, rather than returning the value (as I see from running (source source)). Once again, maybe I just don't grok the whole idea of Lisp and functional. Why wouldn't there be a function to return this information, and return it in a structured form? – Sigismondo

editing to fix. you dont need a seperate function that returns it because you can compose this with with-out-str – Termite

Is there a way to modify the source of a function? – Packet

@YehonathanSharvit: You can use source-fn to produce the string representation of a function, and then, with some Java streaming glue, read it back into a Clojure list. After that you're free to manipulate the Clojure list to your hearts desire and finally eval it. – Coletta

@ArthurUlfeldt: you can call (source-fn 'nth) instead of (with-out-str (source nth)) to get same effect. – Coletta

+1, thanks for the ideas. The source function isn't quite what I was after, but is very useful to me getting started. – Sigismondo

Answer 3

That was my message; nice to meet you ;-) BTW, the references given in that thread for answers were excellent reading; so if you're interested, you might want to take the time to read them. Back to your question though source seems to work for code that was loaded through a file, but it doesn't work in all cases. I think, specifically, it doesn't work for functions defined in the repl.

user=> (def foo (fn [] (+ 2 2)))
#'user/foo
user=> (source foo)
Source not found
nil
user=> (defn foo2 [] (+ 2 2))
#'user/foo2
user=> (source foo2)
Source not found
nil

Digging a little bit...

user=> (source source)
(defmacro source
  "Prints the source code for the given symbol, if it can find it.
  This requires that the symbol resolve to a Var defined in a
  namespace for which the .clj is in the classpath.

  Example: (source filter)"
  [n]
  `(println (or (source-fn '~n) (str "Source not found"))))
nil
user=> (source clojure.repl/source-fn)
(defn source-fn
  "Returns a string of the source code for the given symbol, if it can
  find it.  This requires that the symbol resolve to a Var defined in
  a namespace for which the .clj is in the classpath.  Returns nil if
  it can't find the source.  For most REPL usage, 'source' is more
  convenient.

  Example: (source-fn 'filter)"
  [x]
  (when-let [v (resolve x)]
    (when-let [filepath (:file (meta v))]
      (when-let [strm (.getResourceAsStream (RT/baseLoader) filepath)]
        (with-open [rdr (LineNumberReader. (InputStreamReader. strm))]
          (dotimes [_ (dec (:line (meta v)))] (.readLine rdr))
          (let [text (StringBuilder.)
                pbr (proxy [PushbackReader] [rdr]
                      (read [] (let [i (proxy-super read)]
                                 (.append text (char i))
                                 i)))]
            (read (PushbackReader. pbr))
            (str text)))))))
nil

So yeah, it looks like it tries to load the source file off the classpath to try to spit it out for you. One thing I've learned when working with Clojure is that 9 times out of 10 it is useful to look at the source.

Contraoctave