How can I run eval in ClojureScript with access to the namespace that is calling the eval?

Question

I have a library of functions which I want to let users play with in the browser.

So I want to set up a situation like this :

I'm developing with figwheel and devcards.

In the main core.cljs I require various functions from my library, so they're all in scope.

Now I want to let the user enter some code which calls that library.

I see how I can run that code with eval, but I can't see how to make my library functions visible to the code being evaled.

And I'm confused by most of the documentation I'm seeing about this (eg. How can I make functions available to ClojureScript's eval?)

Is it possible? And if so, does anyone have a simple example of it being done?

cheers

Phil

Diplomatics

Answer 1

Yes, it is possible to provide access to an ambient / pre-compiled library used by evaluated code.

First, you must ensure that the functions in your library are available in the JavaScript runtime. In other words, avoid :advanced optimization, as this will eliminate functions not called at compile time (DCE). Self-hosted ClojureScript is compatible with :simple.

Second, you need to make the analysis metadata available to the self-hosted compiler that will be running in the browser (either making use of cljs.js/load-analysis-cache! or an optional argument to cljs.js/empty-state).

A minimal project illustrating how to do this is below (and also at https://github.com/mfikes/ambient):

Project Code

src/main/core.cljs:

(ns main.core
  (:require-macros [main.core :refer [analyzer-state]])
  (:require [cljs.js]
            [library.core]))

(def state (cljs.js/empty-state))

(defn evaluate [source cb]
  (cljs.js/eval-str state source nil {:eval cljs.js/js-eval :context :expr} cb))

(defn load-library-analysis-cache! []
  (cljs.js/load-analysis-cache! state 'library.core (analyzer-state 'library.core))
  nil)

src/main.core.clj:

(ns main.core
  (:require [cljs.env :as env]))

(defmacro analyzer-state [[_ ns-sym]]
  `'~(get-in @env/*compiler* [:cljs.analyzer/namespaces ns-sym]))

src/library/core.cljs:

(ns library.core)

(defn my-inc [x]
  (inc x))

Usage

We have a main.core namespace which provides an evaluate function, and this example will show how to call functions in an ambient / pre-compiled library.core namespace.

First, start up a browser REPL via

clj -m cljs.main

At the REPL, load our main namespace by evaluating

(require 'main.core)

Test that we can evaluate some code:

(main.core/evaluate "(+ 2 3)" prn)

This should print

{:ns cljs.user, :value 5}

Now, since main.core required library.core, we can call functions in that namespace. Evaluating this at the REPL will yield 11:

(library.core/my-inc 10)

Now, let's try to use this "ambient" function from self-hosted ClojureScript:

(main.core/evaluate "(library.core/my-inc 10)" prn)

You will see the following

WARNING: No such namespace: library.core, could not locate library/core.cljs, library/core.cljc, or JavaScript source providing "library.core" at line 1
WARNING: Use of undeclared Var library.core/my-inc at line 1
{:ns cljs.user, :value 11}

In short, what is going on is that even though library.core.my_inc is available in the JavaScript environment, and can indeed be called, producing the correct answer, you get warnings from the self-hosted compiler that it knows nothing about this namespace.

This is because the compiler analysis metadata is not in the main.core/state atom. (The self-hosted compiler has its own analysis state, held in that atom in the JavaScript environment, which is separate from the JVM compiler analysis state, held via Clojure in the Java environment.)

Note: If we instead had the source for library.core compiled by the self-hosted compiler (by perhaps by using main.core/evaluate to eval "(require 'library.core)", along with properly defining a cljs.js/*load-fn* that could retrieve this source, things would be good, and the compiler analysis metadata would be in main.core/state. But this example is about calling ambient / pre-compiled functions in library.core.

We can fix this by making use of cljs.js/load-analysis-cache! to load the analysis cache associated with the library.core namespace.

This example code embeds this analysis cache directly in the code by employing a macro that snatches the analysis cache from the JVM-based compiler. You can transport this analysis cache to the browser by any mechanism you desire; this just illustrates one way of simply embedding it directly in the shipping code (it's just data).

Go ahead and evaluate the following, just to see what the analysis cache for that namespace looks like:

(main.core/analyzer-state 'library.core)

If you call

(main.core/load-library-analysis-cache!)

this analysis cache will be loaded for use by the self-hosted compiler.

Now if you evaluate

(main.core/evaluate "(library.core/my-inc 10)" prn)

you won't see any warnings and this will be printed:

{:ns cljs.user, :value 11}

Furthermore, since the self-hosted compiler now has the analysis metadata for libraray.core, it can properly warn on arity errors, for example

(main.core/evaluate "(library.core/my-inc 10 12)" prn)

will cause this to be printed:

WARNING: Wrong number of args (2) passed to library.core/my-inc at line 1

The above illustrates what happens when you don't have the analyzer cache present for a namespace and how to fix it using cljs.js/load-analysis-cache!. If you know that you will always want to load the cache upon startup, you can simply things, making use of an optional argument to cljs.js/empty-state to load this cache at initialization time:

(defn init-state [state]
  (assoc-in state [:cljs.analyzer/namespaces 'library.core]
    (analyzer-state 'library.core)))

(def state (cljs.js/empty-state init-state))

Other Projects

A few (more elaborate) projects that make library functions available to self-hosted ClojureScript in the browser:

• Klangmeister
• power-turtle
• life-demo

Bilander

Added a link to a minimal project showing how to call ambient functions from self-hosted ClojureScript (illustrating how to load the analysis cache). – Bilander

Many many thanks for this Mike. I can see it will take some studying but it looks like I may be able to understand it well enough to use it. I may come back with more questions ;-) But many thanks again. – Diplomatics

@Diplomatics I've updated the example to avoid the complexities surrounding using slurp; instead it snatches the analysis cache directly from the JVM compiler's state. – Bilander

Worked nicely for me. The namespace I put my init-state in was the same as the one I had my "ambient" functions in. The "ambient" functions have to appear before the init-state is called otherwise the vars are not picked up by analyzer-state. – Sickening

@MikeFikes Would it be possible to do the same from a file not in the class path i.e not in src/ . can i load the file from say "~/" in a bootstrapped cljs on the node repl? – Hostelry

@Hostelry That seems to be an unrelated question. (Just trying to see how to answer that in the context of this particular SO.) But, to answer your question, in self-hosted ClojureScript it is up to you to define cljs.js/*load-fn* in whatever way you see fit. If doing so, you should honor the ordering described in the docstring for that var. – Bilander

Is it possible to use this approach to refer to macros from other namespaces (ex: provided by some library)? I tried adding this to the load-library-analysis-cache!, but to no avail: (cljs.js/load-analysis-cache! state 'the.other.ns (analyzer-state 'the.other.ns)) – Maltase