Using Clojure multimethods defined across multiple namespaces

Question

Although the below example seems a bit strange, it's because I'm trying to reduce a fairly large problem I've got at present to a minimal example. I'm struggling to work out how to call into multimethods when they're sitting behind a couple of abstraction layers and the defmulti and corresponding defmethods are defined in multiple namespaces. I really feel like I'm missing something obvious here...

Suppose I've got the following scenario:

• I purchase stuff from a variety of suppliers, via their own proprietary interfaces
• I want to implement a common interface to talk to each of those suppliers
• I want to be able to purchase different items from different suppliers

Using Clojure, the recommended ways of implementing a common interface would be via protocols or multimethods. In this case, as I'm switching based on the value of the supplier, I think the best way to handle the situation I'm describing below is via multimethods (but I could be wrong).

My multimethod definitions would look something like this, which defines a common interface I want to use to talk to every supplier's APIs:

(ns myapp.suppliers.interface)
(defmulti purchase-item :supplier)
(defmulti get-item-price :supplier)

For each supplier, I probably want something like:

(ns myapp.suppliers.supplier1
  (:require [myapp.suppliers.interface :as supplier-api]))
(defmethod purchase-item :supplier1 [item quantity] ...)
(defmethod get-item-price :supplier1 [item] ...)

and

(ns myapp.suppliers.supplier2
  (:require [myapp.suppliers.interface :as supplier-api]))
(defmethod purchase-item :supplier2 [item quantity] ...)
(defmethod get-item-price :supplier2 [item] ...)

So far, no problem

Now to my code which calls these abstracted methods, which I assume looks something like:

(ns myapp.suppliers.api
  (:require [myapp.suppliers.supplier1 :as supplier1]
            [myapp.suppliers.supplier2 :as supplier2])
(defn buy-something
  [supplier item quantity]
  (purchase-item [supplier item quantity])
(defn price-something
  [supplier item]
  (get-item-price [supplier item])

This is starting to look a bit ... ugly. Every time I implement a new supplier's API, I'll need to change myapp.suppliers.api to :require that new supplier's methods and recompile.

Now I'm working at the next level up, and I want to buy a widget from supplier2.

(ns myapp.core
  (:require [myapp.suppliers.api :as supplier])
(def buy-widget-from-supplier2    
  (buy-something :supplier2 widget 1)

This can't work, because :supplier2 hasn't been defined anywhere in this namespace.

Is there a more elegant way to write this code? In particular, in myapp.core, how can I buy-something from :supplier2?

Ader

What do you mean by ":supplier2 hasn't been defined anywhere in this namespace"? There's no need to "define" a keyword. It's just a literal (like 1 or "abc") that is accessible everywhere. – Howlan

@Howlan I knew this was a tough problem to write down... What I want to be able to do is call (buy-something :supplier2 widget 1) from myapp.core, but I think I'm missing a good way to bring in all the dependent namespaces – Ader

Answer 1

Initial notes

It's hard to tell if you mixed up some things in the process of simplifying the example, or if they weren't quite right out of the gate. For an example of what I'm referring to, consider purchase-item, though the issues are similar for get-item-price:

• The defmulti call is a single-argument function
• The defmethod calls each take two arguments
• The call in buy-something passes a vector to purchase-item, but looking up the :supplier keyword in a vector will always return nil

Your concerns

• Every time I implement a new supplier's API, I'll need to change myapp.suppliers.api to :require that new supplier's methods and recompile.

  • If you require the myapp.suppliers.interface namespace myapp.suppliers.api, the problem can be avoided

• This can't work, because :supplier2 hasn't been defined anywhere in this namespace.

  • Simply put, this will work. :)

• Is there a more elegant way to write this code? In particular, in myapp.core, how can I buy-something from :supplier2?

  • Certainly, but this solution is going to make some assumption based on the ambiguities in the Initial notes.

Without straying too far from your original design, here's a fully-working example of how I interpret what you were trying to achieve:

• myapp.suppliers.interface

  (ns myapp.suppliers.interface)
  
  (defmulti purchase-item (fn [supplier item quantity] supplier))
  

• myapp.suppliers.supplier1

  (ns myapp.suppliers.supplier1
    (:require [myapp.suppliers.interface :as supplier-api]))
  
  (defmethod supplier-api/purchase-item :supplier1 [supplier item quantity]
    (format "Purchasing %dx %s from %s" quantity (str item) (str supplier)))
  

• myapp.suppliers.supplier2

  (ns myapp.suppliers.supplier2
    (:require [myapp.suppliers.interface :as supplier-api]))
  
  (defmethod supplier-api/purchase-item :supplier2 [supplier item quantity]
    (format "Purchasing %dx %s from %s" quantity (str item) (str supplier)))
  

• myapp.suppliers.api

  (ns myapp.suppliers.api
    (:require [myapp.suppliers.interface :as interface]))
  
  (defn buy-something [supplier item quantity]
    (interface/purchase-item supplier item quantity))
  

• myapp.core

  (ns myapp.core
    (:require [myapp.suppliers.api :as supplier]))
  
  (def widget {:id 1234 :name "Monchkin"})
  
  (supplier/buy-something :supplier1 widget 15)
  ;;=> "Purchasing 15x {:id 1234, :name \"Monchkin\"} from :supplier1"
  
  (supplier/buy-something :supplier2 widget 3)
  ;;=> "Purchasing 3x {:id 1234, :name \"Monchkin\"} from :supplier2"
  

As you can see, the supplier/buy-something calls propagate to the appropriate multimethod implementations. Hopefully this helps you get where you were trying to go.

Nissen

Thanks @Hoagy - you managed to not only give me the answer I was looking for, but also correctly interpret my terrible question. The key piece I was missing was in the myapp.suppliers.supplierN namespaces; doing defmethods in the supplier-api namespace means I only need to :require the supplier-api namespace for all the upstream stuff – Ader

@Ader Don't you mean downstream instead of upstream. – Betweenwhiles

What is the myapp.supplier.api namespace for? Can't you just require the myapp.suppliers.interface directly? Multimethods seem to be first-class functions. – Betweenwhiles

@Betweenwhiles I've simplified this problem significantly from my "real world" problem. There's a valid reason that I have that extra level of abstraction in my real app, but you're right - in this specific context, I could just use myapp.suppliers.interface directly – Ader

I don't think this answer is correct. Something still needs to require the myapp.suppliers.supplier1 and myapp.suppliers.supplier2 namespaces in order for the methods defined there to be read. – Bewail

@Bewail If you don't think it's correct, run the code via copying and pasting to a REPL. Hint: It works. :) – Nissen

This will work in a REPL of course, because you'll be running each snippet and so defmethod calls will execute. In a project however, if nothing requires the supplierN namespaces then they won't get loaded (and their defmethod calls wont execute). – Bewail

I could not make it work as written (I did not try it in the REPL), because the "api" namespace needs to know about the supplier code, which it doesn't. I fixed it by adding "myapp.suppliers.suppier1" and "myapp.suppliers.supplier2" to the "api" require clause. Up a lazy river with Hoagy. ;-) – Subfamily

Also couldn't get this to work unless I create another file that requires the namespaces with the defmethods. – Subhuman

To expand on @joelittlejohn, from Clojurians: Code is only executed, when it's loaded. So when you require a specific namespace, the namespace's code gets evaluated. You require the file containing the defmulti -> You have that multimethod available, but none of its implementations. For each implementation you want, you need to require the file containing the respective defmethod in the client code. Evaluating a defmethod for some dispatch value already specified will override the previous definition, a fact you can use to change behaviour for specific cases of upstream packages. – Intransigeance

One caveat: In the REPL, reloading a file containing a defmulti will not replace the dispatching. You'll have to reload/reset the REPL to do that. – Intransigeance

Answer 2

I tried to edit the accepted answer, but it seems it still has a pending edit to review, so I am creating a new answer based on @Hoagy's answer. As per the discussion in the comments, supplier1 and supplier2 namespaces have to be referenced somewhere. I think the api namespace is a good place for it, so that the api callers do not have to mind about the different implementations.

Hoagy's edited answer below:

Initial notes

It's hard to tell if you mixed up some things in the process of simplifying the example, or if they weren't quite right out of the gate. For an example of what I'm referring to, consider purchase-item, though the issues are similar for get-item-price:

• The defmulti call is a single-argument function
• The defmethod calls each take two arguments
• The call in buy-something passes a vector to purchase-item, but looking up the :supplier keyword in a vector will always return nil

Your concerns

• Every time I implement a new supplier's API, I'll need to change myapp.suppliers.api to :require that new supplier's methods and recompile.

  • If you require the myapp.suppliers.interface namespace myapp.suppliers.api, the problem can be avoided

• This can't work, because :supplier2 hasn't been defined anywhere in this namespace.

  • Simply put, this will work. :)

• Is there a more elegant way to write this code? In particular, in myapp.core, how can I buy-something from :supplier2?

  • Certainly, but this solution is going to make some assumption based on the ambiguities in the Initial notes.

Without straying too far from your original design, here's a fully-working example of how I interpret what you were trying to achieve:

• myapp.suppliers.interface

    (ns myapp.suppliers.interface)
  
    (defmulti purchase-item (fn [supplier item quantity] supplier))
  

• myapp.suppliers.supplier1

    (ns myapp.suppliers.supplier1
      (:require [myapp.suppliers.interface :as supplier-api]))
  
    (defmethod supplier-api/purchase-item :supplier1 [supplier item quantity]
      (format "Purchasing %dx %s from %s" quantity (str item) (str supplier)))
  

• myapp.suppliers.supplier2

    (ns myapp.suppliers.supplier2
      (:require [myapp.suppliers.interface :as supplier-api]))
  
    (defmethod supplier-api/purchase-item :supplier2 [supplier item quantity]
      (format "Purchasing %dx %s from %s" quantity (str item) (str supplier)))
  

• myapp.suppliers.api

    (ns myapp.suppliers.api
      (:require [myapp.suppliers.interface :as interface]
                [myapp.suppliers.supplier1]
                [myapp.suppliers.supplier2]))
  
    (defn buy-something [supplier item quantity]
      (interface/purchase-item supplier item quantity))
  

• myapp.core

    (ns myapp.core
      (:require [myapp.suppliers.api :as supplier]))
  
    (def widget {:id 1234 :name "Monchkin"})
  
    (supplier/buy-something :supplier1 widget 15)
    ;;=> "Purchasing 15x {:id 1234, :name \"Monchkin\"} from :supplier1"
  
    (supplier/buy-something :supplier2 widget 3)
    ;;=> "Purchasing 3x {:id 1234, :name \"Monchkin\"} from :supplier2"
  

As you can see, the supplier/buy-something calls propagate to the appropriate multimethod implementations. Hopefully this helps you get where you were trying to go.

Godrich