Package versus namespace versus module

Question

According to http://www.phyast.pitt.edu/~micheles/scheme/scheme29.html

"It is worth mentioning that if you use a package system (like in Common Lisp) or a namespace system (like in Clojure) in practice variable capture becomes pretty rare. In Scheme instead, which uses a module system, hygiene is essential..."

What is the difference between a package system, a namespace system and a module system as the terms are used here?

Note that this is not about Lisp-1 versus Lisp-2 (which the linked document discusses separately). I'm guessing it might have something to do with the way, in Common Lisp, trying to use the same symbol in two different packages can get you two different symbols with the same name.

Klingensmith

23 things I know about modules for Scheme, C. Queinnec – Metamorphic

One man's module system is another man's package system :-) Therefore the quotation is explicit "package system like Common Lisp". Queinnec's note is an excellent exposition in the design trade-offs. – Deodand

Answer 1

I think a common distinction between a package system and a module system is that a package system deals with mapping source text to names, while a module system deals with mapping names to meanings. (I don't know what a namespace system is, but I suspect it's a package system, perhaps without first-class packages, or even first-class names?)

In the context of Lisp, names are symbols, so a package system controls what symbol you get (specifically, what package it's in) when you read something which has the syntax of a symbol, while in a module system you always get the same symbol but its value depends on the module state.

Here's an example of how these things differ, using the CL package system and Racket's module system. Note that I'm a CL person: I understand the CL package system a lot better than I understand Racket's module system or Racket / Scheme macros.

Let's say I want to define some kind of symbolic algebra system and I'd like to be able to use syntax like (+ a b) when a and b may be things that cl:+ coes not understand, such as polynomials or something.

Well, I can do that in CL with a package definition something like this:

(defpackage :org.tfeb.symalg
  (:use)
  (:export "+" "-" "*" "/"))

(let ((p (find-package :org.tfeb.symalg)))
  (do-external-symbols (s (find-package :cl))
    (ecase (nth-value 1 (find-symbol (symbol-name s) p))
      ((nil)
       (import s p)
       (export s p))
      ((:external)
       nil)
      ((:inherited :internal)
       (error "package botch")))))

(defpackage :org.tfeb.symalg-user
  (:use :org.tfeb.symalg))

(Note that, in real life, you'd obviously write a macro do do the above hair in a declarative & more flexible way: indeed some guy on the internet wrote a system called 'conduits' to do just this, and one day he'll probably get around to publishing it again.)

What this does is to create a package, org.tfeb.symalg which is like cl except that some specific symbols are different, and a package org.tfeb.symalg-user which uses this package instead of cl. In that package, (+ 1 2) means (org.tfeb.symalg:+ 1 2), but (car '(1 . 2)) means (cl:car '(1 . 2)).

But

(defmethod foo (a)
  (:method-combination +))

also means

(defmethod foo (a)
  (:method-combination org.tfeb.symalg:+))

and now I have some trouble: anywhere I wanted to use the symbol + as a symbol I have to type cl:+. (This specific example is easy to get around: I just need to define a method combination for org.tfeb.symalg:+, and I probably want to do that in any case, but there are other cases.)

This makes it a pain to do this sort of 'redefine bits of the language' thing in cases where I want to use names (symbols) which are part of the language as symbols.

Compare Racket: here is a little module definition in Racket which provides (or in fact doesn't) variant versions of some arithmetic symbols):

#lang racket

(provide
 (rename-out
  (plus +)
  (minus -)
  (times *)
  (divide /)))

(define (plus . args)
  (apply + args))

...

(define plus-symbol '+)

What this does is to say that, if you use this module, then the value of the symbol + is the value of the symbol plus in the module and so on. But the symbol is the same symbol. And you can easily check this if you are using the module by, for instance (eq? '+ plus-symbol), which will return #t: nothing funny is going on with what symbol you get when you type +, it's all in the mapping from those symbols to their values.

So this is much nicer: if Racket had a CLOS-style object system (and it probably has about six, some of which might half work), then the + method combination would just work, and in general anything which cares about symbols will work the way you want it to.

Except that one of the most common things you do in CL which manipulates symbols as symbols is macros. And if you try to take the CL approach to macros in Racket there is just hair everywhere.

In CL the approach is typically something like this:

1. The packages get defined somehow, and these definitions are the same throughout the compilation, loading and evaluation of the system.
2. Macros get defined.
3. The program is compiled with the macros being expanded.
4. The program is loaded and run.

And this all works fine: if my macro is defined in a situation where + means org.tfeb.symalg:+ then, if its expansion involves + it really involves org.tfeb.symalg:+, and so long at that package exists at compile time (which it will since compile time and macro expansion time are intertwined) and at run time and the definition is there then all will be well.

But it's not the same thing in Racket: if I write a macro then I know that the symbol + is just the symbol +. But what + means could be completely different at different times depending on the module state. And that could mean, for instance, that at compile time + meant Racket's native addition function, and so the compiler can optimise that into the ground, but perhaps at run time it means something else, because the module state is different now. The symbol + does not contain enough information to know what it should mean.

If you also take into account the information that Scheme people really care about sorting this sort of thing out in a clean way, and are definitely not happy with the CL approach to macros of 'stop thinking so hard & just use gensyms, it will all be fine', none of which is helped by Schemes being Lisp-1s, you can see that there are some fairly significant problems here which need to be addressed.

Wilks

Answer 2

Common Lisp has a package system for symbols.

Notation:

• cl-user::create ; symbol CREATE in package CL-USER
• cl-user:create ; exported symbol CREATE in package CL-USER
• create ; symbol CREATE in the - at read-time - current package
• :create ; symbol CREATE in the package KEYWORD
• #:create ; symbol CREATE, not in any package

Example in the Package CL-USER

Let's tell the reader that CL-USER is the current package:

(in-package "CL-USER")

A function CL-USER::CREATE

(defun create (x)        ;  CL-USER::CREATE
  (list :list x))

A macro which creates code using above function

(defmacro m (x)
  `(create ,x))    ; here we use CL-USER::CREATE

If we stay in this package, we can define a local function, which will shadow the above global function.

(defun foo ()
  (flet ((create (x)             ; CL-USER::CREATE
           (vector :vector x)))
    (m (create 1))))             ; CL-USER::CREATE

So (m (create 1)) is (cl-user::create (cl-user::create 1)). Both use the local function.

Example in the Package BAR

Now we can move our FOO function to another package BAR, which we will define here:

(defpackage "BAR"
  (:use "COMMON-LISP")
  (:export "FOO")
  (:import-from "CL-USER" "M"))

Let's tell the reader to use this package BAR as the current package:

(in-package "BAR")

Now if we define our function FOO similar to above

(defun foo ()
  (flet ((create (x)            ;   BAR::CREATE
           (vector :vector x)))
    (m (create 1))))            ;   BAR::CREATE

So (m (create 1)) is (cl-user::create (bar::create 1)). This means that the user code is not shadowing the function used by the macro.

But we can explicitly use the CL-USER::CREATE symbol:

(defun foo ()
  (flet ((cl-user::create (x)            ;   CL-USER::CREATE
           (vector :vector x)))
    (m (cl-user::create 1))))            ;   CL-USER::CREATE

So (m (create 1)) is (cl-user::create (cl-user::create 1)).

Using uninterned symbols

Common Lisp also provides symbols which are not interned in a package. A special reader macro notation makes it possible to refer to such a symbol multiple times in an expression: #1= is a marker and #1# references the marked object.

(defun foo ()
  (flet ((#1=#:create (x)             ; #:CREATE
           (vector :vector x))) ;
    (m (#1# 1))))                     ; #:CREATE from above

So (m (#1# 1)) is (cl-user::create (#:create 1)), where the `#:create symbol is the uninterned symbol referenced from above.

Using computed symbols

Common Lisp also allows code to be run at read-time using the #. reader macro.

(defun foo ()
  #.(let ((create-symbol (gensym "CREATE-")))
      `(flet ((,create-symbol (x)              ; #:CREATE-NNN
                (vector :vector x))) 
         (m (,create-symbol 1)))))             ; #:CREATE-NNN from above

Which might create code similar to this:

(DEFUN FOO ()
  (FLET ((#:CREATE-815 (X)                 ; #:CREATE-815
           (VECTOR :VECTOR X)))
    (M (#:CREATE-815 1))))                 ; #:CREATE-815 from above

So (m (#:CREATE-815 1)) is (cl-user::create (#:create-815 1)), where the #:create-815 symbol is the symbol referenced from above.

Diez

This doesn't seem to answer the question, "What is the difference between a package system, a namespace system and a module system"? – Harrisharrisburg