Why does using defpackage result in a NAME-CONFLICT?

Question

So I'm practicing lisp with Project Euler, and I'm collecting little utility functions into a separate file to reduce duplication, and I expect that it might get pretty huge, so I've gone ahead and made a package definition. Here is a condensed version of the file that still illustrates my problem:

(defpackage :euler-util
    (:use :common-lisp)
    (:export :divisible-by))

(in-package :euler-util)

(defun divisible-by (x y)
    (equal 0 (mod x y)))

I can compile and load (C-c C-k) that file into SLIME without any warnings or errors. Now, when I go to use it in another file, I do the following:

(load "util.lisp")
(use-package :euler-util)

(defun euler-1 ()
    (loop for i from 3 to 999 when (or (divisible-by i 3) (divisible-by i 5)) sum i))

When I try to compile and load THAT file, I get an error like this:

"USE-PACKAGE # causes name-conflicts in # between the following symbols: EULER-UTILS:DIVISIBLE-BY, COMMON-LISP-USER::DIVISIBLE-BY [Condition of type NAME-CONFLICT]"

Why is this symbol appearing in the COMMON-LISP-USER package, and how do I stop it?

Ratline

Could just change the name of your divisible-by function. – Cateyed

Did you evaluate any of that code ahead of time? It sounds like you're not loading this code into a fresh Lisp. – Deportee

@RobertHarvey But there's no CL:DIVISIBLE-BY to clash with; it's CL-USER:DIVISIBLE-BY, which suggests that some of this code has already been read and symbols interned into CL-USER. – Deportee

Ah, yeah. The environment is apparently not getting cleaned up. https://mcmap.net/q/485153/-reset-state-in-common-lisp – Cateyed

@RobertHarvey That's the common cause of this problem, but it's not the problem here. Xach caught it correctly; here it's that the file with use-package is being compiled but the effects of use-package (a function, not a macro) occur at load-time, not at compile time. Thus the compiled file refers to cl-user::divisible-by, and there's a name conflict once the other package is defined. – Deportee

Answer 1

compile-file reads all the forms in the file, and only executes forms that expand into the appropriate eval-when expressions. In your example above, the util.lisp file is not loaded until after the referencing file is already read (interning all its symbols in cl-user). use-package is similarly a plain function call that is not evaluated until load time, and that's when you're asking it to make two distinct symbols by the same name accessible in the current package.

One option is to put the load and use-package statements in an eval-when form, e.g.

(eval-when (:compile-toplevel :load-toplevel :execute)
  (load "util.lisp")
  (use-package :euler))

I think it would be better to define a new package for your project, and put it at the top of your file:

(defpackage #:euler
  (:use #:cl #:euler-util))

(in-package #:euler)

Those statements are automatically evaluated at compile time, so there's no need for eval-when.

Experienced CL authors tend to avoid this problem by defining packages in a specific order, putting in-package forms in every source file, and by using a system definition facility to compile and load files in the right order.

I use ASDF as a system definition facility. A simple system file could look like this:

;;;; my-project.asd

(asdf:defsystem my-project
  :serial t
  :components ((:file "util")
               (:file "my-project")))

If you put that file somewhere ASDF knows about, (asdf:load-system "my-project") will then compile and load the files in the specified order.

If you use Quicklisp, one easy way to do it is put the project's directory in ~/quicklisp/local-projects/ and then use (ql:quickload "my-project") to automatically load it and its dependencies.

Ceyx

This is exactly right. I didn't catch on to the use-package call and the fact that @Lee is compiling the file until some more comments from him. The symptoms are very similar to the common "dirty-image" problem, but the cause here is sneakily different. Well done, +1. – Deportee

Answer 2

Note: I wrote this answer in response to my initial understanding of the problem, where I though that the image was still "dirty" from some previous code. That's a common case of these sorts of symptoms. This case, however, arises from the fact that the effects of use-package don't occur at compile time, but rather at load time. Thus, when I load the file below, there's no problem, but in compiling it as OP was, there's a reference to cl-user::divisble-by. I'm leaving this answer here because people with this problem are still likely to find it, but it's not exactly the same problem that OP is having.

Your code:

I've made local copies of your code in util.lisp and file2.lisp:

$ cat util.lisp 
(defpackage :euler-util
    (:use :common-lisp)
    (:export :divisible-by))

(in-package :euler-util)

(defun divisible-by (x y)
    (equal 0 (mod x y)))

$ cat file2.lisp 
(load "util.lisp")
(use-package :euler-util)

(defun euler-1 ()
    (loop for i from 3 to 999 when (or (divisible-by i 3) (divisible-by i 5)) sum i))

Reproducing the problem

This problem is easy to reproduce: if you've somehow interned the a symbol named "DIVISIBLE-BY" into the CL-USER package, and later try to use a package that exports a symbol with the same name, you'll have trouble:

$ sbcl --noinform
* (defun divisible-by (&rest args) (declare (ignore args)))

DIVISIBLE-BY
* (load "file2")

debugger invoked on a NAME-CONFLICT in thread #<THREAD "initial thread" RUNNING
                                                 {1002979041}>:
  USE-PACKAGE #<PACKAGE "EULER-UTIL"> causes name-conflicts in
  #<PACKAGE "COMMON-LISP-USER"> between the following symbols:
    EULER-UTIL:DIVISIBLE-BY, COMMON-LISP-USER::DIVISIBLE-BY
See also:
  The ANSI Standard, Section 11.1.1.2.5

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RESOLVE-CONFLICT] Resolve conflict.
  1: [RETRY           ] Retry EVAL of current toplevel form.
  2: [CONTINUE        ] Ignore error and continue loading file "…/file2.lisp".
  3: [ABORT           ] Abort loading file "…/file2.lisp".
  4:                    Exit debugger, returning to top level.

(NAME-CONFLICT
 #<PACKAGE "COMMON-LISP-USER">
 USE-PACKAGE
 #<PACKAGE "EULER-UTIL">
 EULER-UTIL:DIVISIBLE-BY
 DIVISIBLE-BY)

Resolving the conflict

If this happens, you can resolve it manually. I'm using SBCL on the command line here, but you should (I hope) get similar debugger options in SLIME. I'm pushing a lot into the command line for brevity. The points here are that: --noinform keeps the banner from being printed; --eval "'divisible-by" makes sure that there's a symbol cl-user:divisible-by; and --load file2.lisp loads your file. That is, we've recreated the problem on the command line so that we can focus on fixing the problem.

$ sbcl --noinform --eval "'divisible-by" --load file2.lisp 

debugger invoked on a NAME-CONFLICT in thread #<THREAD "initial thread" RUNNING
                                                 {1002979311}>:
  USE-PACKAGE #<PACKAGE "EULER-UTIL"> causes name-conflicts in
  #<PACKAGE "COMMON-LISP-USER"> between the following symbols:
    EULER-UTIL:DIVISIBLE-BY, COMMON-LISP-USER::DIVISIBLE-BY
See also:
  The ANSI Standard, Section 11.1.1.2.5

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RESOLVE-CONFLICT] Resolve conflict.
  1: [RETRY           ] Retry EVAL of current toplevel form.
  2: [CONTINUE        ] Ignore error and continue loading file "/home/taylorj/tmp/package-issue/file2.lisp".
  3: [ABORT           ] Abort loading file "/home/taylorj/tmp/package-issue/file2.lisp".
  4:                    Ignore runtime option --load "file2.lisp".
  5:                    Skip rest of --eval and --load options.
  6:                    Skip to toplevel READ/EVAL/PRINT loop.
  7: [QUIT            ] Quit SBCL (calling #'QUIT, killing the process).

(NAME-CONFLICT
 #<PACKAGE "COMMON-LISP-USER">
 USE-PACKAGE
 #<PACKAGE "EULER-UTIL">
 EULER-UTIL:DIVISIBLE-BY
 DIVISIBLE-BY)

Now you can type 0 if you want to resolve the conflict:

0] 0

Select a symbol to be made accessible in package COMMON-LISP-USER:
  1. EULER-UTIL:DIVISIBLE-BY
  2. COMMON-LISP-USER::DIVISIBLE-BY

Enter an integer (between 1 and 2): 1

By making euler-util:divisible-by the symbol accessible in cl-user, your code will work as you expect:

* (euler-1)

233168

With a fresh image, there's no problem.

While you might not have caused the problem in the same way that I did above, it's probably something similar. It appears that you're loading code in a non-fresh environment (e.g., where you've already entered some code that caused the reader to intern "DIVISIBLE-BY" in the CL-USER package. With a fresh Lisp, your code loads just fine:

$ sbcl --load file2.lisp --noinform 
* (euler-1)

233168

Deportee

Everyone keeps mentioning that I seem to be loading code into a non-fresh image, but unless that phrase means something other than what I would think (i.e. the state of affairs immediately after doing "M-x slime"), I AM loading into a fresh image. – Ratline

@LeeCrabtree Well, it probably means what you think it does, but there might be unexpected ways to sully that fresh image. When the reader reads a form, it has to intern the symbols that it's reading, and it does that based on explicit package prefixes (e.g., foo::bar) and the current value of *package*. Typically this type of problem would occur if you had typed divisible-by at the REPL, where the reader would intern divisible-by into the cl-user package. Then when you try to (use-package :euler-util) which exports a symbol with the same name, there's a conflict. If you're – Deportee

@LeeCrabtree not doing something like that, then something similar is probably happening, but in a sneakier way. For instance, you might C-x C-e in a Lisp buffer and evaluate some code. You might have something in a Slime initialization file that does something with divisible-by; there are lots of ways this could happen. If you start Emacs, do M-x slime, and then at the REPL type (load "file2"), do you still have this problem with the exact code that you've shown us? – Deportee

@LeeCrabtree Actually, looking more closely at your code, I wonder if the problem isn't this: use-package is a function that the compiler doesn't treat differently than any other function. Your file in which use-package appears doesn't include an in-package form. If you're compiling that file, rather than just loading it, the use-package isn't going to affect the call to divisible-by, and since the use-package isn't evaluated for the compilation, you end up with cl-user::divisble-by. The problem here is that you're compiling your file2; we've all been loading it. – Deportee

And now, after writing all that commentary and going to edit my answer, I realize that @Ceyx already caught it answered appropriately. His answer is correct. – Deportee