Why can't constants be used as array dimensions in Common Lisp type specifiers?

Question

At least some implementations of Common Lisp don't allow user-defined constants to be used as array dimensions in some type specifiers. For example, in SBCL, this code:

(defconstant +len+ 3)

(defun foo (x) 
  (declare (type (simple-array fixnum (+len+)) x))
  x)

generates this error:

; in: DEFUN FOO
;     (TYPE (SIMPLE-ARRAY FIXNUM (+LEN+)) X)
; 
; caught ERROR:
;   bad dimension in array type: +LEN+

Why? It seems surprising that user-defined constants can't be used in type specifiers, since it would be desirable to be able to coordinate multiple type specifiers using some kind of global definition. I understand that type specifiers need to be completely understandable at compile-time. But I would have thought that a compiler would be able to replace symbols defined with defconstant by their literal values. I would have thought that this was one of the purposes of defconstant. (I've been unsuccessful, so far, in getting deeper understanding of this issue from the Common Lisp Hyperspec, CLTL2, the SBCL manual, or what Google has turned up. I suspect the answer is there in some form ....)

Aeolis

Are you saying (implying) that there are some LISP compilers that allow this, but others that do no? – Excalibur

I don't want to make claims about compilers or uses of type specifications that I haven't tested. I don't know what might happen in different cases. However, I've found that in CCL, for example, in (map '(simple-array fixnum (4)) #'1+ (the (simple-array fixnum (4)) arr)), replacing the first 4 with a constant generates an error, but replacing the second 4 only generates a warning, and the correct result is returned. In SBCL, by contrast, both replacements cause errors. – Aeolis

@Aeolis When you say "replacing the first 4 with a constant generates an error", it sounds like you mean you've replaced 4 with the name of a constant, which is quite different from replacing 4 with the value of a constant. defconstants aren't like C #defines which do textual substitution. – Hemorrhoid

Yes, @JoshuaTaylor, right--I did mean substituting the string "+len+" for the string "4" in the source file or in what I typed in at the repl prompt. – Aeolis

Answer 1

I imagine that something like this would work:

(defconstant +len+ 3)

(deftype len-3-fixnum-array () `(array fixnum (,+len+)))

(defun foo (x)
  (declare (type len-3-fixnum-array x))
  (print x))

(foo (make-array 3 :element-type 'fixnum))
;; #(0 0 0)

(foo (make-array 4 :element-type 'fixnum))
;; The value #(0 0 0 0) is not of type (VECTOR FIXNUM 3).
;;    [Condition of type TYPE-ERROR]

However, you need to keep in mind that type annotations are only recommendations, compilers are allowed to ignore them altogether.

---

I.e. the problem isn't that you can't use constants in the declare form, it's that it acts as if it was quoted, so your constant there doesn't evaluate to 3, it just stays the symbol +len+.

Chandos

+1 for pointing out the actual problem: the list (+len+) whose element is a symbol isn't a valid dimension specifier, but the list produced by (list +len+), whose element is the value of +len+, is. – Hemorrhoid

Thanks--nice trick wvxvw. Thanks for that extra comment @JoshuaTaylor, highlighting the significance of the backquote/comma in wvxvw's answer. – Aeolis

wvxvw's method doesn't require that the length parameter be a constant, it appears. Also embedding the backquoted declaration directly into the defun seems to work as well: (defun foo (x) (declare (type (simple-array fixnum (,+len+)) x)) (print x))`. This seems to be exactly the solution I wanted. – Aeolis

Accepting wvxvw's answer despite the fact that the other and answers provided important information. wvxvw's answer gives a clean solution to the underlying problem, and I can only accept one. Thanks everyone. – Aeolis

Sorry, there was a typo and an error two comments back. Should have been (defun foo (x) `(declare (type (simple-array fixnum (,+len+)) x)) (print x)), and it doesn't work. Compiles, but doesn't provide type checking, etc. Just gets ignored. – Aeolis

@Aeolis declarations have this peculiar "inconvenient" property. You cannot "glue" them from pieces by using macros either (see here: stackoverflow.com/questions/18645299 ). But similar to the the code in the answer there, you could use a reader-macro (although that would be probably a little weird). – Chandos

@Mars: You can use the read time evaluation reader macro (#.) with your declaration, i.e. (defun foo (x) #.`(declare (type (simple-array fixnum (,+len+)) x)) (print x)). That should solve your problem because now the expansion will happen at read time instead of macro-expansion time. – Veliz

Ah, excellent @Rörd, wvxvw. Pushing the reader macro operator inside, as in (defun foo (x) (declare (type (simple-array fixnum (#.+len+)) x)) (print x)), seems to work as well. This seems like the cleanest solution for what I wanted. I should learn a bit more about reader macros--didn't know about this. – Aeolis

@Mars: BTW, you might want to wrap your defconstant in eval-when to make sure the constant will always be defined at read time, e.g. (eval-when (:compile-toplevel :load-toplevel :execute) (defconstant +len+ 3)). – Veliz

Answer 2

I had the same problem with a 2D array:

(defconstant board-width  4)
(defconstant board-height 3)

(setq *board* (make-array '(board-width board-height) :initial-element 0))

I always got the error

The value BOARD-WITH is not of type SB-INT:INDEX.

So I changed the last line this way:

(setq *board* (make-array (list board-width board-height) :initial-element 0))

and it works perfectly.

Pickerelweed

Answer 3

If you look at the ANSI CL spec, the syntax for types is clearly defined. For simple-array:

simple-array [{element-type | *} [dimension-spec]]

dimension-spec::= rank | * | ({dimension | *}*) 

Where:

dimension---a valid array dimension.

element-type---a type specifier.

rank---a non-negative fixnum.

The valid array dimension is then explained as a fixnum.

There are no variables or constant identifiers in an simple-array type declaration. Type declarations have their own syntax and are not evaluated or similar. They are type expressions which can appear in code in special places (-> declarations).

You can use DEFTYPE to create new type names and use those, though.

Henshaw

I hard read that part of the spec, but I wasn't sure whether a variable defined to a particular fixnum value using defconstant counted as a fixnum for the purposes of defining array dimensions. Your comment clarifies that for me. – Aeolis

Answer 4

Well, because nobody seems to actually give what the questioner needs, here it is:

(defun foo (x) 
  (declare (type (simple-array fixnum (#.+len+)) x))
  x)

#. is a standard readmacro that evaluates the value in the read-time. Lisp form only sees what was expanded. http://www.lispworks.com/documentation/HyperSpec/Body/02_dhf.htm

Deflocculate

Answer 5

The other two answers tell you how to work around the limitation and where the limitation is specified in the standard.

I will try to tell you where this limitation is coming from.

The reason is that the Lisp declarations are intended not just as code documentation for the human reader, but also as a help for the compiler in optimizing the code. I.e., the array size, when declared, should permit the compiler to allocate a fixed size array, if necessary, and infer the storage requirements for array indexes. It is similar to the C limitation which forbids code like

int N = 10;
int v[N];

When you start looking at the declarations from the compiler's POV, the requirements will become quite natural.

Adhere

Thanks sds. That's what I was wondering. But having to put literals in array type specifiers is bug-prone, makes code harder to maintain, etc. So I figured there would be a good reason to allow C-ish macros, and defconstant seemed like the only language element that might do that. However, it turns out that C-style macros aren't needed; Regular Lisp macros are enough, as @wvxvw's answer showed. – Aeolis