Trying to understand setf + aref "magic"

Question

I now have learnt about arrays and aref in Lisp. So far, it's quite easy to grasp, and it works like a charme:

(defparameter *foo* (make-array 5))
(aref *foo* 0) ; => nil
(setf (aref *foo* 0) 23)
(aref *foo* 0) ; => 23

What puzzles me is the aref "magic" that happens when you combine aref and setf. It seems as if aref knew about its calling context, and would then decide whether to return a value or a place that can be used by setf.

Anyway, for the moment I just take this as granted, and don't think about the way this works internally too much.

But now I wanted to create a function that sets an element of the *foo* array to a predefined value, but I don't want to hardcode the *foo* array, instead I want to hand over a place:

(defun set-23 (place)
  …)

So basically this function sets place to 23, whatever place is. My initial naive approach was

(defun set-23 (place)
  (setf place 23))

and call it using:

(set-23 (aref *foo* 0))

This does not result in an error, but it also doesn't change *foo* at all. My guess would be that the call to aref resolves to nil (as the array is currently empty), so this would mean that

(setf nil 23)

is run, but when I try this manually in the REPL, I get an error telling me that:

NIL is a constant, may not be used as a variable

(And this absolutely makes sense!)

So, finally I have two questions:

1. What happens in my sample, and what does this not cause an error, and why doesn't it do anything?
2. How could I solve this to make my set-23 function work?

I also had the idea to use a thunk for this to defer execution of aref, just like:

(defun set-23 (fn)
  (setf (funcall fn) 23))

But this already runs into an error when I try to define this function, as Lisp now tells me:

(SETF FUNCALL) is only defined for functions of the form #'symbol.

Again, I wonder why this is. Why does using setf in combination with funcall apparently work for named functions, but not for lambdas, e.g.?

PS: In "Land of Lisp" (which I'm currently reading to learn about Lisp) it says:

In fact, the first argument in setf is a special sublanguage of Common Lisp, called a generalized reference. Not every Lisp command is allowed in a generalized reference, but you can still put in some pretty complicated stuff: […]

Well, I guess that this is the reason (or at least one of the reasons) here, why all this does not work as I'd expect it, but nevertheless I'm curious to learn more :-)

Final

There's some discussion of setf expansions and the like in About generalized variable in onlisp and the answer to that question includes a list of some other SO posts that use setf-expansions. How do I globally change a variable value within function in lisp also has discussion about places and generalized references. It also has links to some related questions, too. – Chambertin

Also, setf is a macro. It's worth looking at its expansion. E.g., (on SBCL, as this is implementation dependent), (macroexpand '(setf (aref a 2) 42)) => (SB-KERNEL:%ASET A 2 42). – Chambertin

Sorry @Mark, i deleted my unfinished comment; so your response now dangling. – Gearbox

Answer 1

A place is nothing physical, it's just a concept for anything where we can get/set a value. So a place in general can't be returned or passed. Lisp developers wanted a way to easily guess a setter from just knowing what the getter is. So we write the getter, with a surrounding setf form and Lisp figures out how to set something:

      (slot-value vehicle 'speed)        ; gets the speed
(setf (slot-value vehicle 'speed) 100)   ; sets the speed

Without SETF we would need a setter function with its name:

(set-slot-value vehicle 'speed 100)      ; sets the speed

For setting an array we would need another function name:

(set-aref 3d-board 100 100 100 'foo)    ; sets the board at 100/100/100

Note that the above setter functions might exist internally. But you don't need to know them with setf.

Result: we end up with a multitude of different setter function names. The SETF mechanism replaces ALL of them with one common syntax. You know the getter call? Then you know the setter, too. It's just setf around the getter call plus the new value.

Another example

      world-time                         ; may return the world time
(setf world-time (get-current-time))     ; sets the world time

And so on...

Note also that only macros deal with setting places: setf, push, pushnew, remf, ... Only with those you can set a place.

(defun set-23 (place)
  (setf place 23))

Above can be written, but place is just a variable name. You can't pass a place. Let's rename it, which does not change a thing, but reduces confusion:

(defun set-23 (foo)
  (setf foo 23))

Here foo is a local variable. A local variable is a place. Something we can set. So we can use setf to set the local value of the variable. We don't set something that gets passed in, we set the variable itself.

(defmethod set-24 ((vehicle audi-vehicle))
  (setf (vehicle-speed vehicle) 100))

In above method, vehicle is a variable and it is bound to an object of class audi-vehicle. To set the speed of it, we use setf to call the writer method.

Where does Lisp know the writer from? For example a class declaration generates one:

(defclass audi-vehicle ()
   ((speed :accessor vehicle-speed)))

The :accessor vehicle-speed declaration causes both reading and setting functions to be generated.

The setf macro looks at macro expansion time for the registered setter. That's all. All setf operations look similar, but Lisp underneath knows how to set things.

Here are some examples for SETF uses, expanded:

Setting an array item at an index:

CL-USER 86 > (pprint (macroexpand-1 '(setf (aref a1 10) 'foo)))

(LET* ((#:G10336875 A1) (#:G10336876 10) (#:|Store-Var-10336874| 'FOO))
  (SETF::\"COMMON-LISP\"\ \"AREF\" #:|Store-Var-10336874|
                                   #:G10336875
                                   #:G10336876))

Setting a variable:

CL-USER 87 > (pprint (macroexpand-1 '(setf a 'foo)))

(LET* ((#:|Store-Var-10336877| 'FOO))
  (SETQ A #:|Store-Var-10336877|))

Setting a CLOS slot:

CL-USER 88 > (pprint (macroexpand-1 '(setf (slot-value o1 'bar) 'foo)))

(CLOS::SET-SLOT-VALUE O1 'BAR 'FOO)

Setting the first element of a list:

CL-USER 89 > (pprint (macroexpand-1 '(setf (car some-list) 'foo)))

(SYSTEM::%RPLACA SOME-LIST 'FOO)

As you can see it uses a lot of internal code in the expansion. The user just writes a SETF form and Lisp figures out what code would actually do the thing.

Since you can write your own setter, only your imagination limits the things you might want to put under this common syntax:

• setting a value on another machine via some network protocol
• setting some value in a custom data structure you've just invented
• setting a value in a database

Schumacher

Answer 2

In your example:

(defun set-23 (place)
  (setf place 23))

you can't do it just like that, because you have to use setf in context. This will work:

(defmacro set-23 (place)
  `(setf ,place 23))

CL-USER> (set-23 (aref *foo* 0))
23
CL-USER> *foo*
#(23 NIL NIL NIL NIL)

The trick is, setf 'knows' how to look at real place its arguments come from, only for limited number of functions. These functions are called setfable.

setf is a macro, and to use it the way you wanted to, you also have to use macros.

The reason why you have not been getting errors, is that you actually successfully modified lexical variable place which was bound to copy of selected array element.

Radiothermy