How to modify place with arbitrary function

Question

Sometimes we need to modify a place but here is no built-in function that meets our needs.

For instance, here are incf and decf for addition and subtraction:

CL-USER> (defvar *x* 5)
*X*
CL-USER> (incf *x* 3)
8
CL-USER> *x*
8
CL-USER> (decf *x* 10)
-2
CL-USER> *x*
-2

But how about multiplication and division? What if we wish to modify a place with arbitrary function, like this:

(xf (lambda (x) ...) *x*)

xf utility would be very useful, especially when we have to deal with deeply nested structures:

(my-accessor (aref (cdr *my-data*) n))

Steffens

Possible duplicate of defining setf-expanders in Common Lisp – Timm

Answer 1

Defining new macros with define-modify-macro

One simple way to define new handy macros for our needs is define-modify-macro. This is a handy macro which can create other macros for us.

Syntax:

define-modify-macro name lambda-list function [documentation]

⇒ name

We should supply name of new macro, list of parameters (not including place there) and symbol of function that will be used for processing.

Example of use:

(define-modify-macro togglef () not
  "togglef modifies place, changing nil value to t and non-nil value to nil")

(define-modify-macro mulf (&rest args) *
  "mulf modifies place, assigning product to it")

(define-modify-macro divf (&rest args) /
  "divf modifies place, assigning result of division to it")

However, define-modify-macro cannot be used for arbitrary processing. Here we have to take a look at other possibilities.

Function get-setf-expansion

Function get-setf-expansion does not create any macros, but provides information which we can use to write our own.

Syntax:

get-setf-expansion place &optional environment

⇒ vars, vals, store-vars, writer-form, reader-form

As you can see, it returns a bunch of values, so it may be confusing at first sight. Let's try it on example:

CL-USER> (defvar *array* #(1 2 3 4 5))
*ARRAY*
CL-USER> (get-setf-expansion '(aref *array* 1))
; get-setf-expansion is a function, so we have to quote its argument
(#:G6029 #:G6030)        ; list of variables needed to modify place
(*ARRAY* 1)              ; values for these variables
(#:G6031)                ; variable to store result of calculation
(SYSTEM::STORE #:G6029   ; writer-form: we should run it to modify place
               #:G6030   ; ^
               #:G6031)  ; ^  
(AREF #:G6029 #:G6030)   ; reader-form: hm.. looks like our expression

Writing xf macro

It seems like now we've got all information to write our xf macro:

(defmacro xf (fn place &rest args &environment env)
  (multiple-value-bind (vars forms var set access)
      (get-setf-expansion place env)
    (let ((g (gensym)))
      `(let* ((,g ,fn)   ; assign supplied function to generated symbol
              ,@(mapcar #'list vars forms) ; generate pairs (variable value)
              (,(car var) (funcall ,g ,access ,@args))) ; call supplied function
              ; and save the result, we use reader-form here to get intial value
         ,set)))) ; just put writer-from here as provided

Note, that xf macro takes evironment variable and pass it to get-setf-expansion. This variable is needed to ensure that any lexical bindings or definitions established in the compilation environment are taken into account.

Let's try it:

CL-USER> (defvar *var* '(("foo" . "bar") ("baz" . "qux")))
*VAR*
CL-USER> (xf #'reverse (cdr (second *var*)))
"xuq"
CL-USER> *var*
(("foo" . "bar") ("baz" . "xuq"))

Expansion:

(LET* ((#:G6033 #'REVERSE)
       (#:TEMP-6032 (SECOND *VAR*))
       (#:NEW-6031 (FUNCALL #:G6033
                            (CDR #:TEMP-6032))))
  (SYSTEM::%RPLACD #:TEMP-6032 #:NEW-6031))

I hope this information is useful.

This answer is based on Paul Graham's On Lisp, section 12.4 More Complex Utilities.

Steffens

This is neat! There are also (more detailed) examples of define-modify-macro in Writing a destructive macro or function like incf?, and of define-modify-macro and get-setf-expansion in what is to append as push is to cons, in Lisp?. That said, this is actually quite possible with define-modify-macro (plus an additional macro to reverse the argument order), as I've shown in my answer. The thing to note is that the function provided to define-modify-macro… – Rosenwald

…can do a lot, including making a call to apply or funcall with arguments in a different order. I.e., define-modify-macro with (lambda (v f) (funcall f v)) is just like your xf, but with reversed argument order. That's probably an OK stopping point, but except for issues with order of evaluation, you can simply wrap that in another macro that reorders the arguments, and you get your XF. The argument order issue is enough to justify using get-setf-expansion, though, or the simple version with reversed argument order. – Rosenwald

Nice answer. One detail: you should add &environment env to the lambda list of a macro calling get-setf-expansion and pass env as the second argument to get-setf-expansion. This prevents it from setting the wrong place in some rare instances. It doesn't change the calling convention of the macro. – Imbricate

Answer 2

Using define-modify-macro (plus a bit)

Mark's answer provides a thorough way to do this from scratch, but this can actually be approximated with define-modify-macro, and done with define-modify-macro plus another macro:

(define-modify-macro xxf (function)  ; like XF, but the place comes first, then the function
  (lambda (value function)
    (funcall function value)))

(let ((l (copy-tree '(("foo" . "bar") ("baz" . "qux")))))
  (xxf (cdr (second l)) #'reverse)
  l)
;=> (("foo" . "bar") ("baz" . "xuq"))

To reverse the order, it's easy to define a macro xf that expands to an xxf call:

(defmacro xf (function place)
  `(xxf ,place ,function))

(let ((l (copy-tree '(("foo" . "bar") ("baz" . "qux")))))
  (xf #'reverse (cdr (second l)))
  l)
;=> (("foo" . "bar") ("baz" . "xuq"))

Making it better

The current version only accepts a single function as an argument. Many functions take additional arguments, though (e.g., additional required arguments, keyword arguments, and optional arguments). We can still handle those with define-modify-macro though:

(define-modify-macro xxf (function &rest args)
  (lambda (value function &rest args)
    (apply function value args)))

(defmacro xf (function place &rest args)
  `(xxf ,place ,function ,@args))

(let ((l (copy-tree '("HeLlo WoRlD" "HeLlo WoRlD"))))
  (xf #'remove-duplicates (first l) :test #'char=)
  (xf #'remove-duplicates (second l) :test #'char-equal)
  l)
;=> ("HeL WoRlD" "He WoRlD")

Rosenwald

@Mark Using a lambda function with define-modify-macro has some nice advantages. The implementations of XXF I gave work pretty well, and without any real confusion (except that you end up writing (xxf place #'reverse) instead of (xf #'reverse place)). Defining xf as a macro that expands to xxf has some significant pitfalls, though, if the evaluation of place or the function has any side effects. :( – Rosenwald

Well, I would use your xxf with this order of arguments (place first). Exact order of arguments is not so important. Even in Common Lisp standard there is no systematic approach in this respect. For example, nth takes index first, while elt and aref take sequence first... – Steffens

@Mark That's true, but in defining these kinds of modifying macros that are based on existing functions, it can be nice to make the macro take arguments in the same order that the function would accept them. Of course, neither XF nor XXF can do that if the place should be anything but the first or second argument, so you're right, it's not all that big a deal. – Rosenwald