Any good way to declare unused variables in destructuring-bind?

Question

I can't figure, is there any way to put something like _ in erlang, for "unused value" in destructuring-bind?

For example there we have something like that:

(destructuring-bind ((_SNIPPET
                               (_TITLE . title)
                               (_DESCRIPTION . description)
                               _RESOURCE-ID (_VIDEO-ID . video-id)))) entry
                (declare (ignore 
                          _SNIPPET _TITLE _DESCRIPTION _RESOURCE-ID _VIDEO-ID))
                (list video-id title description)))

It'll be great not to put specific variable for every unused value, and write something like that:

 (destructuring-bind ((_
                         (_ . title)
                         (_ . description)
                         (_ (_ . video-id)))) entry
                    (list video-id title description)))

Is there any way to get such behavior with standart destructuring-bind or any other standart macros? Or I have to use some ML-like pattern matching library, and if so - which one?

Sailplane

You don't have to, but if you decide, that said library is optima: github.com/m2ym/optima – Convict

It's not an answer, but there is Issue DESTRUCTURING-NIL on the CLiki about how NIL is handled in destructuring lambda lists. One of the proposed resolutions is "Proposal (DESTRUCTURING-NIL:WILDCARD): Make it explicit that NIL as element of a macro lambda list or destructuring lambda list that has no lambda list keywords matches any object, but produces no variable bindings." – Zackzackariah

@coredump I know you've already got a solution that works in this case, but loop has much less destructuring power than d-bind. Rainer pointed out a number of flaws in my original answer, but it's now much safer and robust. – Zackzackariah

@JoshuaTaylor Yes, you right of course. As I mentioned in comments before, for just this particular case loop is ok, and even better then d-bind, but for more general usage it'll be great to have ability to write something like: (with-matches ((:keyword (10 var0) ("string" var1) ('symbol var2) (_ var3 (_ var4 &rest _)))) '((:keyword (10 20) ("string" 30) ('symbol 40) (3/5 50 ((some unused list) var4 'a "lot" :of (unused) 'stuff)))) (list var0 var1 var2 var3 var4)) And get (20 30 40 50). – Sailplane

Answer 1

It's not possible with DESTRUCTURING-BIND (you can't use a variable more than once, some compiler will complain). You can enumerate the variables, _1, _2, ... But then you have to ignore each of them.

LOOP can do it:

CL-USER 23 > (loop for ((a b nil c) nil d) in '(((1 2 3 4) 5 6)
                                                ((1 2 3 4) 5 6))
                   collect (list a b c d))
((1 2 4 6) (1 2 4 6))

NIL is used as the wildcard variable.

You can reuse the LOOP macro:

(defmacro match-bind (pattern object &body body)
  `(loop with ,pattern = ,object
         while nil
         finally (return (progn ,@body))))

CL-USER 37 > (match-bind ((a b nil c) nil d)
                 '((1 2 3 4) 5 6)
               (list a b c d))
(1 2 4 6)

You can use some LET-MATCH from some library. For example: https://github.com/schani/clickr/blob/master/let-match.lisp There are probably more fancy versions.

Cigar

LOOP's destructuring doesn't allow everything that DESTRUCTURING-BIND does. E.g., LOOP doesn't support keyword, optional, rest, and aux arguments. While this will work for OP's case, this won't be a "drop-in" replacement for code that uses destructuring bind. – Zackzackariah

@JoshuaTaylor: LOOP patterns are different from DESTRUCTUING-BIND patterns. He was not specifically looking for a DESTRUCTURING-BIND replacement it seemed to me, since he was also asking for other 'standard macros' or pattern matching libraries (where it is unusual to provide a DESTRUCTURING-BIND like pattern interface). – Cigar

Since @coredump accepted the answer, I have to assume that it works for the case at hand. The title of the question, though, is "Any good way to declare unused variables in destructuring-bind?" If someone had written (destructuring-bind ((_x &rest xs) (_y &optional z)) ...), and wanted to ignore _x and _y, it won't work to do (match-bind ((nil &rest xs) (nil &optional z)) ...). It will work for lots of cases, but not everything that destructuring-bind works for. Just wanted to make sure the difference is clear. – Zackzackariah

@JoshuaTaylor: true, but your example also does not work. It fails to understand the destructuring-bind syntax. – Cigar

Thank you. For this particular case loop doing all I need. But if we want to speak about some more general usage, it'll be great to have something what you can call like that: (with-matches ((:keyword (10 var0) ("string" var1) ('symbol var2) (_ var3 (_ var4 &rest _)))) '((:keyword (10 20) ("string" 30) ('symbol 40) (3/5 50 ((some unused list) var4 'a "lot" :of (unused) 'stuff)))) (list 20 30 40 50)) To get (20 30 40 50) – Sailplane

A bit off-topic, but I think that somewhere in the comments here you'd mentioned duplicates in lambda lists as not permitted. That makes sense, and some implementations reject it, but others accept it. (E.g., see “duplicates in lambda-list, let bindings, etc.“, a c.l.l thread from 2008. In a pattern matching library, you'd want repeated variables (except for a wildcard) to bind to the same value, but in dest.-bind, etc., that thread applies, and repeated variables might be OK. Did you have a ref. in mind? – Zackzackariah

And a thread from 1999 (that you participated in): groups.google.com/d/msg/comp.lang.lisp/KdRq2_XMVGc/2oNbz4Pa-UcJ. – Zackzackariah

@JoshuaTaylor Yes: CLHS 3.1.1. I'd think the consequences of violating it are undefined. Thus implementations may show an error (SBCL, CCL), some warning (LispWorks) ... Symbolics for example implemented a special name IGNORE: (destructuring-bind (ignore a ignore) ...) should work. But it was mentioned in the discussion about it that other implementations did not want to support it in the standard and repeated variables won't be allowed in ANSI CL. – Cigar

Yeah, Vassil Nikolov brought up 3.1.1., but there was some more discussion after that, too, about the fact that in processing arguments, later initforms can refer to earlier variables, so there are some intermediate environments in play, too. At any rate, the second thread I linked to provides enough to say "it might or might not be allowed." In the process of writing my answer, I started writing some lambda-list parsing code, and pinning down some of these details gets tricky. :) Thanks for the (as always, useful) feedback! – Zackzackariah

Answer 2

There's nothing built into the language for this. Rainer Joswig's answer points out that loop can do some destructuring, but it doesn't do nearly as much. In an earlier version of this answer, I suggested traversing the destructuring lambda list and collecting a list of all the symbols that begin with _ and adding a declaration to the form to ignore those variables. A safer version replaces each one with a fresh variable (so that there are no repeated variables), and ignores them all. Thus something like

(destructuring-bind (_a (_b c)) object
  c)

would expand into

(destructuring-bind (#:g1 (#:g2 c)) object
  (declare (ignore #:g1 #:g2))
  c)

This approach will work OK if you're only using the "data-directed" described in 3.4.4.1.1 Data-directed Destructuring by Lambda Lists. However, if you're using "lambda-list-directed" approach described in 3.4.4.1.2 Lambda-list-directed Destructuring by Lambda Lists, where you can use lambda-list keywords like &optional, &key, etc., then things are much more complicated, because you shouldn't replace variables in some parts of those. For instance, if you have

&optional (_x '_default-x)

then it might be OK to replace _x with something, but not _default-x, because the latter isn't a pattern. But, in Lisp, code is data, so we can still write a macro that maps over the destructuring-lambda-list and replaces only in locations that are patterns. Here's somewhat hairy code that does just that. This takes a function and a destructuring lambda list, and calls the function for each pattern variable in the lambda list, along with the type of the argument (whole, required, optional, etc.).

(defun map-dll (fn list)
  (let ((result '())
        (orig list)
        (keywords '(&allow-other-keys &aux &body
                    &key &optional &rest &whole)))
    (labels ((save (x)
               (push x result))
             (handle (type parameter)
               (etypecase parameter
                 (list (map-dll fn parameter))
                 (symbol (funcall fn type parameter)))))
      (macrolet ((parse-keyword ((&rest symbols) &body body)
                   `(progn
                      (when (and (not (atom list))
                                 (member (first list) ',symbols))
                        (save (pop list))
                        ,@body)))
                 (doparameters ((var) &body body)
                   `(do () ((or (atom list) (member (first list) keywords)))
                      (save (let ((,var (pop list)))
                              ,@body)))))
        (parse-keyword (&whole)
          (save (handle :whole (pop list))))
        (doparameters (required)
         (handle :required required))
        (parse-keyword (&optional)
         (doparameters (opt)
          (if (symbolp opt)
              (handle :optional opt)
              (list* (handle :optional (first opt)) (rest opt)))))
        (when (and (atom list) (not (null list))) ; turn (... . REST) 
          (setq list (list '&rest list)))         ; into (... &rest REST)
        (parse-keyword (&rest &body)
         (save (handle :rest (pop list))))
        (parse-keyword (&key)
         (doparameters (key)
          (if (symbolp key)
              (handle :key key)
              (destructuring-bind (keyspec . more) key
                (if (symbolp keyspec)
                    (list* (handle :key keyspec) more)
                    (destructuring-bind (keyword var) keyspec
                      (list* (list keyword (handle :key var)) more)))))))
        (parse-keyword (&allow-other-keys))
        (parse-keyword (&aux)
         (doparameters (aux) aux))
        (unless (null list)
          (error "Bad destructuring lambda list: ~A." orig))
        (nreverse result)))))

Using this, it's pretty easy to write a destructuring-bind* that replaces each pattern variable beginning with _ with a fresh variable that will be ignored in the body.

(defmacro destructuring-bind* (lambda-list object &body body)
  (let* ((ignores '())
         (lambda-list (map-dll (lambda (type var)
                                 (declare (ignore type))
                                 (if (and (> (length (symbol-name var)) 0)
                                          (char= #\_ (char (symbol-name var) 0)))
                                     (let ((var (gensym)))
                                       (push var ignores)
                                       var)
                                     var))
                               lambda-list)))
    `(destructuring-bind ,lambda-list ,object
       (declare (ignore ,@(nreverse ignores)))
       ,@body)))

Now we should look at the expansions it produces:

(macroexpand-1
 '(destructuring-bind* (&whole (a _ . b)
                        c _ d
                        &optional e (f '_f)
                        &key g _h
                        &aux (_i '_j))
   object
   (list a b c d e f g)))
;=>                            
(DESTRUCTURING-BIND
    (&WHOLE (A #:G1041 &REST B) C #:G1042 D
     &OPTIONAL E (F '_F)
     &KEY G #:G1043
     &AUX (_I '_J))
    OBJECT
  (DECLARE (IGNORE #:G1041 #:G1042 #:G1043))
  (LIST A B C D E F G))

We haven't replaced anywhere we shouldn't (init forms, aux variables, etc.), but we've taken care of the places that we should. We can see this work in your example too:

(macroexpand-1
 '(destructuring-bind* ((_ (_ . title)
                         (_ . description)
                         _
                         (_ . video-id)))
   entry
   (list video-id title description)))
;=>
(DESTRUCTURING-BIND ((#:G1044 (#:G1045 &REST TITLE)
                              (#:G1046 &REST DESCRIPTION)
                              #:G1047
                              (#:G1048 &REST VIDEO-ID)))
    ENTRY
  (DECLARE (IGNORE #:G1044 #:G1045 #:G1046 #:G1047 #:G1048))
  (LIST VIDEO-ID TITLE DESCRIPTION))

Zackzackariah

You can't map over the destructuring-bind lambda list and replace the symbols. The lambda list has a syntax. One would need to take that into account. Example: (destructuring-bind** (a &aux (b '_)) '(1) (list a b)) -> (1 #:G1133) – Cigar

The first one may declare to ignore variables with an underscore, which might be used: (destructuring-bind* (a &aux (_a '_)) '(1) (list a _a)). It may also declare to ignore variables which don't exist in the destructuring-bind lambda list. – Cigar

First: Think about nested calls, where the nested call is done in a key, optional or aux binding. Your outer macro will pick up the embedded pattern variables from the nested calls... Second: with *break-on-warnings* (like in LispWorks) set to true, your additionally ignored variables may trigger a break during compilation - when the compiler emits a warning. Third: if the variable is not in a pattern, it should be fine to use an underscore ... – Cigar

@RainerJoswig You're absolutely right. I've completely reworked this answer to parse destructuring lambda lists, and to only rename variables that occur in pattern locations. There's no more worry of replacing in init forms, or in aux variables, etc. Any bugs in the implementation notwithstanding, I think the approach in this version is correct. – Zackzackariah