mapcan, sharp quote and closures

Question

I'm somewhat new to CL and currently trying to wrap my head around mapcan, #', funcall and closures. Here is a closure, which applies a predicate to a number n and, if correct, returns (list n), else nil:

(defun all-those (predicate)
  (lambda (n)
    (if (funcall predicate n) (list n))))

I understand that I need to call funcall to turn this closure into a function. This works fine:

>  (funcall (all-those #'evenp) 8)
(8)

Now I tried to pass the hereby created function as an argument to mapcan:

>  (mapcan #'(funcall (all-those #'evenp)) '(1 2 3 4))

I get a compile-time-error: (FUNCALL (ALL-THOSE #'EVENP)) is not a legal function name.

But it works if I omit #' as well as funcall:

>  (mapcan (all-those #'evenp) '(1 2 3 4))
(2 4)

Now I'm confused. It was my understanding that I need to sharp-quote a function when using mapcan to follow the symbol's function binding (*) and that I need to call funcall when "closing a closure".

Is it because #' and funcall are cancelling each other out or why do I have to omit both of them in the above example? Thank you in advance for any replies.

(*) I know that in this example I don't really have a symbol whose function binding can be followed. But if I use an anonymous function and mapcan I still need to sharp-quote it: (mapcan #'(lambda ...

Hebrews

Answer 1

To mapcar, funcall, etc., you need to pass either a function object or a symbol. If you pass a symbol, then the symbol-function of the symbol is used as the function. If you pass a function object, then it is used as the function.

Your all-those function returns a function. That means that (mapcan (all-those …) …) is fine.

The sharp quote (#') is just shorthand for the function form. That is, #'foo is the same as (function foo):

The value of function is the functional value of name in the current lexical environment.

If name is a function name, the functional definition of that name is that established by the innermost lexically enclosing flet, labels, or macrolet form, if there is one. Otherwise the global functional definition of the function name is returned.

If name is a lambda expression, then a lexical closure is returned. In situations where a closure over the same set of bindings might be produced more than once, the various resulting closures might or might not be eq.

So you only use #' or function with a function name. That means either a symbol (e.g., #'car) or a lambda expression (e.g., #'(lambda (x) x)). That means that the following doesn't work (or really make sense, even):

#'(funcall (all-those #'evenp))

Now I'm confused. It was my understanding that I need to sharp-quote a function when using mapcan to follow the symbol's function binding (*) and that I need to call funcall when "closing a closure".

The documentation for mapcar, etc., says that its first argument is:

function---a designator for a function that must take as many arguments as there are lists.

From the glossary:

function designator n. a designator for a function; that is, an object that denotes a function and that is one of: a symbol (denoting the function named by that symbol in the global environment), or a function (denoting itself). The consequences are undefined if a symbol is used as a function designator but it does not have a global definition as a function, or it has a global definition as a macro or a special form. See also extended function designator.

So, you can pass a function directly to mapcar, funcall, etc., which is exactly what you're doing in:

(mapcan (all-those …) …)

You can also do:

(mapcan (lambda (x) ...) ...)
(mapcan #'(lambda (x) ...) ...)
(mapcan 'car ...)
(mapcan #'car ...)
(mapcan (symbol-function 'car) ...)

Bumbailiff

Wow, that was quick. Thank you very much!<br> Simplifying a bit, is it correct to say that I can pass a function to mapcar (1) &ensp; directly i.e. passing a function object. In this case I don't need to sharp quote because the function object is "already there". Which is the case in my example. or (2) &ensp; I can pass a symbol with a function bound to it. In this case I need to (sharp) quote in order to get to the function object: (mapcar #'a-symbol …) Thanks again. – Hebrews

@Hebrews If you are confused (you should be) by the fact that lambda works without sharpquotes, it is because (lambda ...) is a macro that expands into (function (lambda ...)), where the inner lambda form is not macroexpanded anymore because it is under the scope of the special function operator. – Tatiana

Answer 2

Summary

(function foo) is a special form, returning a function object. Here retrieved from the name foo. We use it to get a function object.

(funcall foo) is used to call functions passed as an argument - here the variable value of foo. funcall = FUNction CALL = call a function. We use it to call a function object.

Details

Here is a closure, which applies a predicate to a number n and, if correct, returns (list n), else nil:

(defun all-those (predicate)
  (lambda (n)
    (if (funcall predicate n) (list n))))

No, that's not a closure. all-those returns a closure, but itself it isn't one.

? #'all-those
#<Compiled-function ALL-THOSE #x302000C9631F>

? (all-those #'evenp)
#<COMPILED-LEXICAL-CLOSURE (:INTERNAL ALL-THOSE) #x302000E5ECFF>

I understand that I need to call funcall to turn this closure into a function.

A closure is a function object.

? (functionp (all-those #'evenp))
T

Note: all closures are also function objects. Not all function objects are closures.

A closure is a function and associated variable bindings. It is a function object.

Note that anonymous functions are not necessarily closures. (function (lambda () ())) does not return a closure, since there are no variable bindings. You could say that it is a closure with empty bindings, but in CL speak that's not called a closure.

Note that in standard Common Lisp there is no way to determine if a function object is actually a closure and there is no way to access its bindings via variable names.

I understand that I need to call funcall to turn this closure into a function.

funcall is used to call function objects (or function objects which it will retrieve from a symbol's function value) with arguments.

Remember, there are various ways to call a function:

• call a named global function: (foo bar baz)

• call a named lexical function: (foo bar bar)

• call a named global function via a symbol: (funcall 'foo bar baz)

• call a function object from a variable value: (funcall foo bar baz)

• call a function object from a function name (lexical or global): (funcall #'foo bar baz)

• call an anonymous function object: (funcall (function (lambda (foo) foo)) 'bar) or (funcall #'(lambda (foo) foo) 'bar) or (funcall (lambda (foo) foo) 'bar)

• call an anonymous function: ((lambda (foo) foo) 'bar)

Then there is APPLY which is similar to FUNCALL but takes the arguments from a list.

(apply #'+ 1 2 3 (list 4 5 6))  is similar to  (funcall #'+ 1 2 3 4 5 6)

FUNCALL itself is a function. All its argument forms will be evaluated. The first argument needs to evaluate to a symbol or a function object. funcall will call the function object (or the function object retrieved from the symbols's function value with arguments.

FUNCTION is a special operator. It is syntax. It is not a function itself. FUNCTION expects a symbol or a lambda expression as its subform. A FUNCTION form returns a function object, corresponding to the symbol (either from a global function or a lexical function) or to the lambda expression.

? (defun foo (bar) (list bar 'baz))
FOO
? (function foo)        ; a function object from the global function
#<Compiled-function FOO #x302000CC0D1F>
? #'foo                 ; the same, differently written
#<Compiled-function FOO #x302000CC0D1F>
? (funcall #'foo 42)    ; calling the function object
(42 BAZ)
? (funcall 'foo 42)     ; calling the symbol-function of the symbol
(42 BAZ)
? (funcall (symbol-function 'foo) 42)    ; as above
(42 BAZ)
? (flet ((foo (arg) (list arg :foo)))   ; a local lexical function
    (list (foo 43)                      ; calling the local lexical function
          (funcall #'foo 42)            ; calling a function object,
                                        ; from the local lexical function
          (funcall 'foo 41)))           ; calling the symbol-function of the symbol
((43 :FOO) (42 :FOO) (41 BAZ))

Scabrous

Outstanding answer! Thank you very much! There's one thing i still don't get: If I understood correctly the all-those-function returns a function object (a closure). Why do I have to explicitely use funcall to "execute" this function object even though it is the first element of a form: ((allthose #'evenp) 7) doesn't work, (funcall (all-those #'evenp) 7) works? I always thought the first element of a (non-quoted) form is automatically interpreted as a function object and doesn't need to be called explicitely. – Hebrews

@Frank: the first element would need to be evaluated. Scheme does that, it has one namespace. Lisp with two (or more) namespaces, like CL, doesn't evaluate the first element. CL assumes that the first element of a Lisp form is either a function name, a macro name, the name of a special form or a lambda expression. Only those four things are allowed as the first element of a Common Lisp form. If you want to call a function object, you have to use FUNCALL or APPLY. That's how it is defined. – Scabrous

I still have a question. (Had to break it up in parts because it's too long). I'm playing around with funcall: > (defparameter f #'sqrt) => F, (defun f (n) (+ n PI)) => F. Now I tried (1) (funcall f 100) => 10.0 and (2) (funcall 'f 100) => 103.14… . Thanks to your explanation I think I get why (1) works. f ist in argument position so it is evaluated to the content of the value cell of f, wich ist a function object (the sqrt-function). funcall takes this function object and applies it to 100, hence 10.0. (Is this correct?) – Hebrews

Regarding (2) I'm not so sure. Since f is quoted a symbol (and not a function object) is passed to funcall. In this case "it [the symbol] is coerced to a function as if by finding its functional value in the global environment." (HyperSpec) and you said funcall "will retrieve [the function object] from a symbol's function value." Does this mean that, if funcall is passed a symbol, it will retrieve the function object by evaluating the symbol function binding? Or , in other words: Is (funcall 'a-function-name <args>) equivalent to (a-function-name <args>)? – Hebrews

@Hebrews (funcall 'foo ...) is (funcall (symbol-function 'foo) ...) – Scabrous