Why does C++11's lambda require "mutable" keyword for capture-by-value, by default?

Question

Short example:

#include <iostream>

int main()
{
    int n;
    [&](){n = 10;}();             // OK
    [=]() mutable {n = 20;}();    // OK
    // [=](){n = 10;}();          // Error: a by-value capture cannot be modified in a non-mutable lambda
    std::cout << n << "\n";       // "10"
}

The question: Why do we need the mutable keyword? It's quite different from traditional parameter passing to named functions. What's the rationale behind?

I was under the impression that the whole point of capture-by-value is to allow the user to change the temporary -- otherwise I'm almost always better off using capture-by-reference, aren't I?

Any enlightenments?

(I'm using MSVC2010 by the way. AFAIK this should be standard)

Thiazine

Good question; although I'm glad something is finally const by default! – Doorbell

Not an answer, but I think this is a sensible thing: if you take something by value, you shouldn't be changing it just to save you 1 copy to a local variable. At least you won't make the mistake of changing n by replacing = with &. – Galliwasp

I absolutely agree with stefaanv. The rationale is probably that they intended the language to be more straightforward to learn. A beginner might attempt to implement a swap function lambda with value capturing. This is what's commonly referred to as the principle of least surprise. – Servant

@xtofl: Not sure it's good, when everything else is not const by default. – Thiazine

@Tamás Szelei: Not to start an argument, but IMHO the concept "easy to learn" has no place in the C++ language, especially in modern days. Anyway :P – Thiazine

Bjarne himself stated this in talk about two years ago. I'm not saying C++ is easy to learn (nor that C++0x is), but this and the rvalue references both seem to aim at the least surprise principle. – Servant

@Tamás Szelei, just curious, how is rvalue reference "least surprising"? – Thiazine

Because using rvalue references you can pass temporary values to functions that expect references (that is, rvalue references). With only lvalue refs you can't. This, of course relies on the person who writes that particular function (and to be sensible they write one with both r and lvalue overload). A better way to phrase it is that rvalue references allow API implementers to write less surprising code for the API user (that's why I say it aims for that). At least this is my understanding about the motivation behind these things. – Servant

"the whole point of capture-by-value is to allow the user to change the temporary" - No, the whole point is that the lambda may remain valid beyond the lifetime of any captured variables. If C++ lambdas only had capture-by-ref, they would be unusable in way too many scenarios. – Loading

Answer 1

It requires mutable because by default, a function object should produce the same result every time it's called. This is the difference between an object orientated function and a function using a global variable, effectively.

Wasson

This is a good point. I totally agree. In C++0x though, I don't quite see how the default helps enforce the above. Consider I am on the receiving end of the lambda, e.g. I am void f(const std::function<int(int)> g). How am I guaranteed that g is actually referentially transparent? g's supplier might have used mutable anyway. So I won't know. On the other hand, if the default is non-const, and people must add const instead of mutable to function objects, the compiler can actually enforce the const std::function<int(int)> part and now f can assume that g is const, no? – Thiazine

This answer is accepted as I think it most closely answers the why (as in rationale) part. – Thiazine

@kizzx2: In C++, nothing is enforced, only suggested. As per usual, if you do something stupid (documented requirement for referential transparency and then pass non-referentially-transparent function), you get whatever comes to you. – Wasson

Except function objects shouldn't product the same result every time they're called. No inline functions in any other modern language are like that, and the default should be to be like the rest of the world. Leave it to C++ to keep making everything difficult and ugly. – Sev

@Glenn: Guess you must have missed all those e.g. pure lambdas in LINQ. It's not enforced; but it's certainly recommended that functions (inc. lambdas) be pure. – Wasson

@Wasson That's simply not the case in imperative languages. – Sev

This answer opened my eyes. Previously, I thought that in this case lambda only mutates a copy for the current "run". – Barbican

@GlennMaynard: It's completely true. The language paradigm is no excuse for sloppy design. – Wasson

@Wasson It's not sloppy design. Some functions are const (eg. gcc's attribute(const)), such as comparitors for sorting, but a great many aren't, such as completion handlers for async APIs. They're usually far from it, since they may start another async request or even display UI. This is a powerful paradigm, and there's nothing sloppy about it. – Sev

Completion handlers for async APIs have no need to be logically non-const. Asynchronicity doesn't make you non-const, you just have to think about it slightly differently. As for displaying UI, well, that should certainly not be done from totally random function objects... – Wasson

@ZsoltSzatmari Your comment opened my eyes! :-D I didn't get the true meaning of this answer until I read your comment. – Oppression

@Jendas, your comment lead me to ZsoItSzatmari's comment which then opened my eyes! Which probably means this answer isn't very complete. So adding some clarification. – Takeo

Yes, that might be useful. – Oppression

Seriously I can't understand what you are talking about. For example, int i = 1; auto f = [i](){++i;}; f(); f(); f(); f(); If this is not illegal, calling four times f() will produce the same result, right? UNLESS, the i in the lambda and the outer i are the same object. So, the question becomes: does the captured object in lambda and the outer object are the same object when capture-by-value? Well, I think it NOT because it's capture-by-value, instead of capture-by-reference. So why can't we change the captured value when we use capture-by-value? I think Daniel's answer is better. – Camp

I disagree with the basic premise of this answer. C++ has no concept of "functions should always return the same value" anywhere else in the language. As a design principle, I would agree it's a good way to write a function, but I don't think it holds water as the reason for the standard behavior. – Melodize

Totally agree with @lonoclastBrigham and disagree with this answer. Please don't dogmatically state that a best practice is the reason for a technical decision in the standard unless you were actually part of the discussion in the standards committee (and if so, please say so in your answer). – Macerate

I had to read the other answers to understand what the author of this one was thinking. This answer is too terse, and only useful for people who already understand what is going on. – Winne

The example given in the question would clarify the actual point better via [=]() mutable {++n;}();, as n=20 does actually produce the same result every time, hence making the necessity of mutable less obvious. Also note the comment of @ZsoltSzatmari to clarify it further. – Janel

"A function object should produce the same result every time it's called." No, Although this statement may be correct for functional programming languages, and also happens to be good programming practice, C++ is a multi-paradigm languge, not exclusively a functional language. Therefore, one could argue that the standards commitee has no authority to force a functional programming style onto C++ programmers. – Cormorant

@JeffMcClintock, but a function object is a functional programming concept, isn't it? Actually it might be the defining concept of functional programming. – Matrices

@Camp I'm with you on this. – Mayflower

@Janel The lambda under question doesn't change the outer n because it is captured by value by [=]. See the comment for the last line in the original code // "10". With ++n the local copy in side the lambda will become '11` each time. – Pecker

I would rather like to declare each closure variable as mutable or not. – Eyra

Answer 2

Your code is almost equivalent to this:

#include <iostream>

class unnamed1
{
    int& n;
public:
    unnamed1(int& N) : n(N) {}

    /* OK. Your this is const but you don't modify the "n" reference,
    but the value pointed by it. You wouldn't be able to modify a reference
    anyway even if your operator() was mutable. When you assign a reference
    it will always point to the same var.
    */
    void operator()() const {n = 10;}
};

class unnamed2
{
    int n;
public:
    unnamed2(int N) : n(N) {}

    /* OK. Your this pointer is not const (since your operator() is "mutable" instead of const).
    So you can modify the "n" member. */
    void operator()() {n = 20;}
};

class unnamed3
{
    int n;
public:
    unnamed3(int N) : n(N) {}

    /* BAD. Your this is const so you can't modify the "n" member. */
    void operator()() const {n = 10;}
};

int main()
{
    int n;
    unnamed1 u1(n); u1();    // OK
    unnamed2 u2(n); u2();    // OK
    //unnamed3 u3(n); u3();  // Error
    std::cout << n << "\n";  // "10"
}

So you could think of lambdas as generating a class with operator() that defaults to const unless you say that it is mutable.

You can also think of all the variables captured inside [] (explicitly or implicitly) as members of that class: copies of the objects for [=] or references to the objects for [&]. They are initialized when you declare your lambda as if there was a hidden constructor.

Ovolo

While a nice explanation of what a const or mutable lambda would look like if implemented as equivalent user-defined types, the question is (as in the title and elaborated by OP in comments) why const is the default, so this doesn't answer it. – Allerie

Answer 3

You have to understand what capture means! it's capturing not argument passing! let's look at some code samples:

int main()
{
    using namespace std;
    int x = 5;
    int y;
    auto lamb = [x]() {return x + 5; };

    y= lamb();
    cout << y<<","<< x << endl; //outputs 10,5
    x = 20;
    y = lamb();
    cout << y << "," << x << endl; //output 10,20

}

As you can see even though x has been changed to 20 the lambda is still returning 10 ( x is still 5 inside the lambda) Changing x inside the lambda means changing the lambda itself at each call (the lambda is mutating at each call). To enforce correctness the standard introduced the mutable keyword. By specifying a lambda as mutable you are saying that each call to the lambda could cause a change in the lambda itself. Let see another example:

int main()
{
    using namespace std;
    int x = 5;
    int y;
    auto lamb = [x]() mutable {return x++ + 5; };

    y= lamb();
    cout << y<<","<< x << endl; //outputs 10,5
    x = 20;
    y = lamb();
    cout << y << "," << x << endl; //outputs 11,20

}

The above example shows that by making the lambda mutable, changing x inside the lambda "mutates" the lambda at each call with a new value of x that has no thing to do with the actual value of x in the main function

Goth

I liked your answer better than others. Further to add lambda = function + environment/scope. Environment is chosen, when the lambda is defined. C++ has provided concept of the environment to be non-mutable copy, mutable copy or shared environment. – Tolentino

This is the best answer here. Cleared up a lot of things for me. – Burgomaster

Answer 4

I was under the impression that the whole point of capture-by-value is to allow the user to change the temporary -- otherwise I'm almost always better off using capture-by-reference, aren't I?

The question is, is it "almost"? A frequent use-case appears to be to return or pass lambdas:

void registerCallback(std::function<void()> f) { /* ... */ }

void doSomething() {
  std::string name = receiveName();
  registerCallback([name]{ /* do something with name */ });
}

I think that mutable isn't a case of "almost". I consider "capture-by-value" like "allow me to use its value after the captured entity dies" rather than "allow me to change a copy of it". But perhaps this can be argued.

Chevy

Good example. This is a very strong use-case for the use of capture-by-value. But why does it default to const? What purpose does it achieve? mutable seems out of place here, when const is not the default in "almost" (:P) everything else of the language. – Thiazine

@kizzx2: I wish const was the default, at least people would be forced to consider const-correctness :/ – Personal

@Thiazine looking into the lambda papers, it appears to me they make it default to const so they could call it whether or not the lambda object is const. For example they could pass it to a function taking a std::function<void()> const&. To allow the lambda change its captured copies, in the initial papers the data members of the closure were defined mutable internally automatically. Now you have to manually put mutable in the lambda expression. I haven't found a detailed rationale though. – Chevy

See open-std.org/JTC1/SC22/WG21/docs/papers/2008/n2651.pdf for some details. – Chevy

@litb Nice paper. It's funny how the const "Proposal" in the paper got dropped some how. It's actually how I expected it (in line with the rest of the language, like a const member function) (as C++ programmers, we don't mind being taxed to type const -- it's a fact of life). But mutable seems to have emerged in the end for some reason. – Thiazine

At this point, to me, the "real" answer/rationale seems to be "they failed to work around an implementation detail" :/ – Thiazine

Answer 5

FWIW, Herb Sutter, a well-known member of the C++ standardization committee, provides a different answer to that question in Lambda Correctness and Usability Issues:

Consider this straw man example, where the programmer captures a local variable by value and tries to modify the captured value (which is a member variable of the lambda object):

int val = 0;
auto x = [=](item e)            // look ma, [=] means explicit copy
            { use(e,++val); };  // error: count is const, need ‘mutable’
auto y = [val](item e)          // darnit, I really can’t get more explicit
            { use(e,++val); };  // same error: count is const, need ‘mutable’

This feature appears to have been added out of a concern that the user might not realize he got a copy, and in particular that since lambdas are copyable he might be changing a different lambda’s copy.

His paper is about why this should be changed in C++14. It is short, well written, worth reading if you want to know "what's on [committee member] minds" with regards to this particular feature.

Educatory

Answer 6

You need to think what is the closure type of your Lambda function. Every time you declare a Lambda expression, the compiler creates a closure type, which is nothing less than an unnamed class declaration with attributes (environment where the Lambda expression where declared) and the function call ::operator() implemented. When you capture a variable using copy-by-value, the compiler will create a new const attribute in the closure type, so you can't change it inside the Lambda expression because it is a "read-only" attribute, that's the reason they call it a "closure", because in some way, you are closing your Lambda expression by copying the variables from upper scope into the Lambda scope. When you use the keyword mutable, the captured entity will became a non-const attribute of your closure type. This is what causes the changes done in the mutable variable captured by value, to not be propagated to upper scope, but keep inside the stateful Lambda. Always try to imagine the resulting closure type of your Lambda expression, that helped me a lot, and I hope it can help you too.

Vuillard

Answer 7

See this draft, under 5.1.2 [expr.prim.lambda], subclause 5:

The closure type for a lambda-expression has a public inline function call operator (13.5.4) whose parameters and return type are described by the lambda-expression’s parameter-declaration-clause and trailingreturn- type respectively. This function call operator is declared const (9.3.1) if and only if the lambdaexpression’s parameter-declaration-clause is not followed by mutable.

Edit on litb's comment: Maybe they thought of capture-by-value so that outside changes to the variables aren't reflected inside the lambda? References work both ways, so that's my explanation. Don't know if it's any good though.

Edit on kizzx2's comment: The most times when a lambda is to be used is as a functor for algorithms. The default constness lets it be used in a constant environment, just like normal const-qualified functions can be used there, but non-const-qualified ones can't. Maybe they just thought to make it more intuitive for those cases, who know what goes on in their mind. :)

Alainaalaine

It's the standard, but why did they write it this way? – Thiazine

@kizzx2: My explanation is directly under that quote. :) It relates a bit to what litb says about the lifetime of the captured objects, but also goes a little further. – Alainaalaine

@Xeo: Oh yes, I missed that :P It's also another good explanation for a good use of capture-by-value. But why should it be const by default? I already got a new copy, it seems strange not to let me change it -- especially it's not something principally wrong with it -- they just want me to add mutable. – Thiazine

I believe there was an attempt to create a new genral function declaration syntax, looking much like a named lambda. It was also supposed to fix other problems by making everything const by default. Never completed, but the ideas rubbed off on the lambda definition. – Veneering

@Bo Persson, if those are true, what came out of C++0x is really a total mess :/ – Thiazine

@Thiazine - If we could start all over again, we would probably have var as a keyword to allow change and constant being the default for everything else. Now we don't, so we have to live with that. IMO, C++2011 came out pretty well, considering everything. – Veneering

Answer 8

I was under the impression that the whole point of capture-by-value is to allow the user to change the temporary -- otherwise I'm almost always better off using capture-by-reference, aren't I?

n is not a temporary. n is a member of the lambda-function-object that you create with the lambda expression. The default expectation is that calling your lambda does not modify its state, therefore it is const to prevent you from accidentally modifying n.

Cl

The whole lambda object is a temporary, its members also have temporary lifetime. – Overripe

@Ben : IIRC, I was referring to the issue that when someone says "temporary", I understand it to mean unnamed temporary object, which the lambda itself is, but it's members are not. And also that from "inside" the lambda, it doesn't really matter whether the lambda itself is temporary. Re-reading the question though it would appear that OP just meant to say the "n inside the lambda" when he said "temporary". – Cl

Answer 9

To extend Puppy's answer, lambda functions are intended to be pure functions. That means every call given a unique input set always returns the same output. Let's define input as the set of all arguments plus all captured variables when the lambda is called.

In pure functions output solely depends on input and not on some internal state. Therefore any lambda function, if pure, does not need to change its state and is therefore immutable.

When a lambda captures by reference, writing on captured variables is a strain on the concept of pure function, because all a pure function should do is return an output, though the lambda does not certainly mutate because the writing happens to external variables. Even in this case a correct usage implies that if the lambda is called with the same input again, the output will be the same everytime, despite these side effects on by-ref variables. Such side effects are just ways to return some additional input (e.g. update a counter) and could be reformulated into a pure function, for example returning a tuple instead of a single value.

Estell

Answer 10

You might see the difference, if you check 3 different use cases of lambda:

• Capturing an argument by value
• Capturing an argument by value with 'mutable' keyword
• Capturing an argument by reference

case 1: When you capture an argument by value, a few things happen:

• You are not allowed to modify the argument inside the lambda
• The value of the argument remains the same, whenever the lambda is called, not matter what will be the argument value at the time the lambda is called.

so for example:

{
    int x = 100;
    auto lambda1 = [x](){
      // x += 2;  // compile time error. not allowed
                  // to modify an argument that is captured by value
      return x * 2;
    };
    cout << lambda1() << endl;  // 100 * 2 = 200
    cout << "x: " << x << endl; // 100

    x = 300;
    cout << lambda1() << endl;   // in the lambda, x remain 100. 100 * 2 = 200
    cout << "x: " << x << endl;  // 300

}

Output: 
200
x: 100
200
x: 300

case 2: Here, when you capture an argument by value and use the 'mutable' keyword, similar to the first case, you create a "copy" of this argument. This "copy" lives in the "world" of the lambda, but now, you can actually modify the argument within the lambda-world, so its value is changed, and saved and it can be referred to, in the future calls of this lambda. Again, the outside "life" of the argument might be totally different (value wise):

{
    int x = 100;
    auto lambda2 = [x]() mutable {
      x += 2;  // when capture by value, modify the argument is
               // allowed when mutable is used.
      return x;
    };
    cout << lambda2() << endl;  // 100 + 2 = 102
    cout << "x: " << x << endl; // in the outside world - x remains 100
    x = 200;
    cout << lambda2() << endl;  // 104, as the 102 is saved in the lambda world.
    cout << "x: " << x << endl; // 200
}

Output:
102
x: 100
104
x: 200

case 3: This is the easiest case, as no more 2 lives of x. Now there is only one value for x and it's shared between the outside world and the lambda world.

{
    int x = 100;
    auto lambda3 = [&x]() mutable {
        x += 10;  // modify the argument, is allowed when mutable is used.
        return x;
    };
    cout << lambda3() << endl;  // 110
    cout << "x: " << x << endl; // 110
    x = 400;
    cout << lambda3() << endl;  // 410.
    cout << "x: " << x << endl; // 410
}

Output: 
110
x: 110
410
x: 410

Okoka

Answer 11

I also was wondering about it and the simplest explanation why [=] requires explicit mutable is in this example:

int main()
{
    int x {1};
    auto lbd = [=]() mutable { return x += 5; };
    printf("call1:%d\n", lbd());
    printf("call2:%d\n", lbd());
    return 0;
}

Output:

call1:6
call2:11

By words:

You can see that the x value is different at the second call (1 for the call1 and 6 for the call2).

1. A lambda object keeps a captured variable by value (has its own copy) in case of [=].
2. The lambda can be called several times.

And in general case we have to have the same value of the captured variable to have the same predictable behavior of the lambda based on the known captured value, not updated during the lambda work. That's why the default behavior assumed const (to predict changes of the lambda object members) and when a user is aware of consequences he takes this responsibility on himself with mutable.

Same with capturing by value. For my example:

auto lbd = [x]() mutable { return x += 5; };

Hehre

This becomes even more clear if a copy of the lambda is created, say auto lbd2 = lbd. x in lbd2 gets the x value of lbd at the time of the copy. – Pecker

Answer 12

There is now a proposal to alleviate the need for mutable in lambda declarations: n3424

Boman

Any information on what came of that? I personally think it's a bad idea, since the new "capture of arbitrary expressions" smooths out most of the pain points. – Overripe

@BenVoigt Yeah it seems like a change for change's sake. – Nix

@BenVoigt Although to be fair, I expect there are probably many C++ developers that don't know that mutable is even a keyword in C++. – Nix

Answer 13

This really mixes up two questions into one, and separating them makes it easier to provide answers:

1. Why is a lambda capturing by value that modifies a captured variable ill-formed?
2. Why are values captured by lambdas const by default?

Question #1 has a simple factual answer: Values captured by lambdas are const by default. In these cases:

int n = 0;
[=](){n = 10;}();
[n](){n = 10;}();
[val = n](){val = 10;}();

n or val are basically "members" of the lambdas of type const int. You cannot modify a member of a class or struct that has type const int, because it is const.

Question #2 is a question about language design, and the answer can be complicated or opinionated. Many possible answers have been given in the other answers here already, but many of them do not answer question #1, so I think they are not helpful for visitors who are only interested in how lambda capture works, not discuss language design principles and reasoning.

Theory