Using C++11 range-based for loop correctly in Qt

Question

According to this talk there is a certain pitfall when using C++11 range base for on Qt containers. Consider:

QList<MyStruct> list;

for(const MyStruct &item : list)
{
    //...
}

The pitfall, according to the talk, comes from the implicit sharing. Under the hood the ranged-based for gets the iterator from the container. But because the container is not const the iterator will be non-const and that is apparently enough for the container to detach.

When you control the lifetime of a container this is easy to fix, one just passes the const reference to the container to force it to use const_iterator and not to detach.

QList<MyStruct> list;
const Qlist<MyStruct> &constList = list;

for(const MyStruct &item : constList)
{
    //...
}

However what about for example containers as return values.

QList<MyStruct> foo() { //... }

void main()
{
    for(const MyStruct &item : foo())
    {
    }
}

What does happen here? Is the container still copied? Intuitively I would say it is so to avoid that this might need to be done?

QList<MyStruct> foo() { //... }

main()
{ 
    for(const MyStruct &item : const_cast<const QList<MyStruct>>(foo()))
    {
    }
}

I am not sure. I know it is a bit more verbose but I need this because I use ranged based for loops heavily on huge containers a lot so the talk kind of struck the right string with me.

So far I use a helper function to convert the container to the const reference but if there is a shorter/easier way to achieve the same I would like to hear it.

Lyophilize

Stop worrying about that. All Qt containers implements COW pattern. And in latest versions Qt team implements support of C++11, including move ctors. – Neumeyer

Btw, try const MyStruct& const item : foo() to iterate in const style. – Neumeyer

@SaZ I will try your suggestion. But regarding COW the Qt developer in the linked talk explicitly said that creating non-const iterator from a container means it detaches. It makes sense because otherwise they could not detect if you actually did use that iterator to change it, simply the fact you can is enough. – Lyophilize

i have literally never had a problem with just doing for(const auto& bla : blas) i dont see there could be a problem with this even – Corr

Shouldn't it be const QList<MyStruct> &constList = list; instead of Qlist<MyStruct> &constList = list; to get const iterators and prevent detach? If no, why not? – Bugloss

@Bugloss Yes of course, thanks for noticing that! – Lyophilize

Okay now it makes sense to me :) Now wouldn't QList<MyStruct> &constList = foo(); and then for(const MyStruct &item : constList) solve your initial problem? – Bugloss

@Bugloss No because that would also detach in the loop. – Lyophilize

I stumble over the talk mentioned above just a few days ago, so I'm fairly new to this whole issue... So my question is, why will it detach in this case: QList<MyStruct> &constList = foo(); when it won't detach in this case: const QList<MyStruct> &constList = list; ? – Bugloss

@Bugloss The talk described it, no? The issue is that internally Qt containers are COW or shared data. So whenever you do something that could potentially change that data your object will "detach" (make deep copy) of that shared data. This is "hidden" when you return by value (Qt does it a lot because it is cheap because that does not detach on itself). However when you instantiate a non-const iterator on such container it will imediately detach (perform deep copy). const& is fine as it cannot do that byt & can (and is not valid anyway as you cannot take reference to temporary). – Lyophilize

Yes, I already got that from the talk. But where detaches the container here? Let's say we do this (which is a little different from what I posted before): const QList<MyStruct> constList = foo(); So now we have const Container and no detach so far. Is that right? And now we pass the const Container to the for loop for(const MyStruct &item : constList). Since the Container is const it will use const iterators and also no detach. Or getting here something wrong? – Bugloss

@Bugloss That is correct. But if you copied the constList to another non-const list and iterated over that then it would detach again. – Lyophilize

Ok I get that. But when would a copy from the constList to a non-const list happen in the scenario I described above? I don't see where this should happen!?! – Bugloss

Answer 1

template<class T>
std::remove_reference_t<T> const& as_const(T&&t){return t;}

might help. An implicitly shared object returned an rvalue can implicitly detect write-shraring (and detatch) due to non-const iteration.

This gives you:

for(auto&&item : as_const(foo()))
{
}

which lets you iterate in a const way (and pretty clearly).

If you need reference lifetime extension to work, have 2 overloads:

template<class T>
T const as_const(T&&t){return std::forward<T>(t);}
template<class T>
T const& as_const(T&t){return t;}

But iterating over const rvalues and caring about it is often a design error: they are throw away copies, why does it matter if you edit them? And if you behave very differently based off const qualification, that will bite you elsewhere.

Shell

It turns out that one needs two overloads of the as_const, one taking l-value (T&) reference (as given) and the other for r-value (T&&) references (as is actually required in the example). Both have same return value and content of course. – Lyophilize

@Lyophilize oops. But note it can be done with one overload as above edit. – Shell

I have encountered a problem. When a function returns temporary container and it is passed through as_const it gets destroyed after the first element is evaluated instead of lasting for the entire range-loop. I believe the culprit is the as_const function that prolongs the lifetime of the container but only until it itself finishes because it returns only const reference instead of the object and standard defines lifetime of rvalue bound to const ref to be "until expression evaluates" which is as_const in this case and not the loop it seems. Any idea how to solve it? – Lyophilize

Is returning T const deliberate? If so, what's the benefit? – Mentally

@piotr reference lifetime extension wants a value to extend its life. as_const should return a const object, for many reasons (not the least the principle of least surprise). So, it returns a T const - a valur, and const. This also means then auto&& that binds in the for(:) loop generated code will be T const&&, and the const overloads of begin will be called, which is what the OP wanted. – Shell

The reason I want it for temporaries as well is was for completeness sake on one hand and because of implicit sharing (COW) in Qt on the other. Basically making shallow copies is fast and cheap but the moment you spawn non-const iterator you perform deep copy. Qt classes often return containers by value because it is cheap. But if you iterate over them in non-const way you do the deep copy. If you don't need it it would be a waste (and sometimes significant) to do a deep copy to perform const for-range loop... But maybe I understand it wrongly. :-) Thanks a lot for the forward trick anyhow! – Lyophilize

@Lyophilize Good point; this is a problem with Qt's design (it treats the constness of the view object as the constness of the held data: like treating T const* and T* const the same). But I guess you get what you pay for. :) I have run into the same mistake when I wrote my first views, and ran into similar problems and stopped doing it. The right way to distinguish between views of const and non-const data is to have different types, not const-qualification of the view. A T&const is not a T const&, nor should it be. – Shell

Newer Qt versions will have qAsConst, see doc-snapshots.qt.io/qt5-dev/qtglobal.html#qAsConst. – Henninger

Just for completeness, std::as_const was introduced in C++17 and it is equivalent to qAsConst. – Dupleix

as_const( foo() ) doesn't work, it is declared as void as_const(const _Tp&&) = delete; – Canoe

Answer 2

Qt has an implementation to resolve this, qAsConst (see https://doc.qt.io/qt-5/qtglobal.html#qAsConst). The documentation says that it is Qt's version of C++17's std::as_const().

Zavala

When i write something like qAsConst( getStringList() ) i get compilation error call to deleted function. In the header there is // prevent rvalue arguments:\nvoid qAsConst(const T &&) = delete; – Canoe

You probably have to store the result of the getStringList() call into a temp variable, then call qAsConst on that (or declare the temp as const directly?) – Zavala

If i store the result to a local variable, then the compiler no longer warns about detaching a Qt container so the problem probably no longer exists. Also, as you said, i can already declare it as const. So the existence of qAsConst seems kinda pointless. – Canoe