Why does std::optional not have a specialization for reference types?

Question

Why std::optional (std::experimental::optional in libc++ at the moment) does not have specialization for reference types (compared with boost::optional)?

I think it would be very useful option.

Is there some object with reference to maybe already existing object semantics in the standard library?

Odontoblast

Finally I conclude, that I can use std::optional< std::reference_wrapper< T > > for my purposes. – Odontoblast

Yep, that's what I did as well. Put it into template <typename T> using OptionalRef = std::optional<std::reference_wrapper<T>>; for readability. – Centrum

Possible duplicate of Why GCC rejects std::optional for references? – Cortes

Nice reading: Why Optional References Didn’t Make It In C++17 – Westleigh

Answer 1

When n3406 (revision #2 of the proposal) was discussed, some committee members were uncomfortable with optional references. In n3527 (revision #3), the authors decided to make optional references an auxiliary proposal, to increase the chances of getting optional values approved and put into what became C++14. While optional didn't quite make it into C++14 for various other reasons, the committee did not reject optional references and is free to add optional references in the future should someone propose it.

Amerson

I realize this is a really old answer, but I was wondering whether you might be able to provide some clarification on what those specific discomforts had been? Was (I guess better - is) it solely a question of whether assignment rebinds to reference a different object (as per boost implementation)? Were (are) there other concerns? – Howrah

@ShmuelLevine I'm not sure what the concern was at the meeting in question, but a huge concern is: how should int y = 2; int x = 1; std::optional<int&> x_ref {x}; x_ref = y; behave? Should x_ref now be a reference to y or should x now equal 2? – Gussiegussman

Answer 2

The main problem with std::optional <T&> is — what should optRef = obj do in the following case:

optional<T&> optRef;
…;
T obj {…};
optRef = obj; // <-- here!

Variants:

1. Always rebind — (&optRef)->~optional(); new (&optRef) optional<T&>(obj).
2. Assign through — *optRef = obj (UB when !optRef before).
3. Bind if empty, assign through otherwise — if (optRef) {do1;} else {do2;}.
4. No assignment operator — compile-time error "trying to use a deleted operator".

Pros of every variant:

1. Always rebind (chosen by boost::optional and n1878):

   • Consistency between the cases when !optRef and optRef.has_value() — post-condition &*optRef == &obj is always met.
   • Consistency with usual optional<T> in the following aspect: for usual optional<T>, if T::operator= is defined to act as destroying and constructing (and some argue that it must be nothing more than optimization for destroying-and-constructing), opt = … de facto acts similarly like (&opt)->~optional(); new (&opt) optional<T&>(obj).

2. Assign through:

   • Consistency with pure T& in the following aspect: for pure T&, ref = … assigns through (not rebinds the ref).
   • Consistency with usual optional<T> in the following aspect: for usual optional<T>, when opt.has_value(), opt = … is required to assign through, not to destroy-and-construct (see template <class U> optional<T>& optional<T>::operator=(U&& v) in n3672 and on cppreference.com).
   • Consistency with usual optional<T> in the following aspect: both haveoperator= defined at least somehow.

3. Bind if empty, assign through otherwise — I see no real benefits, IMHO this variant arises only when proponents of #1 argue with proponents of #2, however formally it's even more consistent with the letter of requirements for template <class U> optional<T>& optional<T>::operator=(U&& v) (but not with the spirit, IMHO).

4. No assignment operator (chosen by n3406):

   • Consistency with pure T& in the following aspect: pure T& doesn't allow to rebind itself.
   • No ambiguous behavior.

See also:

• Let’s Talk about std::optional<T&> and optional references.
• Why Optional References Didn’t Make It In C++17.

Engstrom

That fluentc++ link is incredible! Thank you – Aegyptus

which of 1. or 2. does std::optional<std::reference_wrapper<T>> end up doing? 1. right? – Allopath

@jwezorek, yes. But in that case, it's absolutely logically correct (and expected), because std::reference_wrapper<T> itself works in the way #1, i.e. always rebinds (unlike raw reference T&, which does #2, i.e. assigns through). In other words, std::optional<std::reference_wrapper<T>> behaves in that way not because std::optional implements #1 itself but because std::reference_wrapper implements #1 (while std::optional is kinda transparent in that aspect). – Engstrom

Answer 3

There is indeed something that has reference to maybe existing object semantics. It is called a (const) pointer. A plain old non-owning pointer. There are three differences between references and pointers:

1. Pointers can be null, references can not. This is exactly the difference you want to circumvent with std::optional.
2. Pointers can be redirected to point to something else. Make it const, and that difference disappears as well.
3. References need not be dereferenced by -> or *. This is pure syntactic sugar and possible because of 1. And the pointer syntax (dereferencing and convertible to bool) is exactly what std::optional provides for accessing the value and testing its presence.

Update: optional is a container for values. Like other containers (vector, for example) it is not designed to contain references. If you want an optional reference, use a pointer, or if you indeed need an interface with a similar syntax to std::optional, create a small (and trivial) wrapper for pointers.

Update2: As for the question why there is no such specialization: because the committee simply did opt it out. The rationale might be found somewhere in the papers. It possibly is because they considered pointers to be sufficient.

Kirstinkirstyn

Still, for generic programming you'd want std::optional<T> wo work even when T==U&. You explain quite well how std::optinal<U&> can be implemented with U* const but that's not really a help in generic programming. – Hercegovina

@Hercegovina I don't see why you would want that. optional is strictly for values. See my update to the answer. "generic programming" is not a blanco justification to have anything imaginable. If you happen to come across an occasion where you want to support optional values and optional references, use specialization to distingusih between optional and pointers or the wrapper class I mention. – Kirstinkirstyn

Update 1 is the key - optional is a value-type. Object are created inside it or not created at all, has_value means exactly that - has value (inside) or not, when you copy, you copy the object, etc. The issue with references is that they are not separate objects which leads to semantic problems making optional both a value and a handle type and the committee does not want to go there. – Combo

By today's standards I'd say this answer is wrong. If I see a pointer, I immediately start to wonder if it owns the object it refers to, if the object is dynamically allocated (and thus might be deleted at any moment) etc. I don't have to wonder re that about a reference nor about an optional. The reason is that pointers traditionally cover a lot of possible use cases, while references and optionals cover way less. They were designed for specific purposes, and when they are used, I know that they are used for those purposes. Hence, std::optional<T&> should be supported by the standard. – Womanly

By today's standards, a raw pointer should never own the referred object. References, optional or otherwise, can be bound to heap objects as well, so there's no difference at all regarding the dangers of dynamic object lifetimes. Granted, in old-school C++, a raw pointer could mean anything from owner to reference to array iterator. In today's code, all those cases should be handled differently. std::optional<T&> is heavily discussed currently and the semantics are not overly clear - AFAIK there's about a 50/50 split regarding whether assigning should rebind or assign the referred object. – Kirstinkirstyn

"By today's standards, a raw pointer should never own the referred object." Is an extremely narrow-minded and unrealistic POV. In a perfect world, all code would be up to "today's standards", but in the real world, we have to interoperate with older codebases because we can't reinvent the wheel every few years. Especially if working with any C libraries, raw pointers must represent ownership. – Coquille

@Coquille A raw point cannot represent ownership, ever. Regardless of whether we're talking about C or C++. You were always having to guess, and its neutral if we continue having to do so in the context of legacy code. – Feliciafeliciano

@Feliciafeliciano that completely misrepresents what my comment was in reference to. By definition a raw pointer cannot have semantic ownership, but in practical code it is absolutely essential that objects which have a lifetime (and therefore an owner) must sometimes interoperate with code that relies on raw pointers. In these scenarios, the pointer itself must carry the concept of ownership, or else your code is fundamentally broken and will fail in the event that the pointer outlives the pointed-to object. – Coquille

Except that is obviously not true. You might be referring to some kind of conventions in C APIs (though I can only think of counter-examples right now). Interface functions can document lifetime/ownership assumptions. Of course it's much easier to use expressive types that not only make the intent clear, but also allow checking or automatic lifetime guarantees. Basically, when you have to "interoperate with code that relies on raw pointers" you are by definition relying on ownership semantics being defined outside the code/types. – Feliciafeliciano

@Feliciafeliciano you're intentionally and maliciously misrepresenting the context of what I'm saying. Again. If you use any API that takes a raw pointer to an object, then that pointer must represent either unique or shared ownership of the object, OR undefined behavior may occur at any time. If a pointer carries only an observer view of an object, then that object may cease to exist at any time. If the API dictates that the pointed-to object must not cease to exist, then that is a contract of ownership, which is exactly and exclusively what was always being discussed here. – Coquille

Lifetime is not ownership. That is the source of our miscommunication. Lifetime and ownership are related, but very different concepts. I do not appreciate the way in which you have assumed malice in what could have been a normal attempt to clear up that misunderstanding. – Feliciafeliciano

@Feliciafeliciano ownership in OOP is defined as "managing the lifetime of an object". You can call it a miscommunication but you intentionally misrepresented the definition of terms to justify altering what I said. If you pass a raw pointer to any API, that pointer must be associated with ownership of the object or UB could occur at literally any time. Therefore, a well formed program dictates that raw pointers passed into any API as an object address (assumed non-null) must, by the definition of terms, be directly associated with ownership of said object. – Coquille

@Coquille "Owership [...] managing the lifetime of an object" That's how they are related. Still not the same thing. If you pass a raw pointer, the only requirement is that it points to something. Nothing needs to own it for that to be satisfied. Also, the raw pointers must by definition not be directly associated with ownership (which is one of the reasons why you can simply copy raw pointers or even set them to other values without modifying any aspect of ownership of any pointed-to object(s)). – Feliciafeliciano

@Feliciafeliciano you continue to misrepresent what I said. I never said that raw pointers carry syntactic ownership. By definition they can't. But if you dereference a pointer that does not point to a living object, then your program is malformed. A living object that does not have an owner (syntactic or conceptual) is leaked. If the raw pointer is not provided by an owner of a living object, then you cannot provide any guarantee that your program is well formed. Well formed programs cannot dereference raw pointers that do not have conceptual ownership of the pointed-to object. – Coquille

I wasn't talking about "syntactic ownership" (there literally is no such thing, see value categories). I was talking about ownership semantics. What you are talking about is ownership convention/beliefs. It's not semantics if it's just in your head. It's semantics if it describes behavior of the C++ abstract machine. So, you're the one muddying concepts. – Feliciafeliciano

Great answer, I realized that indeed I do not really need optional in my current task. – Epistasis

A raw pointer does not tell ANYTHING about ownership. It just is. It points to data. It is TECHNICALLY unknown "who" owns the data [and therefore who should deleted]. – Placable

Of course, you can "add" ownership by CONVENTION. It was mentioned that in C libs, if you got a raw pointer, you own it. In "modern C++", if you get a raw pointer, you don't own it. We can also document (by comments) each time there is a raw pointer, what has to be done. Not bad, not great. – Placable

The solution is a set of abstractions that make the conventions/comments unnecessary – the TECHNICAL solution. Whoever's got a UniquePtr, they own the object under the pointer. If you've got a reference, you don't own the object "behind" the reference. – Placable

Optional for "reference to sth" or "reference to null" IS needed to express the ownership -> that we do NOT own the object behind the reference, therefore we don't need to manage its life -> no matter if the object exists or not. – Placable

Answer 4

IMHO it is very okay to make std::optional<T&> available. However there is a subtle issue about templates. Template parameters can become tricky to deal with if there are references.

Just as the way we solved the problem of references in template parameters, we can use a std::reference_wrapper to circumvent the absence of std::optional<T&>. So now it becomes std::optional<std::reference_wrapper<T>>. However I recommend against this use because 1) it is way too verbose to both write the signature (trailing return type saves us a bit) and the use of it (we have to call std::reference_wrapper<T>::get() to get the real reference), and 2) most programmers have already been tortured by pointers so that it is like an instinctive reaction that when they receive a pointer they test first whether it is null so it is not quite much an issue now.

Aelber

template <typename T> using optional_ref = std::optional<std::reference_wrapper<T>> partially fixed the issue of the verbosity. – Goldman

The problem of raw pointers is not only about to check nullptr: Should I take ownership of that pointer returned by a function? Can I save in my object that pointer? While using optional_ref, you expect RAII and can apply all those how-to that we are used to follow all the days. – Krell

Answer 5

If I would hazard a guess, it would be because of this sentence in the specification of std::experimental::optional. (Section 5.2, p1)

A program that necessitates the instantiation of template optional for a reference type, or for possibly cv-qualified types in_place_t or nullopt_t is ill-formed.

Frosty

Maybe it is right, but I can't understand the background of cieted conclusion. – Odontoblast

@Orient that is standardese for "optional may not be used with references and these two special types" - in other words it is not designed to work with them. In fact, your question is another means to say "why does the standard contain that clause?" – Kirstinkirstyn

@ArneMertz You are right, but it is important to understand the intentions of cometee. – Odontoblast

Answer 6

I stumbled upon this several times and I finally decided to implement my solution that doesn't depend on boost. For reference types it disables assignment operator and doesn't allow for comparison of pointers or r-values. It is based on a similar work I did some time ago, and it uses nullptr instead of nullopt to signal absence of value. For this reason, the type is called nullable and compilation is disabled for pointer types (they have nullptr anyway). Please let me know if you find any obvious or any non-obvious problem with it.

#ifndef COMMON_NULLABLE_H
#define COMMON_NULLABLE_H
#pragma once

#include <cstddef>
#include <stdexcept>
#include <type_traits>

namespace COMMON_NAMESPACE
{
    class bad_nullable_access : public std::runtime_error
    {
    public:
        bad_nullable_access()
            : std::runtime_error("nullable object doesn't have a value") { }
    };

    /**
     * Alternative to std::optional that supports reference (but not pointer) types
     */
    template <typename T, typename = std::enable_if_t<!std::is_pointer<T>::value>>
    class nullable final
    {
    public:
        nullable()
            : m_hasValue(false), m_value{ } { }

        nullable(T value)
            : m_hasValue(true), m_value(std::move(value)) { }

        nullable(std::nullptr_t)
            : m_hasValue(false), m_value{ } { }

        nullable(const nullable& value) = default;

        nullable& operator=(const nullable& value) = default;

        nullable& operator=(T value)
        {
            m_hasValue = true;
            m_value = std::move(value);
            return *this;
        }

        nullable& operator=(std::nullptr_t)
        {
            m_hasValue = false;
            m_value = { };
            return *this;
        }

        const T& value() const
        {
            if (!m_hasValue)
                throw bad_nullable_access();

            return m_value;
        }

        T& value()
        {
            if (!m_hasValue)
                throw bad_nullable_access();

            return m_value;
        }

        bool has_value() const { return m_hasValue; }
        const T* operator->() const { return &m_value; }
        T* operator->() { return &m_value; }
        const T& operator*() const { return m_value; }
        T& operator*() { return m_value; }

    public:
        template <typename T2>
        friend bool operator==(const nullable<T2>& lhs, const nullable<T2>& rhs);

        template <typename T2>
        friend bool operator!=(const nullable<T2>& lhs, const nullable<T2>& rhs);

        template <typename T2>
        friend bool operator==(const nullable<std::decay_t<T2>>& lhs, const nullable<T2&>& rhs);

        template <typename T2>
        friend bool operator!=(const nullable<std::decay_t<T2>>& lhs, const nullable<T2&>& rhs);

        template <typename T2>
        friend bool operator==(const nullable<T2&>& lhs, const nullable<std::decay_t<T2>>& rhs);

        template <typename T2>
        friend bool operator!=(const nullable<T2&>& lhs, const nullable<std::decay_t<T2>>& rhs);

        template <typename T2>
        friend bool operator==(const nullable<T2>& lhs, const T2& rhs);

        template <typename T2>
        friend bool operator==(const T2& lhs, const nullable<T2>& rhs);

        template <typename T2>
        friend bool operator==(const nullable<T2>& lhs, std::nullptr_t);

        template <typename T2>
        friend bool operator!=(const nullable<T2>& lhs, const T2& rhs);

        template <typename T2>
        friend bool operator!=(const T2& lhs, const nullable<T2>& rhs);

        template <typename T2>
        friend bool operator==(std::nullptr_t, const nullable<T2>& rhs);

        template <typename T2>
        friend bool operator!=(const nullable<T2>& lhs, std::nullptr_t);

        template <typename T2>
        friend bool operator!=(std::nullptr_t, const nullable<T2>& rhs);

    private:
        bool m_hasValue;
        T m_value;
    };

    // Template spacialization for references
    template <typename T>
    class nullable<T&> final
    {
    public:
        nullable()
            : m_hasValue(false), m_value{ } { }

        nullable(T& value)
            : m_hasValue(true), m_value(&value) { }

        nullable(std::nullptr_t)
            : m_hasValue(false), m_value{ } { }

        nullable(const nullable& value) = default;

        nullable& operator=(const nullable& value) = default;

        const T& value() const
        {
            if (!m_hasValue)
                throw bad_nullable_access();

            return *m_value;
        }

        T& value()
        {
            if (!m_hasValue)
                throw bad_nullable_access();

            return *m_value;
        }

        bool has_value() const { return m_hasValue; }
        const T* operator->() const { return m_value; }
        T* operator->() { return m_value; }
        const T& operator*() const { return *m_value; }
        T& operator*() { return *m_value; }

    public:
        template <typename T2>
        friend bool operator==(const nullable<std::decay_t<T2>>& lhs, const nullable<T2&>& rhs);

        template <typename T2>
        friend bool operator!=(const nullable<std::decay_t<T2>>& lhs, const nullable<T2&>& rhs);

        template <typename T2>
        friend bool operator==(const nullable<T2&>& lhs, const nullable<std::decay_t<T2>>& rhs);

        template <typename T2>
        friend bool operator!=(const nullable<T2&>& lhs, const nullable<std::decay_t<T2>>& rhs);

        template <typename T2>
        friend bool operator==(const nullable<T2&>& lhs, const nullable<T2&>& rhs);

        template <typename T2>
        friend bool operator!=(const nullable<T2&>& lhs, const nullable<T2&>& rhs);

        template <typename T2>
        friend bool operator==(const nullable<T2&>& lhs, const std::decay_t<T2>& rhs);

        template <typename T2>
        friend bool operator!=(const nullable<T2&>& lhs, const std::decay_t<T2>& rhs);

        template <typename T2>
        friend bool operator==(const std::decay_t<T2>& lhs, const nullable<T2&>& rhs);

        template <typename T2>
        friend bool operator!=(const std::decay_t<T2>& lhs, const nullable<T2&>& rhs);

        template <typename T2>
        friend bool operator==(const nullable<T2>& lhs, std::nullptr_t);

        template <typename T2>
        friend bool operator==(std::nullptr_t, const nullable<T2>& rhs);

        template <typename T2>
        friend bool operator!=(const nullable<T2>& lhs, std::nullptr_t);

        template <typename T2>
        friend bool operator!=(std::nullptr_t, const nullable<T2>& rhs);

    private:
        bool m_hasValue;
        T* m_value;
    };

    template <typename T>
    using nullableref = nullable<T&>;

    template <typename T2>
    bool operator==(const nullable<T2>& lhs, const nullable<T2>& rhs)
    {
        if (lhs.m_hasValue != rhs.m_hasValue)
            return false;

        if (lhs.m_hasValue)
            return lhs.m_value == rhs.m_value;
        else
            return true;
    }

    template <typename T2>
    bool operator!=(const nullable<T2>& lhs, const nullable<T2>& rhs)
    {
        if (lhs.m_hasValue != rhs.m_hasValue)
            return true;

        if (lhs.m_hasValue)
            return lhs.m_value != rhs.m_value;
        else
            return false;
    }

    template <typename T2>
    bool operator==(const nullable<std::decay_t<T2>>& lhs, const nullable<T2&>& rhs)
    {
        if (lhs.m_hasValue != rhs.m_hasValue)
            return true;

        if (lhs.m_hasValue)
            return lhs.m_value != *rhs.m_value;
        else
            return false;
    }

    template <typename T2>
    bool operator!=(const nullable<std::decay_t<T2>>& lhs, const nullable<T2&>& rhs)
    {
        if (lhs.m_hasValue != rhs.m_hasValue)
            return true;

        if (lhs.m_hasValue)
            return lhs.m_value != *rhs.m_value;
        else
            return false;
    }

    template <typename T2>
    bool operator==(const nullable<T2&>& lhs, const nullable<std::decay_t<T2>>& rhs)
    {
        if (lhs.m_hasValue != rhs.m_hasValue)
            return false;

        if (lhs.m_hasValue)
            return *lhs.m_value == rhs.m_value;
        else
            return true;
    }

    template <typename T2>
    bool operator!=(const nullable<T2&>& lhs, const nullable<std::decay_t<T2>>& rhs)
    {
        if (lhs.m_hasValue != rhs.m_hasValue)
            return true;

        if (lhs.m_hasValue)
            return *lhs.m_value != rhs.m_value;
        else
            return false;
    }

    template <typename T2>
    bool operator==(const nullable<T2&>& lhs, const nullable<T2&>& rhs)
    {
        if (lhs.m_hasValue != rhs.m_hasValue)
            return false;

        if (lhs.m_hasValue)
            return *lhs.m_value == *rhs.m_value;
        else
            return true;
    }

    template <typename T2>
    bool operator!=(const nullable<T2&>& lhs, const nullable<T2&>& rhs)
    {
        if (lhs.m_hasValue != rhs.m_hasValue)
            return true;

        if (lhs.m_hasValue)
            return *lhs.m_value != *rhs.m_value;
        else
            return false;
    }

    template <typename T2>
    bool operator==(const nullable<T2&>& lhs, const std::decay_t<T2>& rhs)
    {
        if (!lhs.m_hasValue)
            return false;

        return *lhs.m_value == rhs;
    }

    template <typename T2>
    bool operator!=(const nullable<T2&>& lhs, const std::decay_t<T2>& rhs)
    {
        if (!lhs.m_hasValue)
            return true;

        return *lhs.m_value != rhs;
    }

    template <typename T2>
    bool operator==(const std::decay_t<T2>& lhs, const nullable<T2&>& rhs)
    {
        if (!rhs.m_hasValue)
            return false;

        return lhs == *rhs.m_value;
    }

    template <typename T2>
    bool operator!=(const std::decay_t<T2>& lhs, const nullable<T2&>& rhs)
    {
        if (!rhs.m_hasValue)
            return true;

        return lhs != *rhs.m_value;
    }

    template <typename T2>
    bool operator==(const nullable<T2>& lhs, const T2& rhs)
    {
        if (!lhs.m_hasValue)
            return false;

        return lhs.m_value == rhs;
    }

    template <typename T2>
    bool operator!=(const nullable<T2>& lhs, const T2& rhs)
    {
        if (!lhs.m_hasValue)
            return true;

        return lhs.m_value != rhs;
    }

    template <typename T2>
    bool operator==(const T2& lhs, const nullable<T2>& rhs)
    {
        if (!rhs.m_hasValue)
            return false;

        return lhs == rhs.m_value;
    }

    template <typename T2>
    bool operator!=(const T2& lhs, const nullable<T2>& rhs)
    {
        if (!rhs.m_hasValue)
            return true;

        return lhs != rhs.m_value;
    }

    template <typename T2>
    bool operator==(const nullable<T2>& lhs, std::nullptr_t)
    {
        return !lhs.m_hasValue;
    }

    template <typename T2>
    bool operator!=(const nullable<T2>& lhs, std::nullptr_t)
    {
        return lhs.m_hasValue;
    }

    template <typename T2>
    bool operator==(std::nullptr_t, const nullable<T2>& rhs)
    {
        return !rhs.m_hasValue;
    }

    template <typename T2>
    bool operator!=(std::nullptr_t, const nullable<T2>& rhs)
    {
        return rhs.m_hasValue;
    }
}

#endif // COMMON_NULLABLE_H

Friar